Jul 28 17:06:45.703720 ip-10-0-141-34 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 28 17:06:45.703740 ip-10-0-141-34 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 28 17:06:45.703751 ip-10-0-141-34 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 28 17:06:45.704168 ip-10-0-141-34 systemd[1]: Failed to start Kubernetes Kubelet. Jul 28 17:06:57.161762 ip-10-0-141-34 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 28 17:06:57.161780 ip-10-0-141-34 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot bcb909551835415eb8a3118ad4922eb2 -- Jul 28 17:08:59.170147 ip-10-0-141-34 systemd[1]: Starting Kubernetes Kubelet... Jul 28 17:08:59.534050 ip-10-0-141-34 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 17:08:59.534050 ip-10-0-141-34 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 28 17:08:59.534050 ip-10-0-141-34 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 17:08:59.534050 ip-10-0-141-34 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 28 17:08:59.534050 ip-10-0-141-34 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 17:08:59.534977 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.534853 2571 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 28 17:08:59.539276 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539257 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:08:59.539276 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539276 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:08:59.539345 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539281 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:08:59.539345 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539284 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:08:59.539345 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539289 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:08:59.539345 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539292 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:08:59.539345 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539295 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:08:59.539345 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539298 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:08:59.539345 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539301 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:08:59.539345 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539304 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:08:59.539345 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539307 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:08:59.539345 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539321 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:08:59.539345 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539324 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:08:59.539345 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539326 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:08:59.539345 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539329 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:08:59.539345 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539332 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:08:59.539345 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539334 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:08:59.539345 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539337 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:08:59.539345 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539341 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:08:59.539345 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539345 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:08:59.539345 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539348 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:08:59.539345 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539352 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:08:59.539875 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539355 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:08:59.539875 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539358 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:08:59.539875 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539361 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:08:59.539875 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539363 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:08:59.539875 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539366 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:08:59.539875 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539369 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:08:59.539875 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539371 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:08:59.539875 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539375 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:08:59.539875 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539379 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:08:59.539875 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539383 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:08:59.539875 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539387 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:08:59.539875 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539390 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:08:59.539875 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539392 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:08:59.539875 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539395 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:08:59.539875 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539397 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:08:59.539875 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539400 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:08:59.539875 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539403 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:08:59.539875 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539407 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:08:59.539875 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539409 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:08:59.540328 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539412 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:08:59.540328 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539414 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:08:59.540328 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539417 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:08:59.540328 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539425 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:08:59.540328 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539428 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:08:59.540328 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539430 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:08:59.540328 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539432 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:08:59.540328 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539435 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:08:59.540328 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539437 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:08:59.540328 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539440 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:08:59.540328 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539442 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:08:59.540328 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539445 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:08:59.540328 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539447 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:08:59.540328 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539449 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:08:59.540328 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539453 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:08:59.540328 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539456 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:08:59.540328 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539459 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:08:59.540328 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539462 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:08:59.540328 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539465 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:08:59.540328 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539467 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:08:59.540863 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539470 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:08:59.540863 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539472 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:08:59.540863 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539475 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:08:59.540863 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539477 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:08:59.540863 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539480 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:08:59.540863 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539482 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:08:59.540863 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539485 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:08:59.540863 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539487 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:08:59.540863 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539490 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:08:59.540863 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539492 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:08:59.540863 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539495 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:08:59.540863 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539497 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:08:59.540863 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539500 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:08:59.540863 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539502 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:08:59.540863 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539505 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:08:59.540863 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539507 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:08:59.540863 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539517 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:08:59.540863 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539520 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:08:59.540863 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539522 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:08:59.541333 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539525 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:08:59.541333 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539527 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:08:59.541333 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539530 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:08:59.541333 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539533 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:08:59.541333 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539535 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:08:59.541333 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539538 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:08:59.541333 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539541 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:08:59.541333 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539546 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:08:59.541333 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539548 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:08:59.541333 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539551 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:08:59.541333 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539554 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:08:59.541333 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539556 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:08:59.541333 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539558 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:08:59.541333 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539561 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:08:59.541333 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539563 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:08:59.541333 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539566 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:08:59.541333 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539568 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:08:59.541333 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539570 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:08:59.541333 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.539573 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:08:59.541333 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540323 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:08:59.541900 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540331 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:08:59.541900 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540334 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:08:59.541900 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540337 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:08:59.541900 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540340 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:08:59.541900 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540343 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:08:59.541900 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540345 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:08:59.541900 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540348 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:08:59.541900 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540351 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:08:59.541900 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540353 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:08:59.541900 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540356 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:08:59.541900 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540360 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:08:59.541900 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540363 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:08:59.541900 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540365 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:08:59.541900 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540368 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:08:59.541900 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540370 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:08:59.541900 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540373 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:08:59.541900 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540376 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:08:59.541900 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540378 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:08:59.541900 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540381 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:08:59.541900 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540385 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:08:59.542404 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540388 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:08:59.542404 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540391 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:08:59.542404 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540394 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:08:59.542404 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540396 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:08:59.542404 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540399 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:08:59.542404 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540401 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:08:59.542404 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540404 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:08:59.542404 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540406 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:08:59.542404 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540409 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:08:59.542404 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540413 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:08:59.542404 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540417 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:08:59.542404 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540420 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:08:59.542404 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540422 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:08:59.542404 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540425 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:08:59.542404 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540429 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:08:59.542404 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540433 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:08:59.542404 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540435 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:08:59.542404 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540438 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:08:59.542404 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540441 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:08:59.542934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540444 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:08:59.542934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540446 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:08:59.542934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540449 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:08:59.542934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540451 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:08:59.542934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540455 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:08:59.542934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540457 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:08:59.542934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540460 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:08:59.542934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540462 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:08:59.542934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540465 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:08:59.542934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540468 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:08:59.542934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540470 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:08:59.542934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540472 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:08:59.542934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540476 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:08:59.542934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540479 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:08:59.542934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540482 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:08:59.542934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540484 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:08:59.542934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540487 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:08:59.542934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540489 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:08:59.542934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540492 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:08:59.542934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540494 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:08:59.543493 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540497 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:08:59.543493 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540500 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:08:59.543493 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540502 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:08:59.543493 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540505 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:08:59.543493 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540507 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:08:59.543493 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540509 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:08:59.543493 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540512 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:08:59.543493 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540515 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:08:59.543493 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540517 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:08:59.543493 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540519 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:08:59.543493 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540522 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:08:59.543493 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540525 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:08:59.543493 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540527 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:08:59.543493 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540530 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:08:59.543493 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540532 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:08:59.543493 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540534 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:08:59.543493 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540537 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:08:59.543493 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540540 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:08:59.543493 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540542 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:08:59.543493 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540545 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:08:59.544056 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540548 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:08:59.544056 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540551 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:08:59.544056 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540554 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:08:59.544056 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540557 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:08:59.544056 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540560 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:08:59.544056 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540563 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:08:59.544056 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540565 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:08:59.544056 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540568 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:08:59.544056 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540570 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:08:59.544056 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540573 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:08:59.544056 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540576 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:08:59.544056 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540578 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:08:59.544056 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540581 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:08:59.544056 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540583 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:08:59.544056 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540586 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:08:59.544056 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540589 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:08:59.544056 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540591 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:08:59.544056 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540593 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:08:59.544056 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.540596 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:08:59.544056 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541053 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:08:59.544678 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541059 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:08:59.544678 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541062 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:08:59.544678 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541065 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:08:59.544678 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541069 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:08:59.544678 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541074 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:08:59.544678 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541077 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:08:59.544678 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541079 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:08:59.544678 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541082 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:08:59.544678 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541085 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:08:59.544678 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541088 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:08:59.544678 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541091 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:08:59.544678 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541094 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:08:59.544678 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541096 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:08:59.544678 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541098 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:08:59.544678 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541101 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:08:59.544678 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541104 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:08:59.544678 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541109 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:08:59.544678 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541112 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:08:59.544678 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541114 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:08:59.545240 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541117 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:08:59.545240 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541120 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:08:59.545240 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541122 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:08:59.545240 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541125 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:08:59.545240 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541128 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:08:59.545240 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541130 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:08:59.545240 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541133 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:08:59.545240 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541136 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:08:59.545240 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541139 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:08:59.545240 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541143 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:08:59.545240 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541146 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:08:59.545240 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541149 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:08:59.545240 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541152 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:08:59.545240 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541155 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:08:59.545240 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541158 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:08:59.545240 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541161 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:08:59.545240 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541164 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:08:59.545240 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541166 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:08:59.545240 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541169 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:08:59.545781 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541171 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:08:59.545781 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541174 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:08:59.545781 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541177 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:08:59.545781 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541179 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:08:59.545781 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541186 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:08:59.545781 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541188 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:08:59.545781 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541192 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:08:59.545781 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541194 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:08:59.545781 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541197 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:08:59.545781 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541199 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:08:59.545781 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541204 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:08:59.545781 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541206 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:08:59.545781 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541209 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:08:59.545781 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541211 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:08:59.545781 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541214 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:08:59.545781 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541216 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:08:59.545781 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541219 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:08:59.545781 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541222 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:08:59.545781 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541226 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:08:59.545781 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541229 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:08:59.546353 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541232 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:08:59.546353 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541234 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:08:59.546353 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541237 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:08:59.546353 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541240 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:08:59.546353 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541243 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:08:59.546353 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541245 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:08:59.546353 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541247 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:08:59.546353 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541250 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:08:59.546353 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541252 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:08:59.546353 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541255 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:08:59.546353 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541257 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:08:59.546353 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541260 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:08:59.546353 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541262 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:08:59.546353 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541265 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:08:59.546353 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541268 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:08:59.546353 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541270 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:08:59.546353 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541273 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:08:59.546353 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541276 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:08:59.546353 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541280 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:08:59.546353 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541282 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:08:59.546934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541285 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:08:59.546934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541287 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:08:59.546934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541292 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:08:59.546934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541294 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:08:59.546934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541297 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:08:59.546934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541300 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:08:59.546934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541302 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:08:59.546934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541305 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:08:59.546934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541307 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:08:59.546934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541310 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:08:59.546934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541313 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:08:59.546934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541315 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:08:59.546934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541317 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:08:59.546934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541320 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:08:59.546934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541323 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:08:59.546934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541325 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:08:59.546934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541328 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:08:59.546934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541332 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:08:59.546934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541335 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:08:59.546934 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.541338 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:08:59.547489 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541412 2571 flags.go:64] FLAG: --address="0.0.0.0" Jul 28 17:08:59.547489 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541420 2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 28 17:08:59.547489 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541427 2571 flags.go:64] FLAG: --anonymous-auth="true" Jul 28 17:08:59.547489 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541431 2571 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 28 17:08:59.547489 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541435 2571 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 28 17:08:59.547489 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541439 2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 28 17:08:59.547489 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541443 2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 28 17:08:59.547489 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541447 2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 28 17:08:59.547489 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541451 2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 28 17:08:59.547489 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541454 2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 28 17:08:59.547489 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541460 2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 28 17:08:59.547489 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541463 2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 28 17:08:59.547489 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541467 2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 28 17:08:59.547489 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541470 2571 flags.go:64] FLAG: --cgroup-root="" Jul 28 17:08:59.547489 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541474 2571 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 28 17:08:59.547489 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541477 2571 flags.go:64] FLAG: --client-ca-file="" Jul 28 17:08:59.547489 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541481 2571 flags.go:64] FLAG: --cloud-provider="external" Jul 28 17:08:59.547489 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541484 2571 flags.go:64] FLAG: --cluster-dns="[]" Jul 28 17:08:59.547489 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541489 2571 flags.go:64] FLAG: --cluster-domain="" Jul 28 17:08:59.547489 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541492 2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 28 17:08:59.547489 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541495 2571 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 28 17:08:59.547489 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541498 2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 28 17:08:59.547489 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541502 2571 flags.go:64] FLAG: --container-log-max-files="5" Jul 28 17:08:59.547489 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541506 2571 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541510 2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541513 2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541516 2571 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541519 2571 flags.go:64] FLAG: --contention-profiling="false" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541522 2571 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541527 2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541530 2571 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541534 2571 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541538 2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541541 2571 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541544 2571 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541547 2571 flags.go:64] FLAG: --enable-load-reader="false" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541550 2571 flags.go:64] FLAG: --enable-server="true" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541553 2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541558 2571 flags.go:64] FLAG: --event-burst="100" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541561 2571 flags.go:64] FLAG: --event-qps="50" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541564 2571 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541567 2571 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541570 2571 flags.go:64] FLAG: --eviction-hard="" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541575 2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541578 2571 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541581 2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541585 2571 flags.go:64] FLAG: --eviction-soft="" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541588 2571 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 28 17:08:59.548131 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541591 2571 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 28 17:08:59.548815 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541594 2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 28 17:08:59.548815 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541597 2571 flags.go:64] FLAG: --experimental-mounter-path="" Jul 28 17:08:59.548815 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541600 2571 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 28 17:08:59.548815 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541603 2571 flags.go:64] FLAG: --fail-swap-on="true" Jul 28 17:08:59.548815 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541606 2571 flags.go:64] FLAG: --feature-gates="" Jul 28 17:08:59.548815 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541610 2571 flags.go:64] FLAG: --file-check-frequency="20s" Jul 28 17:08:59.548815 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541613 2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 28 17:08:59.548815 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541617 2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 28 17:08:59.548815 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541620 2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 28 17:08:59.548815 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541623 2571 flags.go:64] FLAG: --healthz-port="10248" Jul 28 17:08:59.548815 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541626 2571 flags.go:64] FLAG: --help="false" Jul 28 17:08:59.548815 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541629 2571 flags.go:64] FLAG: --hostname-override="ip-10-0-141-34.ec2.internal" Jul 28 17:08:59.548815 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541636 2571 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 28 17:08:59.548815 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541639 2571 flags.go:64] FLAG: --http-check-frequency="20s" Jul 28 17:08:59.548815 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541642 2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 28 17:08:59.548815 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541645 2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 28 17:08:59.548815 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541649 2571 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 28 17:08:59.548815 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541652 2571 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 28 17:08:59.548815 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541655 2571 flags.go:64] FLAG: --image-service-endpoint="" Jul 28 17:08:59.548815 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541658 2571 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 28 17:08:59.548815 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541661 2571 flags.go:64] FLAG: --kube-api-burst="100" Jul 28 17:08:59.548815 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541664 2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 28 17:08:59.548815 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541667 2571 flags.go:64] FLAG: --kube-api-qps="50" Jul 28 17:08:59.548815 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541670 2571 flags.go:64] FLAG: --kube-reserved="" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541673 2571 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541676 2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541679 2571 flags.go:64] FLAG: --kubelet-cgroups="" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541689 2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541692 2571 flags.go:64] FLAG: --lock-file="" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541696 2571 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541699 2571 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541702 2571 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541713 2571 flags.go:64] FLAG: --log-json-split-stream="false" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541716 2571 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541719 2571 flags.go:64] FLAG: --log-text-split-stream="false" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541723 2571 flags.go:64] FLAG: --logging-format="text" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541725 2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541729 2571 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541732 2571 flags.go:64] FLAG: --manifest-url="" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541735 2571 flags.go:64] FLAG: --manifest-url-header="" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541740 2571 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541743 2571 flags.go:64] FLAG: --max-open-files="1000000" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541747 2571 flags.go:64] FLAG: --max-pods="110" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541750 2571 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541754 2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541769 2571 flags.go:64] FLAG: --memory-manager-policy="None" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541773 2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541776 2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 28 17:08:59.549422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541779 2571 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541782 2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541790 2571 flags.go:64] FLAG: --node-status-max-images="50" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541793 2571 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541796 2571 flags.go:64] FLAG: --oom-score-adj="-999" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541799 2571 flags.go:64] FLAG: --pod-cidr="" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541802 2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541809 2571 flags.go:64] FLAG: --pod-manifest-path="" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541812 2571 flags.go:64] FLAG: --pod-max-pids="-1" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541815 2571 flags.go:64] FLAG: --pods-per-core="0" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541818 2571 flags.go:64] FLAG: --port="10250" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541821 2571 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541830 2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02d1c7ba37a308bc3" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541834 2571 flags.go:64] FLAG: --qos-reserved="" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541838 2571 flags.go:64] FLAG: --read-only-port="10255" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541842 2571 flags.go:64] FLAG: --register-node="true" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541845 2571 flags.go:64] FLAG: --register-with-taints="" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541849 2571 flags.go:64] FLAG: --registry-burst="10" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541852 2571 flags.go:64] FLAG: --registry-qps="5" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541855 2571 flags.go:64] FLAG: --reserved-cpus="" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541858 2571 flags.go:64] FLAG: --reserved-memory="" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541861 2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541864 2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541867 2571 flags.go:64] FLAG: --rotate-certificates="false" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541870 2571 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 28 17:08:59.550081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541873 2571 flags.go:64] FLAG: --runonce="false" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541876 2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541879 2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541884 2571 flags.go:64] FLAG: --seccomp-default="false" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541887 2571 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541890 2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541893 2571 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541896 2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541900 2571 flags.go:64] FLAG: --storage-driver-password="root" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541903 2571 flags.go:64] FLAG: --storage-driver-secure="false" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541906 2571 flags.go:64] FLAG: --storage-driver-table="stats" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541909 2571 flags.go:64] FLAG: --storage-driver-user="root" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541912 2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541916 2571 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541919 2571 flags.go:64] FLAG: --system-cgroups="" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541922 2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541927 2571 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541930 2571 flags.go:64] FLAG: --tls-cert-file="" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541933 2571 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541939 2571 flags.go:64] FLAG: --tls-min-version="" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541942 2571 flags.go:64] FLAG: --tls-private-key-file="" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541946 2571 flags.go:64] FLAG: --topology-manager-policy="none" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541949 2571 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541953 2571 flags.go:64] FLAG: --topology-manager-scope="container" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541956 2571 flags.go:64] FLAG: --v="2" Jul 28 17:08:59.550701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541961 2571 flags.go:64] FLAG: --version="false" Jul 28 17:08:59.551338 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541965 2571 flags.go:64] FLAG: --vmodule="" Jul 28 17:08:59.551338 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541969 2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 28 17:08:59.551338 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.541972 2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 28 17:08:59.551338 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542093 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:08:59.551338 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542097 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:08:59.551338 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542100 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:08:59.551338 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542103 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:08:59.551338 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542106 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:08:59.551338 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542108 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:08:59.551338 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542112 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:08:59.551338 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542116 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:08:59.551338 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542120 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:08:59.551338 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542123 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:08:59.551338 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542126 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:08:59.551338 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542129 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:08:59.551338 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542132 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:08:59.551338 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542135 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:08:59.551338 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542138 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:08:59.551338 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542141 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:08:59.551338 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542143 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:08:59.551880 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542146 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:08:59.551880 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542148 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:08:59.551880 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542151 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:08:59.551880 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542154 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:08:59.551880 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542157 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:08:59.551880 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542160 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:08:59.551880 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542164 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:08:59.551880 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542169 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:08:59.551880 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542172 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:08:59.551880 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542175 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:08:59.551880 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542178 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:08:59.551880 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542181 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:08:59.551880 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542183 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:08:59.551880 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542186 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:08:59.551880 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542188 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:08:59.551880 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542191 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:08:59.551880 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542193 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:08:59.551880 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542196 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:08:59.551880 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542198 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:08:59.552364 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542201 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:08:59.552364 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542204 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:08:59.552364 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542208 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:08:59.552364 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542210 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:08:59.552364 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542213 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:08:59.552364 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542215 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:08:59.552364 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542218 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:08:59.552364 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542221 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:08:59.552364 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542223 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:08:59.552364 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542226 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:08:59.552364 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542228 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:08:59.552364 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542231 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:08:59.552364 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542233 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:08:59.552364 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542236 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:08:59.552364 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542238 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:08:59.552364 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542241 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:08:59.552364 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542243 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:08:59.552364 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542246 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:08:59.552364 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542248 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:08:59.552364 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542251 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:08:59.552942 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542255 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:08:59.552942 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542258 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:08:59.552942 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542261 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:08:59.552942 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542263 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:08:59.552942 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542266 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:08:59.552942 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542268 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:08:59.552942 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542271 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:08:59.552942 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542274 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:08:59.552942 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542276 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:08:59.552942 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542280 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:08:59.552942 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542283 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:08:59.552942 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542286 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:08:59.552942 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542290 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:08:59.552942 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542293 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:08:59.552942 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542297 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:08:59.552942 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542300 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:08:59.552942 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542302 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:08:59.552942 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542305 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:08:59.552942 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542308 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:08:59.552942 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542311 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:08:59.553448 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542313 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:08:59.553448 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542316 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:08:59.553448 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542319 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:08:59.553448 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542322 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:08:59.553448 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542324 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:08:59.553448 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542327 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:08:59.553448 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542329 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:08:59.553448 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542332 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:08:59.553448 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542334 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:08:59.553448 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542337 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:08:59.553448 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542339 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:08:59.553448 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542342 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:08:59.553448 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542345 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:08:59.553448 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542348 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:08:59.553448 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542351 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:08:59.553448 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542354 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:08:59.553448 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542356 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:08:59.553448 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542359 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:08:59.553448 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542361 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:08:59.553448 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542364 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:08:59.553964 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542366 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:08:59.553964 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542369 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:08:59.553964 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.542371 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:08:59.553964 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.542378 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 17:08:59.553964 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.550008 2571 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 28 17:08:59.553964 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.550030 2571 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 28 17:08:59.553964 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.550726 2571 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 28 17:08:59.553964 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.550738 2571 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 28 17:08:59.553964 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550826 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:08:59.553964 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550832 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:08:59.553964 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550835 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:08:59.553964 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550839 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:08:59.553964 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550842 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:08:59.553964 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550845 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:08:59.553964 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550848 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:08:59.554342 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550851 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:08:59.554342 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550853 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:08:59.554342 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550856 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:08:59.554342 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550859 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:08:59.554342 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550861 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:08:59.554342 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550864 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:08:59.554342 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550867 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:08:59.554342 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550870 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:08:59.554342 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550873 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:08:59.554342 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550875 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:08:59.554342 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550878 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:08:59.554342 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550881 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:08:59.554342 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550883 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:08:59.554342 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550886 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:08:59.554342 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550889 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:08:59.554342 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550891 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:08:59.554342 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550894 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:08:59.554342 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550896 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:08:59.554342 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550899 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:08:59.554832 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550901 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:08:59.554832 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550905 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:08:59.554832 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550910 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:08:59.554832 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550921 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:08:59.554832 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550924 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:08:59.554832 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550926 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:08:59.554832 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550929 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:08:59.554832 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550932 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:08:59.554832 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550935 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:08:59.554832 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550937 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:08:59.554832 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550940 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:08:59.554832 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550942 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:08:59.554832 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550945 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:08:59.554832 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550947 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:08:59.554832 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550950 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:08:59.554832 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550952 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:08:59.554832 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550955 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:08:59.554832 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550957 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:08:59.554832 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550960 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:08:59.554832 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550963 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:08:59.555321 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550965 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:08:59.555321 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550968 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:08:59.555321 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550970 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:08:59.555321 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550973 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:08:59.555321 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550975 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:08:59.555321 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550978 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:08:59.555321 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550980 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:08:59.555321 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550983 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:08:59.555321 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550985 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:08:59.555321 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550988 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:08:59.555321 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550991 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:08:59.555321 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550993 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:08:59.555321 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550995 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:08:59.555321 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.550998 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:08:59.555321 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551000 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:08:59.555321 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551004 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:08:59.555321 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551013 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:08:59.555321 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551016 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:08:59.555321 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551019 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:08:59.555321 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551021 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:08:59.555825 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551024 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:08:59.555825 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551027 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:08:59.555825 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551029 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:08:59.555825 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551033 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:08:59.555825 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551037 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:08:59.555825 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551040 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:08:59.555825 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551043 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:08:59.555825 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551046 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:08:59.555825 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551048 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:08:59.555825 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551051 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:08:59.555825 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551054 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:08:59.555825 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551056 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:08:59.555825 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551059 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:08:59.555825 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551061 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:08:59.555825 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551064 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:08:59.555825 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551067 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:08:59.555825 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551069 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:08:59.555825 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551072 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:08:59.555825 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551074 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:08:59.556315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551077 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:08:59.556315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551080 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:08:59.556315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551082 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:08:59.556315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551085 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:08:59.556315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551087 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:08:59.556315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551090 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:08:59.556315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551092 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:08:59.556315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551095 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:08:59.556315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551098 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:08:59.556315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551102 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:08:59.556315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551111 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:08:59.556315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551114 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:08:59.556315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551117 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:08:59.556315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551120 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:08:59.556315 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.551126 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 17:08:59.556701 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551281 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:08:59.556701 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551286 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:08:59.556701 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551289 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:08:59.556701 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551291 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:08:59.556701 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551294 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:08:59.556701 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551298 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:08:59.556701 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551302 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:08:59.556701 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551305 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:08:59.556701 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551308 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:08:59.556701 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551310 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:08:59.556701 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551313 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:08:59.556701 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551316 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:08:59.556701 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551318 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:08:59.556701 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551320 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:08:59.556701 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551323 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:08:59.556701 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551325 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:08:59.556701 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551328 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:08:59.556701 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551330 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:08:59.556701 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551333 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:08:59.557190 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551335 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:08:59.557190 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551338 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:08:59.557190 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551340 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:08:59.557190 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551343 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:08:59.557190 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551346 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:08:59.557190 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551349 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:08:59.557190 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551352 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:08:59.557190 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551355 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:08:59.557190 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551359 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:08:59.557190 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551368 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:08:59.557190 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551372 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:08:59.557190 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551375 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:08:59.557190 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551377 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:08:59.557190 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551380 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:08:59.557190 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551383 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:08:59.557190 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551385 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:08:59.557190 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551388 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:08:59.557190 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551390 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:08:59.557190 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551393 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:08:59.557190 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551395 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:08:59.557709 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551398 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:08:59.557709 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551400 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:08:59.557709 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551403 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:08:59.557709 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551405 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:08:59.557709 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551408 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:08:59.557709 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551410 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:08:59.557709 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551413 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:08:59.557709 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551415 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:08:59.557709 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551418 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:08:59.557709 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551420 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:08:59.557709 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551423 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:08:59.557709 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551425 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:08:59.557709 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551428 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:08:59.557709 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551430 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:08:59.557709 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551432 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:08:59.557709 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551435 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:08:59.557709 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551438 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:08:59.557709 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551440 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:08:59.557709 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551443 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:08:59.557709 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551446 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:08:59.558315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551449 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:08:59.558315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551451 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:08:59.558315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551459 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:08:59.558315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551462 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:08:59.558315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551465 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:08:59.558315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551467 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:08:59.558315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551470 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:08:59.558315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551472 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:08:59.558315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551475 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:08:59.558315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551477 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:08:59.558315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551479 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:08:59.558315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551482 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:08:59.558315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551484 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:08:59.558315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551487 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:08:59.558315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551489 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:08:59.558315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551492 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:08:59.558315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551494 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:08:59.558315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551497 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:08:59.558315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551499 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:08:59.558315 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551502 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:08:59.558822 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551504 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:08:59.558822 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551507 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:08:59.558822 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551510 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:08:59.558822 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551512 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:08:59.558822 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551515 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:08:59.558822 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551517 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:08:59.558822 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551519 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:08:59.558822 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551522 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:08:59.558822 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551524 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:08:59.558822 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551526 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:08:59.558822 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551529 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:08:59.558822 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551534 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:08:59.558822 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551537 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:08:59.558822 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551541 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:08:59.558822 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551545 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:08:59.558822 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551553 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:08:59.558822 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551556 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:08:59.558822 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551559 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:08:59.558822 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551562 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:08:59.559281 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:08:59.551565 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:08:59.559281 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.551571 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 17:08:59.559281 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.551736 2571 server.go:962] "Client rotation is on, will bootstrap in background" Jul 28 17:08:59.559281 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.553938 2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 28 17:08:59.559281 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.555561 2571 server.go:1019] "Starting client certificate rotation" Jul 28 17:08:59.559281 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.555674 2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 17:08:59.559281 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.555726 2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 17:08:59.559281 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.558786 2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 17:08:59.562546 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.562525 2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 17:08:59.571735 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.571706 2571 log.go:25] "Validated CRI v1 runtime API" Jul 28 17:08:59.573686 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.573665 2571 log.go:25] "Validated CRI v1 image API" Jul 28 17:08:59.575665 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.575646 2571 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 28 17:08:59.578957 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.578926 2571 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8db8f83b-1a10-4e11-894b-737240569334:/dev/nvme0n1p3 ce147553-8d27-49c4-9c11-e113043cef53:/dev/nvme0n1p4] Jul 28 17:08:59.579068 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.578954 2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 28 17:08:59.584050 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.583936 2571 manager.go:216] Machine: {Timestamp:2026-07-28 17:08:59.582882678 +0000 UTC m=+0.302710375 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100000 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2804306fcb9ca083f19edcb56a40c3 SystemUUID:ec280430-6fcb-9ca0-83f1-9edcb56a40c3 BootID:bcb90955-1835-415e-b8a3-118ad4922eb2 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a7:e6:d5:f5:4b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a7:e6:d5:f5:4b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:32:d2:27:e8:db:78 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 28 17:08:59.584050 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.584046 2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 28 17:08:59.584188 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.584177 2571 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 28 17:08:59.586682 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.586645 2571 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 28 17:08:59.586848 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.586683 2571 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-34.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 28 17:08:59.587403 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.587387 2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 17:08:59.587787 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.587774 2571 topology_manager.go:138] "Creating topology manager with none policy" Jul 28 17:08:59.587822 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.587789 2571 container_manager_linux.go:309] "Creating device plugin manager" Jul 28 17:08:59.587822 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.587797 2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 17:08:59.588334 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.588321 2571 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 17:08:59.589103 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.589092 2571 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 28 17:08:59.590681 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.590660 2571 state_mem.go:36] "Initialized new in-memory state store" Jul 28 17:08:59.590833 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.590822 2571 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 28 17:08:59.590868 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.590852 2571 kubelet.go:486] "Attempting to sync node with API server" Jul 28 17:08:59.590868 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.590863 2571 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 28 17:08:59.590919 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.590874 2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 28 17:08:59.590919 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.590883 2571 kubelet.go:398] "Adding apiserver pod source" Jul 28 17:08:59.590919 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.590892 2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 28 17:08:59.591273 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.591261 2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 17:08:59.591315 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.591286 2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 17:08:59.593321 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.593306 2571 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 28 17:08:59.595498 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.595478 2571 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 28 17:08:59.595609 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.595517 2571 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 28 17:08:59.597568 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.597539 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 28 17:08:59.597647 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.597571 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 28 17:08:59.597647 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.597581 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 28 17:08:59.597647 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.597590 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 28 17:08:59.597647 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.597603 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 28 17:08:59.597647 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.597611 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 28 17:08:59.597647 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.597622 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 28 17:08:59.597647 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.597630 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 28 17:08:59.597647 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.597639 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 28 17:08:59.597647 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.597649 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 28 17:08:59.598035 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.597662 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 28 17:08:59.598035 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.597669 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 28 17:08:59.598035 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.597699 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 28 17:08:59.598035 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.597707 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 28 17:08:59.599831 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.599809 2571 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-34.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 28 17:08:59.600277 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:08:59.600253 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-34.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 28 17:08:59.600315 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:08:59.600252 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 28 17:08:59.601003 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.600984 2571 server.go:1268] "Started kubelet" Jul 28 17:08:59.601112 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.601060 2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 28 17:08:59.601211 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.601125 2571 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 28 17:08:59.601211 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.601192 2571 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 28 17:08:59.601819 ip-10-0-141-34 systemd[1]: Started Kubernetes Kubelet. Jul 28 17:08:59.602580 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.602500 2571 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 28 17:08:59.603047 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.603025 2571 server.go:310] "Adding debug handlers to kubelet server" Jul 28 17:08:59.605390 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:08:59.605369 2571 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 28 17:08:59.606842 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.606828 2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p7r2t" Jul 28 17:08:59.608567 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.608550 2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 28 17:08:59.608567 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.608558 2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 28 17:08:59.608810 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:08:59.608725 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-34.ec2.internal\" not found" Jul 28 17:08:59.609071 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:08:59.607671 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-34.ec2.internal.18c6832edd25ccba default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-34.ec2.internal,UID:ip-10-0-141-34.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-34.ec2.internal,},FirstTimestamp:2026-07-28 17:08:59.600956602 +0000 UTC m=+0.320784300,LastTimestamp:2026-07-28 17:08:59.600956602 +0000 UTC m=+0.320784300,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-34.ec2.internal,}" Jul 28 17:08:59.609164 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.609148 2571 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 28 17:08:59.609225 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.609153 2571 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 28 17:08:59.609298 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.609285 2571 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 28 17:08:59.609384 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.609344 2571 factory.go:55] Registering systemd factory Jul 28 17:08:59.609384 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.609359 2571 factory.go:223] Registration of the systemd container factory successfully Jul 28 17:08:59.609384 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.609364 2571 reconstruct.go:97] "Volume reconstruction finished" Jul 28 17:08:59.609384 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.609371 2571 reconciler.go:29] "Reconciler: start to sync state" Jul 28 17:08:59.609622 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.609609 2571 factory.go:153] Registering CRI-O factory Jul 28 17:08:59.609668 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.609625 2571 factory.go:223] Registration of the crio container factory successfully Jul 28 17:08:59.609712 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.609671 2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 28 17:08:59.609712 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.609693 2571 factory.go:103] Registering Raw factory Jul 28 17:08:59.609712 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.609706 2571 manager.go:1196] Started watching for new ooms in manager Jul 28 17:08:59.610178 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.610166 2571 manager.go:349] Starting recovery of all containers Jul 28 17:08:59.614034 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.614009 2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p7r2t" Jul 28 17:08:59.614376 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:08:59.614342 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 28 17:08:59.614471 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:08:59.614442 2571 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-34.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 28 17:08:59.621514 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.619255 2571 manager.go:354] Recovery completed Jul 28 17:08:59.624934 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.624919 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:08:59.626624 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.626608 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-34.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:08:59.626711 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.626641 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-34.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:08:59.626711 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.626651 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-34.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:08:59.627248 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.627232 2571 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 28 17:08:59.627248 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.627243 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:08:59.627329 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.627245 2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 28 17:08:59.627329 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.627306 2571 state_mem.go:36] "Initialized new in-memory state store" Jul 28 17:08:59.627926 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.627909 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-34.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:08:59.628007 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.627939 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-34.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:08:59.628007 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.627954 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-34.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:08:59.628871 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.628859 2571 policy_none.go:49] "None policy: Start" Jul 28 17:08:59.628945 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.628877 2571 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 28 17:08:59.628945 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.628891 2571 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 28 17:08:59.628945 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:08:59.628864 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-34.ec2.internal.18c6832edead7b45 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-34.ec2.internal,UID:ip-10-0-141-34.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-34.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-34.ec2.internal,},FirstTimestamp:2026-07-28 17:08:59.626625861 +0000 UTC m=+0.346453568,LastTimestamp:2026-07-28 17:08:59.626625861 +0000 UTC m=+0.346453568,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-34.ec2.internal,}" Jul 28 17:08:59.630070 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.630058 2571 policy_none.go:47] "Start" Jul 28 17:08:59.671310 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.671296 2571 manager.go:337] "Starting Device Plugin manager" Jul 28 17:08:59.671390 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:08:59.671334 2571 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 28 17:08:59.671390 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.671346 2571 server.go:87] "Starting device plugin registration server" Jul 28 17:08:59.671625 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.671609 2571 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 28 17:08:59.671722 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.671625 2571 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 28 17:08:59.671722 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.671704 2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 28 17:08:59.671827 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.671797 2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 28 17:08:59.671827 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.671807 2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 28 17:08:59.672303 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:08:59.672286 2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 28 17:08:59.672364 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:08:59.672340 2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-34.ec2.internal\" not found" Jul 28 17:08:59.771992 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.771960 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:08:59.772942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.772926 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-34.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:08:59.773000 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.772951 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-34.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:08:59.773000 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.772962 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-34.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:08:59.773000 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.772988 2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-34.ec2.internal" Jul 28 17:08:59.777806 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.777772 2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 28 17:08:59.778973 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.778957 2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 28 17:08:59.779036 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.778991 2571 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 28 17:08:59.779036 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.779009 2571 kubelet.go:2448] "Starting kubelet main sync loop" Jul 28 17:08:59.779119 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:08:59.779040 2571 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 28 17:08:59.779698 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.779681 2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-34.ec2.internal" Jul 28 17:08:59.779792 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:08:59.779715 2571 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-34.ec2.internal\": node \"ip-10-0-141-34.ec2.internal\" not found" Jul 28 17:08:59.781780 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.781748 2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:08:59.799803 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:08:59.799693 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-34.ec2.internal\" not found" Jul 28 17:08:59.879271 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.879211 2571 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-141-34.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-34.ec2.internal"] Jul 28 17:08:59.879450 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.879337 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:08:59.880316 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.880297 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-34.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:08:59.880433 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.880330 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-34.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:08:59.880433 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.880345 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-34.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:08:59.881526 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.881512 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:08:59.881618 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.881602 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-34.ec2.internal" Jul 28 17:08:59.881659 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.881637 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:08:59.882161 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.882146 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-34.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:08:59.882237 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.882171 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-34.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:08:59.882237 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.882180 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-34.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:08:59.882333 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.882249 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-34.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:08:59.882333 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.882279 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-34.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:08:59.882333 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.882292 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-34.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:08:59.883405 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.883387 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-34.ec2.internal" Jul 28 17:08:59.883486 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.883413 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:08:59.884121 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.884106 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-34.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:08:59.884198 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.884133 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-34.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:08:59.884198 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.884143 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-34.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:08:59.896393 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:08:59.896373 2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-34.ec2.internal\" not found" node="ip-10-0-141-34.ec2.internal" Jul 28 17:08:59.900546 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:08:59.900527 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-34.ec2.internal\" not found" Jul 28 17:08:59.900638 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:08:59.900614 2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-34.ec2.internal\" not found" node="ip-10-0-141-34.ec2.internal" Jul 28 17:08:59.911195 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.911170 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/220364fcdea4bfcb02dba704a736fdd7-config\") pod \"kube-apiserver-proxy-ip-10-0-141-34.ec2.internal\" (UID: \"220364fcdea4bfcb02dba704a736fdd7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-34.ec2.internal" Jul 28 17:08:59.911277 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.911198 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/04fe575350fb222dd4149f78daabf09c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-34.ec2.internal\" (UID: \"04fe575350fb222dd4149f78daabf09c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-34.ec2.internal" Jul 28 17:08:59.911277 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:08:59.911217 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/04fe575350fb222dd4149f78daabf09c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-34.ec2.internal\" (UID: \"04fe575350fb222dd4149f78daabf09c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-34.ec2.internal" Jul 28 17:09:00.001655 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:00.001624 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-34.ec2.internal\" not found" Jul 28 17:09:00.011444 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:00.011416 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/04fe575350fb222dd4149f78daabf09c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-34.ec2.internal\" (UID: \"04fe575350fb222dd4149f78daabf09c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-34.ec2.internal" Jul 28 17:09:00.011499 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:00.011446 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/220364fcdea4bfcb02dba704a736fdd7-config\") pod \"kube-apiserver-proxy-ip-10-0-141-34.ec2.internal\" (UID: \"220364fcdea4bfcb02dba704a736fdd7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-34.ec2.internal" Jul 28 17:09:00.011499 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:00.011462 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/04fe575350fb222dd4149f78daabf09c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-34.ec2.internal\" (UID: \"04fe575350fb222dd4149f78daabf09c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-34.ec2.internal" Jul 28 17:09:00.011556 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:00.011503 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/04fe575350fb222dd4149f78daabf09c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-34.ec2.internal\" (UID: \"04fe575350fb222dd4149f78daabf09c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-34.ec2.internal" Jul 28 17:09:00.011556 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:00.011542 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/04fe575350fb222dd4149f78daabf09c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-34.ec2.internal\" (UID: \"04fe575350fb222dd4149f78daabf09c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-34.ec2.internal" Jul 28 17:09:00.011614 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:00.011566 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/220364fcdea4bfcb02dba704a736fdd7-config\") pod \"kube-apiserver-proxy-ip-10-0-141-34.ec2.internal\" (UID: \"220364fcdea4bfcb02dba704a736fdd7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-34.ec2.internal" Jul 28 17:09:00.101809 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:00.101713 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-34.ec2.internal\" not found" Jul 28 17:09:00.199202 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:00.199178 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-34.ec2.internal" Jul 28 17:09:00.202814 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:00.202732 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-34.ec2.internal\" not found" Jul 28 17:09:00.203818 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:00.203802 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-34.ec2.internal" Jul 28 17:09:00.303707 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:00.303664 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-34.ec2.internal\" not found" Jul 28 17:09:00.404245 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:00.404166 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-34.ec2.internal\" not found" Jul 28 17:09:00.504573 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:00.504547 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-34.ec2.internal\" not found" Jul 28 17:09:00.554983 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:00.554953 2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 28 17:09:00.555384 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:00.555099 2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 28 17:09:00.605027 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:00.605000 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-34.ec2.internal\" not found" Jul 28 17:09:00.622198 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:00.622146 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 17:03:59 +0000 UTC" deadline="2028-01-06 03:31:40.656325262 +0000 UTC" Jul 28 17:09:00.622198 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:00.622190 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12634h22m40.034139172s" Jul 28 17:09:00.663135 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:09:00.663071 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04fe575350fb222dd4149f78daabf09c.slice/crio-7b2a5e0ba3f6ce62d0a59dfc241d88d1bac126b40ac0b987381d59b34273320f WatchSource:0}: Error finding container 7b2a5e0ba3f6ce62d0a59dfc241d88d1bac126b40ac0b987381d59b34273320f: Status 404 returned error can't find the container with id 7b2a5e0ba3f6ce62d0a59dfc241d88d1bac126b40ac0b987381d59b34273320f Jul 28 17:09:00.669307 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:00.669289 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 17:09:00.676542 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:09:00.676517 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod220364fcdea4bfcb02dba704a736fdd7.slice/crio-5f9c906998da10768a7fffe8ec4700d381d1232b672dec915e63044afee48b71 WatchSource:0}: Error finding container 5f9c906998da10768a7fffe8ec4700d381d1232b672dec915e63044afee48b71: Status 404 returned error can't find the container with id 5f9c906998da10768a7fffe8ec4700d381d1232b672dec915e63044afee48b71 Jul 28 17:09:00.705582 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:00.705530 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-34.ec2.internal\" not found" Jul 28 17:09:00.782451 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:00.782408 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-34.ec2.internal" event={"ID":"220364fcdea4bfcb02dba704a736fdd7","Type":"ContainerStarted","Data":"5f9c906998da10768a7fffe8ec4700d381d1232b672dec915e63044afee48b71"} Jul 28 17:09:00.783351 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:00.783328 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-34.ec2.internal" event={"ID":"04fe575350fb222dd4149f78daabf09c","Type":"ContainerStarted","Data":"7b2a5e0ba3f6ce62d0a59dfc241d88d1bac126b40ac0b987381d59b34273320f"} Jul 28 17:09:00.805178 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:00.805145 2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:09:00.806375 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:00.806356 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-34.ec2.internal\" not found" Jul 28 17:09:00.898106 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:00.898077 2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:09:00.906601 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:00.906576 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-34.ec2.internal\" not found" Jul 28 17:09:01.007082 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:01.006998 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-34.ec2.internal\" not found" Jul 28 17:09:01.107671 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:01.107632 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-34.ec2.internal\" not found" Jul 28 17:09:01.191517 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.191488 2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:09:01.209457 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.208846 2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-34.ec2.internal" Jul 28 17:09:01.218011 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.217977 2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 17:09:01.219026 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.219007 2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-34.ec2.internal" Jul 28 17:09:01.233858 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.233828 2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 17:09:01.591346 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.591300 2571 apiserver.go:52] "Watching apiserver" Jul 28 17:09:01.598506 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.598477 2571 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 28 17:09:01.601808 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.601780 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-dlxqx","kube-system/konnectivity-agent-nmkgg","kube-system/kube-apiserver-proxy-ip-10-0-141-34.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb","openshift-cluster-node-tuning-operator/tuned-qwhwb","openshift-image-registry/node-ca-dthjv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-34.ec2.internal","openshift-multus/multus-xwqnd","openshift-multus/network-metrics-daemon-w2qls","kube-system/global-pull-secret-syncer-w95vd","openshift-dns/node-resolver-6vlbb","openshift-multus/multus-additional-cni-plugins-v9cpp","openshift-network-diagnostics/network-check-target-6kkbm","openshift-network-operator/iptables-alerter-knlrx"] Jul 28 17:09:01.603414 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.603393 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:01.603510 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:01.603479 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w95vd" podUID="0af3fecb-e79a-4b70-ba3a-158278da3aaa" Jul 28 17:09:01.604450 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.604428 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:01.604573 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:01.604495 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kkbm" podUID="ab2a66ee-852a-4d34-88fc-1000fef6b92c" Jul 28 17:09:01.605447 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.605426 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-knlrx" Jul 28 17:09:01.606517 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.606497 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" Jul 28 17:09:01.607860 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.607595 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 28 17:09:01.607860 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.607622 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 28 17:09:01.607860 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.607648 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bgvtz\"" Jul 28 17:09:01.607860 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.607595 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:09:01.607860 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.607792 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.608459 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.608441 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 28 17:09:01.608695 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.608670 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8lv9m\"" Jul 28 17:09:01.608695 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.608686 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 28 17:09:01.608899 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.608778 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 28 17:09:01.608899 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.608873 2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 28 17:09:01.609033 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.609016 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dthjv" Jul 28 17:09:01.610174 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.610155 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:01.610267 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:01.610226 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2qls" podUID="c0d05255-0573-4806-84b3-c03ff36cea82" Jul 28 17:09:01.610267 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.610234 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 28 17:09:01.610383 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.610281 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:09:01.610722 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.610427 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xxbdm\"" Jul 28 17:09:01.610722 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.610607 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 28 17:09:01.610722 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.610707 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wswlp\"" Jul 28 17:09:01.610910 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.610809 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 28 17:09:01.610910 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.610824 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 28 17:09:01.612008 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.611990 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.613351 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.612957 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nmkgg" Jul 28 17:09:01.614591 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.614566 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 28 17:09:01.614993 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.614974 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 28 17:09:01.615061 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.615012 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 28 17:09:01.615228 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.615213 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 28 17:09:01.615415 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.615399 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7nsxk\"" Jul 28 17:09:01.615502 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.615493 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 28 17:09:01.615592 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.615578 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 28 17:09:01.615658 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.615643 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ksmbc\"" Jul 28 17:09:01.615971 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.615955 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.620424 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.618603 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 28 17:09:01.620424 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.618724 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qqvtv\"" Jul 28 17:09:01.620424 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.618862 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 28 17:09:01.620424 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.618920 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 28 17:09:01.620424 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619088 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6vlbb" Jul 28 17:09:01.620424 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619215 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-etc-sysctl-d\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.620424 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619244 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwbnb\" (UniqueName: \"kubernetes.io/projected/bd1b17ab-76be-4cfe-bab5-6bea59920888-kube-api-access-dwbnb\") pod \"aws-ebs-csi-driver-node-pz4tb\" (UID: \"bd1b17ab-76be-4cfe-bab5-6bea59920888\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" Jul 28 17:09:01.620424 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619274 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-run\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.620424 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619299 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-etc-tuned\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.620424 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619327 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqqcp\" (UniqueName: \"kubernetes.io/projected/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-kube-api-access-sqqcp\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.620424 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619357 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs\") pod \"network-metrics-daemon-w2qls\" (UID: \"c0d05255-0573-4806-84b3-c03ff36cea82\") " pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:01.620424 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619379 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8f0df316-8fe9-4dcf-ab0c-c502dd565001-iptables-alerter-script\") pod \"iptables-alerter-knlrx\" (UID: \"8f0df316-8fe9-4dcf-ab0c-c502dd565001\") " pod="openshift-network-operator/iptables-alerter-knlrx" Jul 28 17:09:01.620424 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619407 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bd1b17ab-76be-4cfe-bab5-6bea59920888-sys-fs\") pod \"aws-ebs-csi-driver-node-pz4tb\" (UID: \"bd1b17ab-76be-4cfe-bab5-6bea59920888\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" Jul 28 17:09:01.620424 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619433 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-etc-modprobe-d\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.620424 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619459 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-host\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.620424 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619487 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/507a907d-af5f-4815-93c4-0efe8cc04000-serviceca\") pod \"node-ca-dthjv\" (UID: \"507a907d-af5f-4815-93c4-0efe8cc04000\") " pod="openshift-image-registry/node-ca-dthjv" Jul 28 17:09:01.620424 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619524 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0af3fecb-e79a-4b70-ba3a-158278da3aaa-dbus\") pod \"global-pull-secret-syncer-w95vd\" (UID: \"0af3fecb-e79a-4b70-ba3a-158278da3aaa\") " pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:01.620424 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619570 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-etc-systemd\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.620424 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619604 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bd1b17ab-76be-4cfe-bab5-6bea59920888-device-dir\") pod \"aws-ebs-csi-driver-node-pz4tb\" (UID: \"bd1b17ab-76be-4cfe-bab5-6bea59920888\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" Jul 28 17:09:01.621531 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619636 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bd1b17ab-76be-4cfe-bab5-6bea59920888-etc-selinux\") pod \"aws-ebs-csi-driver-node-pz4tb\" (UID: \"bd1b17ab-76be-4cfe-bab5-6bea59920888\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" Jul 28 17:09:01.621531 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619665 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bd1b17ab-76be-4cfe-bab5-6bea59920888-socket-dir\") pod \"aws-ebs-csi-driver-node-pz4tb\" (UID: \"bd1b17ab-76be-4cfe-bab5-6bea59920888\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" Jul 28 17:09:01.621531 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619704 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-etc-sysconfig\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.621531 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619727 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-sys\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.621531 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619784 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 28 17:09:01.621531 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619796 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-tmp\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.621531 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619826 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/507a907d-af5f-4815-93c4-0efe8cc04000-host\") pod \"node-ca-dthjv\" (UID: \"507a907d-af5f-4815-93c4-0efe8cc04000\") " pod="openshift-image-registry/node-ca-dthjv" Jul 28 17:09:01.621531 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619854 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m984\" (UniqueName: \"kubernetes.io/projected/507a907d-af5f-4815-93c4-0efe8cc04000-kube-api-access-6m984\") pod \"node-ca-dthjv\" (UID: \"507a907d-af5f-4815-93c4-0efe8cc04000\") " pod="openshift-image-registry/node-ca-dthjv" Jul 28 17:09:01.621531 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619886 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnxqr\" (UniqueName: \"kubernetes.io/projected/c0d05255-0573-4806-84b3-c03ff36cea82-kube-api-access-gnxqr\") pod \"network-metrics-daemon-w2qls\" (UID: \"c0d05255-0573-4806-84b3-c03ff36cea82\") " pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:01.621531 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619915 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv27t\" (UniqueName: \"kubernetes.io/projected/ab2a66ee-852a-4d34-88fc-1000fef6b92c-kube-api-access-rv27t\") pod \"network-check-target-6kkbm\" (UID: \"ab2a66ee-852a-4d34-88fc-1000fef6b92c\") " pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:01.621531 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619939 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-etc-kubernetes\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.621531 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619965 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-etc-sysctl-conf\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.621531 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.619992 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bd1b17ab-76be-4cfe-bab5-6bea59920888-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pz4tb\" (UID: \"bd1b17ab-76be-4cfe-bab5-6bea59920888\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" Jul 28 17:09:01.621531 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.620022 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.621531 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.620050 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 28 17:09:01.621531 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.620211 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 28 17:09:01.621531 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.620023 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-lib-modules\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.621531 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.620327 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq7df\" (UniqueName: \"kubernetes.io/projected/8f0df316-8fe9-4dcf-ab0c-c502dd565001-kube-api-access-fq7df\") pod \"iptables-alerter-knlrx\" (UID: \"8f0df316-8fe9-4dcf-ab0c-c502dd565001\") " pod="openshift-network-operator/iptables-alerter-knlrx" Jul 28 17:09:01.622414 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.620374 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-var-lib-kubelet\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.622414 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.620431 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8f0df316-8fe9-4dcf-ab0c-c502dd565001-host-slash\") pod \"iptables-alerter-knlrx\" (UID: \"8f0df316-8fe9-4dcf-ab0c-c502dd565001\") " pod="openshift-network-operator/iptables-alerter-knlrx" Jul 28 17:09:01.622414 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.620494 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bd1b17ab-76be-4cfe-bab5-6bea59920888-registration-dir\") pod \"aws-ebs-csi-driver-node-pz4tb\" (UID: \"bd1b17ab-76be-4cfe-bab5-6bea59920888\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" Jul 28 17:09:01.622414 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.620633 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0af3fecb-e79a-4b70-ba3a-158278da3aaa-kubelet-config\") pod \"global-pull-secret-syncer-w95vd\" (UID: \"0af3fecb-e79a-4b70-ba3a-158278da3aaa\") " pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:01.622414 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.620706 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0af3fecb-e79a-4b70-ba3a-158278da3aaa-original-pull-secret\") pod \"global-pull-secret-syncer-w95vd\" (UID: \"0af3fecb-e79a-4b70-ba3a-158278da3aaa\") " pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:01.622414 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.620833 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 28 17:09:01.622414 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.620983 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ctzqn\"" Jul 28 17:09:01.622414 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.621190 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 28 17:09:01.622414 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.622360 2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 17:09:01.622414 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.622398 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 28 17:09:01.623108 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.622959 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 28 17:09:01.623670 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.623653 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5q9fn\"" Jul 28 17:09:01.642725 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.642693 2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ncsrt" Jul 28 17:09:01.651903 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.651880 2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ncsrt" Jul 28 17:09:01.710006 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.709981 2571 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 28 17:09:01.721159 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721126 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/52ae3ebe-36b6-44ca-94aa-ee34533ca203-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v9cpp\" (UID: \"52ae3ebe-36b6-44ca-94aa-ee34533ca203\") " pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.721159 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721160 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-host-var-lib-cni-bin\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.721371 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721183 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-etc-kubernetes\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.721371 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721204 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bd1b17ab-76be-4cfe-bab5-6bea59920888-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pz4tb\" (UID: \"bd1b17ab-76be-4cfe-bab5-6bea59920888\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" Jul 28 17:09:01.721371 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721225 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-host-kubelet\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.721371 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721244 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-run-openvswitch\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.721371 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721252 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-etc-kubernetes\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.721371 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721265 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/52ae3ebe-36b6-44ca-94aa-ee34533ca203-cni-binary-copy\") pod \"multus-additional-cni-plugins-v9cpp\" (UID: \"52ae3ebe-36b6-44ca-94aa-ee34533ca203\") " pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.721371 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721273 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bd1b17ab-76be-4cfe-bab5-6bea59920888-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pz4tb\" (UID: \"bd1b17ab-76be-4cfe-bab5-6bea59920888\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" Jul 28 17:09:01.721371 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721289 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/52ae3ebe-36b6-44ca-94aa-ee34533ca203-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v9cpp\" (UID: \"52ae3ebe-36b6-44ca-94aa-ee34533ca203\") " pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.721371 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721340 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-cnibin\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.721796 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721375 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-host-run-multus-certs\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.721796 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721411 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq7df\" (UniqueName: \"kubernetes.io/projected/8f0df316-8fe9-4dcf-ab0c-c502dd565001-kube-api-access-fq7df\") pod \"iptables-alerter-knlrx\" (UID: \"8f0df316-8fe9-4dcf-ab0c-c502dd565001\") " pod="openshift-network-operator/iptables-alerter-knlrx" Jul 28 17:09:01.721796 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721428 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-var-lib-openvswitch\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.721796 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721441 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-etc-openvswitch\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.721796 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721457 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-host-cni-netd\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.721796 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721478 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dkld\" (UniqueName: \"kubernetes.io/projected/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-kube-api-access-8dkld\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.721796 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721500 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p82c8\" (UniqueName: \"kubernetes.io/projected/52ae3ebe-36b6-44ca-94aa-ee34533ca203-kube-api-access-p82c8\") pod \"multus-additional-cni-plugins-v9cpp\" (UID: \"52ae3ebe-36b6-44ca-94aa-ee34533ca203\") " pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.721796 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721523 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-var-lib-kubelet\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.721796 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721543 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8f0df316-8fe9-4dcf-ab0c-c502dd565001-host-slash\") pod \"iptables-alerter-knlrx\" (UID: \"8f0df316-8fe9-4dcf-ab0c-c502dd565001\") " pod="openshift-network-operator/iptables-alerter-knlrx" Jul 28 17:09:01.721796 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721566 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bd1b17ab-76be-4cfe-bab5-6bea59920888-registration-dir\") pod \"aws-ebs-csi-driver-node-pz4tb\" (UID: \"bd1b17ab-76be-4cfe-bab5-6bea59920888\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" Jul 28 17:09:01.721796 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721605 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8f0df316-8fe9-4dcf-ab0c-c502dd565001-host-slash\") pod \"iptables-alerter-knlrx\" (UID: \"8f0df316-8fe9-4dcf-ab0c-c502dd565001\") " pod="openshift-network-operator/iptables-alerter-knlrx" Jul 28 17:09:01.721796 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721612 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-var-lib-kubelet\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.721796 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721625 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bd1b17ab-76be-4cfe-bab5-6bea59920888-registration-dir\") pod \"aws-ebs-csi-driver-node-pz4tb\" (UID: \"bd1b17ab-76be-4cfe-bab5-6bea59920888\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" Jul 28 17:09:01.721796 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721659 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0af3fecb-e79a-4b70-ba3a-158278da3aaa-kubelet-config\") pod \"global-pull-secret-syncer-w95vd\" (UID: \"0af3fecb-e79a-4b70-ba3a-158278da3aaa\") " pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:01.721796 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721680 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-ovnkube-config\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.721796 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721716 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-multus-cni-dir\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.721796 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721725 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0af3fecb-e79a-4b70-ba3a-158278da3aaa-kubelet-config\") pod \"global-pull-secret-syncer-w95vd\" (UID: \"0af3fecb-e79a-4b70-ba3a-158278da3aaa\") " pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:01.722585 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721740 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-etc-sysctl-d\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.722585 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721780 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-env-overrides\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.722585 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721798 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/52ae3ebe-36b6-44ca-94aa-ee34533ca203-system-cni-dir\") pod \"multus-additional-cni-plugins-v9cpp\" (UID: \"52ae3ebe-36b6-44ca-94aa-ee34533ca203\") " pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.722585 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721832 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-host-run-k8s-cni-cncf-io\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.722585 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721856 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-run\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.722585 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721877 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-etc-tuned\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.722585 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721914 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-run\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.722585 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721936 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8f0df316-8fe9-4dcf-ab0c-c502dd565001-iptables-alerter-script\") pod \"iptables-alerter-knlrx\" (UID: \"8f0df316-8fe9-4dcf-ab0c-c502dd565001\") " pod="openshift-network-operator/iptables-alerter-knlrx" Jul 28 17:09:01.722585 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721945 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-etc-sysctl-d\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.722585 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721961 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-host-run-ovn-kubernetes\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.722585 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.721979 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0cbdabd3-4530-4336-b84e-4b1bb7f75650-konnectivity-ca\") pod \"konnectivity-agent-nmkgg\" (UID: \"0cbdabd3-4530-4336-b84e-4b1bb7f75650\") " pod="kube-system/konnectivity-agent-nmkgg" Jul 28 17:09:01.722585 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722001 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-etc-modprobe-d\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.722585 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722075 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-host-run-netns\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.722585 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722098 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-etc-modprobe-d\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.722585 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722129 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/52ae3ebe-36b6-44ca-94aa-ee34533ca203-os-release\") pod \"multus-additional-cni-plugins-v9cpp\" (UID: \"52ae3ebe-36b6-44ca-94aa-ee34533ca203\") " pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.722585 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722169 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-etc-kubernetes\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.722585 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722194 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.723319 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722223 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bd1b17ab-76be-4cfe-bab5-6bea59920888-etc-selinux\") pod \"aws-ebs-csi-driver-node-pz4tb\" (UID: \"bd1b17ab-76be-4cfe-bab5-6bea59920888\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" Jul 28 17:09:01.723319 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722260 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-systemd-units\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.723319 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722280 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-host-slash\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.723319 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722273 2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 28 17:09:01.723319 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722302 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-host-run-netns\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.723319 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722325 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnqhn\" (UniqueName: \"kubernetes.io/projected/50a88180-0620-4676-ac16-95806564a133-kube-api-access-bnqhn\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.723319 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722342 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bd1b17ab-76be-4cfe-bab5-6bea59920888-etc-selinux\") pod \"aws-ebs-csi-driver-node-pz4tb\" (UID: \"bd1b17ab-76be-4cfe-bab5-6bea59920888\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" Jul 28 17:09:01.723319 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722347 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-tmp\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.723319 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722381 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/507a907d-af5f-4815-93c4-0efe8cc04000-host\") pod \"node-ca-dthjv\" (UID: \"507a907d-af5f-4815-93c4-0efe8cc04000\") " pod="openshift-image-registry/node-ca-dthjv" Jul 28 17:09:01.723319 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722402 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m984\" (UniqueName: \"kubernetes.io/projected/507a907d-af5f-4815-93c4-0efe8cc04000-kube-api-access-6m984\") pod \"node-ca-dthjv\" (UID: \"507a907d-af5f-4815-93c4-0efe8cc04000\") " pod="openshift-image-registry/node-ca-dthjv" Jul 28 17:09:01.723319 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722425 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv27t\" (UniqueName: \"kubernetes.io/projected/ab2a66ee-852a-4d34-88fc-1000fef6b92c-kube-api-access-rv27t\") pod \"network-check-target-6kkbm\" (UID: \"ab2a66ee-852a-4d34-88fc-1000fef6b92c\") " pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:01.723319 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722433 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/507a907d-af5f-4815-93c4-0efe8cc04000-host\") pod \"node-ca-dthjv\" (UID: \"507a907d-af5f-4815-93c4-0efe8cc04000\") " pod="openshift-image-registry/node-ca-dthjv" Jul 28 17:09:01.723319 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722444 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-system-cni-dir\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.723319 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722466 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-etc-sysctl-conf\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.723319 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722473 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8f0df316-8fe9-4dcf-ab0c-c502dd565001-iptables-alerter-script\") pod \"iptables-alerter-knlrx\" (UID: \"8f0df316-8fe9-4dcf-ab0c-c502dd565001\") " pod="openshift-network-operator/iptables-alerter-knlrx" Jul 28 17:09:01.723319 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722487 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-run-systemd\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.723319 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722510 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/52ae3ebe-36b6-44ca-94aa-ee34533ca203-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v9cpp\" (UID: \"52ae3ebe-36b6-44ca-94aa-ee34533ca203\") " pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.723978 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722556 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-host-var-lib-cni-multus\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.723978 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722604 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-lib-modules\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.723978 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722628 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-etc-sysctl-conf\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.723978 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722644 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0af3fecb-e79a-4b70-ba3a-158278da3aaa-original-pull-secret\") pod \"global-pull-secret-syncer-w95vd\" (UID: \"0af3fecb-e79a-4b70-ba3a-158278da3aaa\") " pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:01.723978 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722678 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-hostroot\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.723978 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:01.722715 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:09:01.723978 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722733 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-multus-conf-dir\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.723978 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722734 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-lib-modules\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.723978 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722788 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwbnb\" (UniqueName: \"kubernetes.io/projected/bd1b17ab-76be-4cfe-bab5-6bea59920888-kube-api-access-dwbnb\") pod \"aws-ebs-csi-driver-node-pz4tb\" (UID: \"bd1b17ab-76be-4cfe-bab5-6bea59920888\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" Jul 28 17:09:01.723978 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:01.722815 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0af3fecb-e79a-4b70-ba3a-158278da3aaa-original-pull-secret podName:0af3fecb-e79a-4b70-ba3a-158278da3aaa nodeName:}" failed. No retries permitted until 2026-07-28 17:09:02.222781077 +0000 UTC m=+2.942608778 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0af3fecb-e79a-4b70-ba3a-158278da3aaa-original-pull-secret") pod "global-pull-secret-syncer-w95vd" (UID: "0af3fecb-e79a-4b70-ba3a-158278da3aaa") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:09:01.723978 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722846 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-ovn-node-metrics-cert\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.723978 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722866 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-os-release\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.723978 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722884 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqqcp\" (UniqueName: \"kubernetes.io/projected/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-kube-api-access-sqqcp\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.723978 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722905 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs\") pod \"network-metrics-daemon-w2qls\" (UID: \"c0d05255-0573-4806-84b3-c03ff36cea82\") " pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:01.723978 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722927 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bd1b17ab-76be-4cfe-bab5-6bea59920888-sys-fs\") pod \"aws-ebs-csi-driver-node-pz4tb\" (UID: \"bd1b17ab-76be-4cfe-bab5-6bea59920888\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" Jul 28 17:09:01.723978 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722947 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-log-socket\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.723978 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722970 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7fc05f85-4048-4422-8800-59ca1e11e254-hosts-file\") pod \"node-resolver-6vlbb\" (UID: \"7fc05f85-4048-4422-8800-59ca1e11e254\") " pod="openshift-dns/node-resolver-6vlbb" Jul 28 17:09:01.724485 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.722995 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-host\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.724485 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723022 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/507a907d-af5f-4815-93c4-0efe8cc04000-serviceca\") pod \"node-ca-dthjv\" (UID: \"507a907d-af5f-4815-93c4-0efe8cc04000\") " pod="openshift-image-registry/node-ca-dthjv" Jul 28 17:09:01.724485 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723038 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bd1b17ab-76be-4cfe-bab5-6bea59920888-sys-fs\") pod \"aws-ebs-csi-driver-node-pz4tb\" (UID: \"bd1b17ab-76be-4cfe-bab5-6bea59920888\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" Jul 28 17:09:01.724485 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723050 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0af3fecb-e79a-4b70-ba3a-158278da3aaa-dbus\") pod \"global-pull-secret-syncer-w95vd\" (UID: \"0af3fecb-e79a-4b70-ba3a-158278da3aaa\") " pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:01.724485 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723075 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/50a88180-0620-4676-ac16-95806564a133-cni-binary-copy\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.724485 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723083 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-host\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.724485 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723101 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-multus-socket-dir-parent\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.724485 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723128 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-host-var-lib-kubelet\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.724485 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723154 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/50a88180-0620-4676-ac16-95806564a133-multus-daemon-config\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.724485 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723179 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0cbdabd3-4530-4336-b84e-4b1bb7f75650-agent-certs\") pod \"konnectivity-agent-nmkgg\" (UID: \"0cbdabd3-4530-4336-b84e-4b1bb7f75650\") " pod="kube-system/konnectivity-agent-nmkgg" Jul 28 17:09:01.724485 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723201 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0af3fecb-e79a-4b70-ba3a-158278da3aaa-dbus\") pod \"global-pull-secret-syncer-w95vd\" (UID: \"0af3fecb-e79a-4b70-ba3a-158278da3aaa\") " pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:01.724485 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723205 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-ovnkube-script-lib\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.724485 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723236 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-etc-systemd\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.724485 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723263 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bd1b17ab-76be-4cfe-bab5-6bea59920888-device-dir\") pod \"aws-ebs-csi-driver-node-pz4tb\" (UID: \"bd1b17ab-76be-4cfe-bab5-6bea59920888\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" Jul 28 17:09:01.724485 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723290 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-node-log\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.724485 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723298 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-etc-systemd\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.724485 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723324 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-host-cni-bin\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.725030 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723345 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fc05f85-4048-4422-8800-59ca1e11e254-tmp-dir\") pod \"node-resolver-6vlbb\" (UID: \"7fc05f85-4048-4422-8800-59ca1e11e254\") " pod="openshift-dns/node-resolver-6vlbb" Jul 28 17:09:01.725030 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723366 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bd1b17ab-76be-4cfe-bab5-6bea59920888-device-dir\") pod \"aws-ebs-csi-driver-node-pz4tb\" (UID: \"bd1b17ab-76be-4cfe-bab5-6bea59920888\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" Jul 28 17:09:01.725030 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723367 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/52ae3ebe-36b6-44ca-94aa-ee34533ca203-cnibin\") pod \"multus-additional-cni-plugins-v9cpp\" (UID: \"52ae3ebe-36b6-44ca-94aa-ee34533ca203\") " pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.725030 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723398 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bd1b17ab-76be-4cfe-bab5-6bea59920888-socket-dir\") pod \"aws-ebs-csi-driver-node-pz4tb\" (UID: \"bd1b17ab-76be-4cfe-bab5-6bea59920888\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" Jul 28 17:09:01.725030 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723422 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-etc-sysconfig\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.725030 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723443 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-sys\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.725030 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723466 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnxqr\" (UniqueName: \"kubernetes.io/projected/c0d05255-0573-4806-84b3-c03ff36cea82-kube-api-access-gnxqr\") pod \"network-metrics-daemon-w2qls\" (UID: \"c0d05255-0573-4806-84b3-c03ff36cea82\") " pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:01.725030 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723496 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bd1b17ab-76be-4cfe-bab5-6bea59920888-socket-dir\") pod \"aws-ebs-csi-driver-node-pz4tb\" (UID: \"bd1b17ab-76be-4cfe-bab5-6bea59920888\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" Jul 28 17:09:01.725030 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723508 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/507a907d-af5f-4815-93c4-0efe8cc04000-serviceca\") pod \"node-ca-dthjv\" (UID: \"507a907d-af5f-4815-93c4-0efe8cc04000\") " pod="openshift-image-registry/node-ca-dthjv" Jul 28 17:09:01.725030 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:01.723520 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:09:01.725030 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723525 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-etc-sysconfig\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.725030 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723512 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-run-ovn\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.725030 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:01.723565 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs podName:c0d05255-0573-4806-84b3-c03ff36cea82 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:02.223548622 +0000 UTC m=+2.943376321 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs") pod "network-metrics-daemon-w2qls" (UID: "c0d05255-0573-4806-84b3-c03ff36cea82") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:09:01.725030 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723568 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-sys\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.725030 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.723580 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76kk9\" (UniqueName: \"kubernetes.io/projected/7fc05f85-4048-4422-8800-59ca1e11e254-kube-api-access-76kk9\") pod \"node-resolver-6vlbb\" (UID: \"7fc05f85-4048-4422-8800-59ca1e11e254\") " pod="openshift-dns/node-resolver-6vlbb" Jul 28 17:09:01.725927 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.725902 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-tmp\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.726032 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.725950 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-etc-tuned\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.728857 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:01.728837 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:09:01.728986 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:01.728861 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:09:01.728986 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:01.728873 2571 projected.go:196] Error preparing data for projected volume kube-api-access-rv27t for pod openshift-network-diagnostics/network-check-target-6kkbm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:09:01.728986 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:01.728948 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab2a66ee-852a-4d34-88fc-1000fef6b92c-kube-api-access-rv27t podName:ab2a66ee-852a-4d34-88fc-1000fef6b92c nodeName:}" failed. No retries permitted until 2026-07-28 17:09:02.228931648 +0000 UTC m=+2.948759354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rv27t" (UniqueName: "kubernetes.io/projected/ab2a66ee-852a-4d34-88fc-1000fef6b92c-kube-api-access-rv27t") pod "network-check-target-6kkbm" (UID: "ab2a66ee-852a-4d34-88fc-1000fef6b92c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:09:01.729130 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.729012 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq7df\" (UniqueName: \"kubernetes.io/projected/8f0df316-8fe9-4dcf-ab0c-c502dd565001-kube-api-access-fq7df\") pod \"iptables-alerter-knlrx\" (UID: \"8f0df316-8fe9-4dcf-ab0c-c502dd565001\") " pod="openshift-network-operator/iptables-alerter-knlrx" Jul 28 17:09:01.731126 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.731103 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m984\" (UniqueName: \"kubernetes.io/projected/507a907d-af5f-4815-93c4-0efe8cc04000-kube-api-access-6m984\") pod \"node-ca-dthjv\" (UID: \"507a907d-af5f-4815-93c4-0efe8cc04000\") " pod="openshift-image-registry/node-ca-dthjv" Jul 28 17:09:01.731244 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.731221 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwbnb\" (UniqueName: \"kubernetes.io/projected/bd1b17ab-76be-4cfe-bab5-6bea59920888-kube-api-access-dwbnb\") pod \"aws-ebs-csi-driver-node-pz4tb\" (UID: \"bd1b17ab-76be-4cfe-bab5-6bea59920888\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" Jul 28 17:09:01.734714 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.734692 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnxqr\" (UniqueName: \"kubernetes.io/projected/c0d05255-0573-4806-84b3-c03ff36cea82-kube-api-access-gnxqr\") pod \"network-metrics-daemon-w2qls\" (UID: \"c0d05255-0573-4806-84b3-c03ff36cea82\") " pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:01.736650 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.736626 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqqcp\" (UniqueName: \"kubernetes.io/projected/a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3-kube-api-access-sqqcp\") pod \"tuned-qwhwb\" (UID: \"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3\") " pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.824712 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.824681 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-host-cni-bin\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.824897 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.824719 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fc05f85-4048-4422-8800-59ca1e11e254-tmp-dir\") pod \"node-resolver-6vlbb\" (UID: \"7fc05f85-4048-4422-8800-59ca1e11e254\") " pod="openshift-dns/node-resolver-6vlbb" Jul 28 17:09:01.824897 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.824737 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/52ae3ebe-36b6-44ca-94aa-ee34533ca203-cnibin\") pod \"multus-additional-cni-plugins-v9cpp\" (UID: \"52ae3ebe-36b6-44ca-94aa-ee34533ca203\") " pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.824897 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.824799 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/52ae3ebe-36b6-44ca-94aa-ee34533ca203-cnibin\") pod \"multus-additional-cni-plugins-v9cpp\" (UID: \"52ae3ebe-36b6-44ca-94aa-ee34533ca203\") " pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.824897 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.824820 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-host-cni-bin\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.824897 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.824865 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-run-ovn\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.825150 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.824899 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76kk9\" (UniqueName: \"kubernetes.io/projected/7fc05f85-4048-4422-8800-59ca1e11e254-kube-api-access-76kk9\") pod \"node-resolver-6vlbb\" (UID: \"7fc05f85-4048-4422-8800-59ca1e11e254\") " pod="openshift-dns/node-resolver-6vlbb" Jul 28 17:09:01.825150 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.824918 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/52ae3ebe-36b6-44ca-94aa-ee34533ca203-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v9cpp\" (UID: \"52ae3ebe-36b6-44ca-94aa-ee34533ca203\") " pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.825150 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.824934 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-host-var-lib-cni-bin\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.825150 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.824945 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-run-ovn\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.825150 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.824955 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-host-kubelet\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.825150 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.824975 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-run-openvswitch\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.825150 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.824991 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-host-var-lib-cni-bin\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.825150 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.824998 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/52ae3ebe-36b6-44ca-94aa-ee34533ca203-cni-binary-copy\") pod \"multus-additional-cni-plugins-v9cpp\" (UID: \"52ae3ebe-36b6-44ca-94aa-ee34533ca203\") " pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.825150 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825020 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/52ae3ebe-36b6-44ca-94aa-ee34533ca203-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v9cpp\" (UID: \"52ae3ebe-36b6-44ca-94aa-ee34533ca203\") " pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.825150 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825028 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-host-kubelet\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.825150 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825039 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-cnibin\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.825150 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825043 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fc05f85-4048-4422-8800-59ca1e11e254-tmp-dir\") pod \"node-resolver-6vlbb\" (UID: \"7fc05f85-4048-4422-8800-59ca1e11e254\") " pod="openshift-dns/node-resolver-6vlbb" Jul 28 17:09:01.825150 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825062 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-host-run-multus-certs\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.825150 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825092 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-run-openvswitch\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.825150 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825101 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-var-lib-openvswitch\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.825150 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825147 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-cnibin\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.825843 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825158 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-var-lib-openvswitch\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.825843 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825172 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-etc-openvswitch\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.825843 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825200 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-host-cni-netd\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.825843 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825224 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dkld\" (UniqueName: \"kubernetes.io/projected/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-kube-api-access-8dkld\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.825843 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825230 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-etc-openvswitch\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.825843 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825258 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p82c8\" (UniqueName: \"kubernetes.io/projected/52ae3ebe-36b6-44ca-94aa-ee34533ca203-kube-api-access-p82c8\") pod \"multus-additional-cni-plugins-v9cpp\" (UID: \"52ae3ebe-36b6-44ca-94aa-ee34533ca203\") " pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.825843 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825269 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-host-cni-netd\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.825843 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825202 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-host-run-multus-certs\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.825843 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825284 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-ovnkube-config\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.825843 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825307 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-multus-cni-dir\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.825843 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825328 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-env-overrides\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.825843 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825334 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/52ae3ebe-36b6-44ca-94aa-ee34533ca203-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v9cpp\" (UID: \"52ae3ebe-36b6-44ca-94aa-ee34533ca203\") " pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.825843 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825367 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/52ae3ebe-36b6-44ca-94aa-ee34533ca203-system-cni-dir\") pod \"multus-additional-cni-plugins-v9cpp\" (UID: \"52ae3ebe-36b6-44ca-94aa-ee34533ca203\") " pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.825843 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825387 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-host-run-k8s-cni-cncf-io\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.825843 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825403 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-host-run-ovn-kubernetes\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.825843 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825418 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0cbdabd3-4530-4336-b84e-4b1bb7f75650-konnectivity-ca\") pod \"konnectivity-agent-nmkgg\" (UID: \"0cbdabd3-4530-4336-b84e-4b1bb7f75650\") " pod="kube-system/konnectivity-agent-nmkgg" Jul 28 17:09:01.825843 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825439 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-host-run-netns\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.826557 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825457 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/52ae3ebe-36b6-44ca-94aa-ee34533ca203-os-release\") pod \"multus-additional-cni-plugins-v9cpp\" (UID: \"52ae3ebe-36b6-44ca-94aa-ee34533ca203\") " pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.826557 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825472 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-etc-kubernetes\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.826557 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825499 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.826557 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825524 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-systemd-units\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.826557 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825525 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-host-run-ovn-kubernetes\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.826557 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825579 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-systemd-units\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.826557 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825581 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.826557 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825528 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/52ae3ebe-36b6-44ca-94aa-ee34533ca203-system-cni-dir\") pod \"multus-additional-cni-plugins-v9cpp\" (UID: \"52ae3ebe-36b6-44ca-94aa-ee34533ca203\") " pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.826557 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825609 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-host-run-k8s-cni-cncf-io\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.826557 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825630 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/52ae3ebe-36b6-44ca-94aa-ee34533ca203-cni-binary-copy\") pod \"multus-additional-cni-plugins-v9cpp\" (UID: \"52ae3ebe-36b6-44ca-94aa-ee34533ca203\") " pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.826557 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825652 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/52ae3ebe-36b6-44ca-94aa-ee34533ca203-os-release\") pod \"multus-additional-cni-plugins-v9cpp\" (UID: \"52ae3ebe-36b6-44ca-94aa-ee34533ca203\") " pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.826557 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825672 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-multus-cni-dir\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.826557 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825690 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-host-slash\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.826557 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825693 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-host-run-netns\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.826557 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825715 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-host-run-netns\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.826557 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825732 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnqhn\" (UniqueName: \"kubernetes.io/projected/50a88180-0620-4676-ac16-95806564a133-kube-api-access-bnqhn\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.826557 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825731 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-env-overrides\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.827278 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825795 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-etc-kubernetes\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.827278 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825802 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-host-run-netns\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.827278 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825839 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-system-cni-dir\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.827278 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825875 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-run-systemd\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.827278 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825894 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-host-slash\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.827278 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825928 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/52ae3ebe-36b6-44ca-94aa-ee34533ca203-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v9cpp\" (UID: \"52ae3ebe-36b6-44ca-94aa-ee34533ca203\") " pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.827278 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825937 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-run-systemd\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.827278 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825954 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-host-var-lib-cni-multus\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.827278 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.825990 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-host-var-lib-cni-multus\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.827278 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826006 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-system-cni-dir\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.827278 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826031 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-hostroot\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.827278 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826052 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-multus-conf-dir\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.827278 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826078 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-ovn-node-metrics-cert\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.827278 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826080 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-hostroot\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.827278 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826049 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/52ae3ebe-36b6-44ca-94aa-ee34533ca203-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v9cpp\" (UID: \"52ae3ebe-36b6-44ca-94aa-ee34533ca203\") " pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.827278 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826084 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-multus-conf-dir\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.827278 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826099 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-os-release\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.827278 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826091 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0cbdabd3-4530-4336-b84e-4b1bb7f75650-konnectivity-ca\") pod \"konnectivity-agent-nmkgg\" (UID: \"0cbdabd3-4530-4336-b84e-4b1bb7f75650\") " pod="kube-system/konnectivity-agent-nmkgg" Jul 28 17:09:01.827973 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826149 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-os-release\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.827973 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826235 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-log-socket\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.827973 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826288 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7fc05f85-4048-4422-8800-59ca1e11e254-hosts-file\") pod \"node-resolver-6vlbb\" (UID: \"7fc05f85-4048-4422-8800-59ca1e11e254\") " pod="openshift-dns/node-resolver-6vlbb" Jul 28 17:09:01.827973 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826319 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-ovnkube-config\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.827973 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826332 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/50a88180-0620-4676-ac16-95806564a133-cni-binary-copy\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.827973 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826353 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/52ae3ebe-36b6-44ca-94aa-ee34533ca203-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v9cpp\" (UID: \"52ae3ebe-36b6-44ca-94aa-ee34533ca203\") " pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.827973 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826360 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-multus-socket-dir-parent\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.827973 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826394 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-multus-socket-dir-parent\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.827973 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826403 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-host-var-lib-kubelet\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.827973 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826424 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-log-socket\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.827973 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826427 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/50a88180-0620-4676-ac16-95806564a133-multus-daemon-config\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.827973 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826449 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0cbdabd3-4530-4336-b84e-4b1bb7f75650-agent-certs\") pod \"konnectivity-agent-nmkgg\" (UID: \"0cbdabd3-4530-4336-b84e-4b1bb7f75650\") " pod="kube-system/konnectivity-agent-nmkgg" Jul 28 17:09:01.827973 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826458 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7fc05f85-4048-4422-8800-59ca1e11e254-hosts-file\") pod \"node-resolver-6vlbb\" (UID: \"7fc05f85-4048-4422-8800-59ca1e11e254\") " pod="openshift-dns/node-resolver-6vlbb" Jul 28 17:09:01.827973 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826471 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-ovnkube-script-lib\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.827973 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826482 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50a88180-0620-4676-ac16-95806564a133-host-var-lib-kubelet\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.827973 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826496 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-node-log\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.827973 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826560 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-node-log\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.827973 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.826859 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/50a88180-0620-4676-ac16-95806564a133-cni-binary-copy\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.828630 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.827326 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/50a88180-0620-4676-ac16-95806564a133-multus-daemon-config\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.828630 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.827467 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-ovnkube-script-lib\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.829055 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.829029 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-ovn-node-metrics-cert\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.829142 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.829069 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0cbdabd3-4530-4336-b84e-4b1bb7f75650-agent-certs\") pod \"konnectivity-agent-nmkgg\" (UID: \"0cbdabd3-4530-4336-b84e-4b1bb7f75650\") " pod="kube-system/konnectivity-agent-nmkgg" Jul 28 17:09:01.833512 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.833488 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dkld\" (UniqueName: \"kubernetes.io/projected/bfc273ea-75a3-41b1-9158-6a8e00aeaa2e-kube-api-access-8dkld\") pod \"ovnkube-node-dlxqx\" (UID: \"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.833608 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.833575 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnqhn\" (UniqueName: \"kubernetes.io/projected/50a88180-0620-4676-ac16-95806564a133-kube-api-access-bnqhn\") pod \"multus-xwqnd\" (UID: \"50a88180-0620-4676-ac16-95806564a133\") " pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.833681 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.833660 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p82c8\" (UniqueName: \"kubernetes.io/projected/52ae3ebe-36b6-44ca-94aa-ee34533ca203-kube-api-access-p82c8\") pod \"multus-additional-cni-plugins-v9cpp\" (UID: \"52ae3ebe-36b6-44ca-94aa-ee34533ca203\") " pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:01.833849 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.833834 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76kk9\" (UniqueName: \"kubernetes.io/projected/7fc05f85-4048-4422-8800-59ca1e11e254-kube-api-access-76kk9\") pod \"node-resolver-6vlbb\" (UID: \"7fc05f85-4048-4422-8800-59ca1e11e254\") " pod="openshift-dns/node-resolver-6vlbb" Jul 28 17:09:01.917716 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.917637 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-knlrx" Jul 28 17:09:01.926312 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.926276 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" Jul 28 17:09:01.932461 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.932430 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" Jul 28 17:09:01.934088 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.934066 2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:09:01.937576 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.937552 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dthjv" Jul 28 17:09:01.944230 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.944199 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xwqnd" Jul 28 17:09:01.950185 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.950161 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nmkgg" Jul 28 17:09:01.957146 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.957121 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:01.962066 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.962038 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6vlbb" Jul 28 17:09:01.967821 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:01.967795 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v9cpp" Jul 28 17:09:02.229244 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:02.229158 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv27t\" (UniqueName: \"kubernetes.io/projected/ab2a66ee-852a-4d34-88fc-1000fef6b92c-kube-api-access-rv27t\") pod \"network-check-target-6kkbm\" (UID: \"ab2a66ee-852a-4d34-88fc-1000fef6b92c\") " pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:02.229244 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:02.229201 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0af3fecb-e79a-4b70-ba3a-158278da3aaa-original-pull-secret\") pod \"global-pull-secret-syncer-w95vd\" (UID: \"0af3fecb-e79a-4b70-ba3a-158278da3aaa\") " pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:02.229244 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:02.229232 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs\") pod \"network-metrics-daemon-w2qls\" (UID: \"c0d05255-0573-4806-84b3-c03ff36cea82\") " pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:02.229492 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:02.229319 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:09:02.229492 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:02.229339 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:09:02.229492 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:02.229361 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:09:02.229492 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:02.229375 2571 projected.go:196] Error preparing data for projected volume kube-api-access-rv27t for pod openshift-network-diagnostics/network-check-target-6kkbm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:09:02.229492 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:02.229388 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0af3fecb-e79a-4b70-ba3a-158278da3aaa-original-pull-secret podName:0af3fecb-e79a-4b70-ba3a-158278da3aaa nodeName:}" failed. No retries permitted until 2026-07-28 17:09:03.229371325 +0000 UTC m=+3.949199018 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0af3fecb-e79a-4b70-ba3a-158278da3aaa-original-pull-secret") pod "global-pull-secret-syncer-w95vd" (UID: "0af3fecb-e79a-4b70-ba3a-158278da3aaa") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:09:02.229492 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:02.229318 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:09:02.229492 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:02.229429 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab2a66ee-852a-4d34-88fc-1000fef6b92c-kube-api-access-rv27t podName:ab2a66ee-852a-4d34-88fc-1000fef6b92c nodeName:}" failed. No retries permitted until 2026-07-28 17:09:03.229413112 +0000 UTC m=+3.949240819 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rv27t" (UniqueName: "kubernetes.io/projected/ab2a66ee-852a-4d34-88fc-1000fef6b92c-kube-api-access-rv27t") pod "network-check-target-6kkbm" (UID: "ab2a66ee-852a-4d34-88fc-1000fef6b92c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:09:02.229492 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:02.229448 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs podName:c0d05255-0573-4806-84b3-c03ff36cea82 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:03.229440003 +0000 UTC m=+3.949267702 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs") pod "network-metrics-daemon-w2qls" (UID: "c0d05255-0573-4806-84b3-c03ff36cea82") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:09:02.378916 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:09:02.378878 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda47e2ee1_1e91_4768_a10d_6b5a8fb46dd3.slice/crio-94e93091a3521c2719efbb58815504946e93b52394a15add5c78be836f969664 WatchSource:0}: Error finding container 94e93091a3521c2719efbb58815504946e93b52394a15add5c78be836f969664: Status 404 returned error can't find the container with id 94e93091a3521c2719efbb58815504946e93b52394a15add5c78be836f969664 Jul 28 17:09:02.380513 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:09:02.380014 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cbdabd3_4530_4336_b84e_4b1bb7f75650.slice/crio-8003a177ca104b0504127fb297bb48c086706ab789a310b04df60dff0147f866 WatchSource:0}: Error finding container 8003a177ca104b0504127fb297bb48c086706ab789a310b04df60dff0147f866: Status 404 returned error can't find the container with id 8003a177ca104b0504127fb297bb48c086706ab789a310b04df60dff0147f866 Jul 28 17:09:02.382574 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:09:02.382548 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f0df316_8fe9_4dcf_ab0c_c502dd565001.slice/crio-28b6ce1381e5e48b1c815075d0a6f5735fdcfa885691578b14eb21512d38fa90 WatchSource:0}: Error finding container 28b6ce1381e5e48b1c815075d0a6f5735fdcfa885691578b14eb21512d38fa90: Status 404 returned error can't find the container with id 28b6ce1381e5e48b1c815075d0a6f5735fdcfa885691578b14eb21512d38fa90 Jul 28 17:09:02.383610 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:09:02.383590 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod507a907d_af5f_4815_93c4_0efe8cc04000.slice/crio-d29bef2c18747bbf79a0355540870afec6ba7f1117da3dd1404b36d8bfbbacc9 WatchSource:0}: Error finding container d29bef2c18747bbf79a0355540870afec6ba7f1117da3dd1404b36d8bfbbacc9: Status 404 returned error can't find the container with id d29bef2c18747bbf79a0355540870afec6ba7f1117da3dd1404b36d8bfbbacc9 Jul 28 17:09:02.384865 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:09:02.384828 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd1b17ab_76be_4cfe_bab5_6bea59920888.slice/crio-0e62b63330be512893d8d3ecba812f25b0f911506775f65d6bef28ad86d2de12 WatchSource:0}: Error finding container 0e62b63330be512893d8d3ecba812f25b0f911506775f65d6bef28ad86d2de12: Status 404 returned error can't find the container with id 0e62b63330be512893d8d3ecba812f25b0f911506775f65d6bef28ad86d2de12 Jul 28 17:09:02.385885 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:09:02.385864 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50a88180_0620_4676_ac16_95806564a133.slice/crio-56a56d491f4c4b0ab9328581dfd908ae9b2a78aefa8bd67ec9b33bbca9f41f53 WatchSource:0}: Error finding container 56a56d491f4c4b0ab9328581dfd908ae9b2a78aefa8bd67ec9b33bbca9f41f53: Status 404 returned error can't find the container with id 56a56d491f4c4b0ab9328581dfd908ae9b2a78aefa8bd67ec9b33bbca9f41f53 Jul 28 17:09:02.387669 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:09:02.387332 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52ae3ebe_36b6_44ca_94aa_ee34533ca203.slice/crio-9dd7fe3f30b70ae8693558408a08df050276cfb96091e34a3e96325020eda479 WatchSource:0}: Error finding container 9dd7fe3f30b70ae8693558408a08df050276cfb96091e34a3e96325020eda479: Status 404 returned error can't find the container with id 9dd7fe3f30b70ae8693558408a08df050276cfb96091e34a3e96325020eda479 Jul 28 17:09:02.388201 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:09:02.388178 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fc05f85_4048_4422_8800_59ca1e11e254.slice/crio-804c1cb06d79ffca7b2e799042446b451790a484417c838de6a731bf526bc08f WatchSource:0}: Error finding container 804c1cb06d79ffca7b2e799042446b451790a484417c838de6a731bf526bc08f: Status 404 returned error can't find the container with id 804c1cb06d79ffca7b2e799042446b451790a484417c838de6a731bf526bc08f Jul 28 17:09:02.390237 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:09:02.390214 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfc273ea_75a3_41b1_9158_6a8e00aeaa2e.slice/crio-d2a0e81a916553eb6d76709784286d84cf39d7fb6bf8a4134acf9073d32ef84f WatchSource:0}: Error finding container d2a0e81a916553eb6d76709784286d84cf39d7fb6bf8a4134acf9073d32ef84f: Status 404 returned error can't find the container with id d2a0e81a916553eb6d76709784286d84cf39d7fb6bf8a4134acf9073d32ef84f Jul 28 17:09:02.654210 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:02.654162 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:04:01 +0000 UTC" deadline="2028-04-22 06:22:07.710915814 +0000 UTC" Jul 28 17:09:02.654210 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:02.654206 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15205h13m5.056712751s" Jul 28 17:09:02.790000 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:02.789884 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nmkgg" event={"ID":"0cbdabd3-4530-4336-b84e-4b1bb7f75650","Type":"ContainerStarted","Data":"8003a177ca104b0504127fb297bb48c086706ab789a310b04df60dff0147f866"} Jul 28 17:09:02.791454 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:02.791409 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" event={"ID":"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e","Type":"ContainerStarted","Data":"d2a0e81a916553eb6d76709784286d84cf39d7fb6bf8a4134acf9073d32ef84f"} Jul 28 17:09:02.793192 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:02.793167 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6vlbb" event={"ID":"7fc05f85-4048-4422-8800-59ca1e11e254","Type":"ContainerStarted","Data":"804c1cb06d79ffca7b2e799042446b451790a484417c838de6a731bf526bc08f"} Jul 28 17:09:02.795030 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:02.794975 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v9cpp" event={"ID":"52ae3ebe-36b6-44ca-94aa-ee34533ca203","Type":"ContainerStarted","Data":"9dd7fe3f30b70ae8693558408a08df050276cfb96091e34a3e96325020eda479"} Jul 28 17:09:02.796548 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:02.796468 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" event={"ID":"bd1b17ab-76be-4cfe-bab5-6bea59920888","Type":"ContainerStarted","Data":"0e62b63330be512893d8d3ecba812f25b0f911506775f65d6bef28ad86d2de12"} Jul 28 17:09:02.798190 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:02.798161 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-knlrx" event={"ID":"8f0df316-8fe9-4dcf-ab0c-c502dd565001","Type":"ContainerStarted","Data":"28b6ce1381e5e48b1c815075d0a6f5735fdcfa885691578b14eb21512d38fa90"} Jul 28 17:09:02.800002 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:02.799976 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" event={"ID":"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3","Type":"ContainerStarted","Data":"94e93091a3521c2719efbb58815504946e93b52394a15add5c78be836f969664"} Jul 28 17:09:02.802221 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:02.802195 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-34.ec2.internal" event={"ID":"220364fcdea4bfcb02dba704a736fdd7","Type":"ContainerStarted","Data":"66732975c9d489ca1d1fe1f62176e53990a533474e8a611e4fca9e327b05caec"} Jul 28 17:09:02.804261 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:02.804237 2571 generic.go:356] "Generic (PLEG): container finished" podID="04fe575350fb222dd4149f78daabf09c" containerID="0f85336793640f18f93f61dd20336c08b24dd672d9014a3805abfd7fc40c3485" exitCode=0 Jul 28 17:09:02.804454 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:02.804430 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-34.ec2.internal" event={"ID":"04fe575350fb222dd4149f78daabf09c","Type":"ContainerDied","Data":"0f85336793640f18f93f61dd20336c08b24dd672d9014a3805abfd7fc40c3485"} Jul 28 17:09:02.806831 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:02.806805 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xwqnd" event={"ID":"50a88180-0620-4676-ac16-95806564a133","Type":"ContainerStarted","Data":"56a56d491f4c4b0ab9328581dfd908ae9b2a78aefa8bd67ec9b33bbca9f41f53"} Jul 28 17:09:02.810045 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:02.809992 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dthjv" event={"ID":"507a907d-af5f-4815-93c4-0efe8cc04000","Type":"ContainerStarted","Data":"d29bef2c18747bbf79a0355540870afec6ba7f1117da3dd1404b36d8bfbbacc9"} Jul 28 17:09:02.835386 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:02.835328 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-34.ec2.internal" podStartSLOduration=2.835306455 podStartE2EDuration="2.835306455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:09:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:02.817409035 +0000 UTC m=+3.537236743" watchObservedRunningTime="2026-07-28 17:09:02.835306455 +0000 UTC m=+3.555134161" Jul 28 17:09:03.238901 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:03.237914 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv27t\" (UniqueName: \"kubernetes.io/projected/ab2a66ee-852a-4d34-88fc-1000fef6b92c-kube-api-access-rv27t\") pod \"network-check-target-6kkbm\" (UID: \"ab2a66ee-852a-4d34-88fc-1000fef6b92c\") " pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:03.238901 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:03.237966 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0af3fecb-e79a-4b70-ba3a-158278da3aaa-original-pull-secret\") pod \"global-pull-secret-syncer-w95vd\" (UID: \"0af3fecb-e79a-4b70-ba3a-158278da3aaa\") " pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:03.238901 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:03.238069 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs\") pod \"network-metrics-daemon-w2qls\" (UID: \"c0d05255-0573-4806-84b3-c03ff36cea82\") " pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:03.238901 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:03.238216 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:09:03.238901 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:03.238277 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs podName:c0d05255-0573-4806-84b3-c03ff36cea82 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:05.238259451 +0000 UTC m=+5.958087143 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs") pod "network-metrics-daemon-w2qls" (UID: "c0d05255-0573-4806-84b3-c03ff36cea82") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:09:03.238901 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:03.238366 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:09:03.238901 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:03.238382 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:09:03.238901 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:03.238393 2571 projected.go:196] Error preparing data for projected volume kube-api-access-rv27t for pod openshift-network-diagnostics/network-check-target-6kkbm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:09:03.238901 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:03.238426 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab2a66ee-852a-4d34-88fc-1000fef6b92c-kube-api-access-rv27t podName:ab2a66ee-852a-4d34-88fc-1000fef6b92c nodeName:}" failed. No retries permitted until 2026-07-28 17:09:05.238414879 +0000 UTC m=+5.958242568 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rv27t" (UniqueName: "kubernetes.io/projected/ab2a66ee-852a-4d34-88fc-1000fef6b92c-kube-api-access-rv27t") pod "network-check-target-6kkbm" (UID: "ab2a66ee-852a-4d34-88fc-1000fef6b92c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:09:03.238901 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:03.238479 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:09:03.238901 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:03.238507 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0af3fecb-e79a-4b70-ba3a-158278da3aaa-original-pull-secret podName:0af3fecb-e79a-4b70-ba3a-158278da3aaa nodeName:}" failed. No retries permitted until 2026-07-28 17:09:05.238498244 +0000 UTC m=+5.958325934 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0af3fecb-e79a-4b70-ba3a-158278da3aaa-original-pull-secret") pod "global-pull-secret-syncer-w95vd" (UID: "0af3fecb-e79a-4b70-ba3a-158278da3aaa") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:09:03.655144 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:03.655053 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:04:01 +0000 UTC" deadline="2028-01-22 19:22:31.698619808 +0000 UTC" Jul 28 17:09:03.655144 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:03.655085 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13034h13m28.043538236s" Jul 28 17:09:03.782324 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:03.782292 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:03.782491 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:03.782411 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kkbm" podUID="ab2a66ee-852a-4d34-88fc-1000fef6b92c" Jul 28 17:09:03.782491 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:03.782485 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:03.782607 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:03.782549 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w95vd" podUID="0af3fecb-e79a-4b70-ba3a-158278da3aaa" Jul 28 17:09:03.782607 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:03.782604 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:03.782740 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:03.782691 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2qls" podUID="c0d05255-0573-4806-84b3-c03ff36cea82" Jul 28 17:09:03.837480 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:03.837405 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-34.ec2.internal" event={"ID":"04fe575350fb222dd4149f78daabf09c","Type":"ContainerStarted","Data":"bb449e6c4f1c89938232d6130b83ef7bdb3243678eff2e9a00745f364a4ee9b2"} Jul 28 17:09:05.254175 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:05.253111 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0af3fecb-e79a-4b70-ba3a-158278da3aaa-original-pull-secret\") pod \"global-pull-secret-syncer-w95vd\" (UID: \"0af3fecb-e79a-4b70-ba3a-158278da3aaa\") " pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:05.254175 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:05.253170 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs\") pod \"network-metrics-daemon-w2qls\" (UID: \"c0d05255-0573-4806-84b3-c03ff36cea82\") " pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:05.254175 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:05.253244 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv27t\" (UniqueName: \"kubernetes.io/projected/ab2a66ee-852a-4d34-88fc-1000fef6b92c-kube-api-access-rv27t\") pod \"network-check-target-6kkbm\" (UID: \"ab2a66ee-852a-4d34-88fc-1000fef6b92c\") " pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:05.254175 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:05.253383 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:09:05.254175 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:05.253401 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:09:05.254175 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:05.253414 2571 projected.go:196] Error preparing data for projected volume kube-api-access-rv27t for pod openshift-network-diagnostics/network-check-target-6kkbm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:09:05.254175 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:05.253470 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab2a66ee-852a-4d34-88fc-1000fef6b92c-kube-api-access-rv27t podName:ab2a66ee-852a-4d34-88fc-1000fef6b92c nodeName:}" failed. No retries permitted until 2026-07-28 17:09:09.253451175 +0000 UTC m=+9.973278869 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rv27t" (UniqueName: "kubernetes.io/projected/ab2a66ee-852a-4d34-88fc-1000fef6b92c-kube-api-access-rv27t") pod "network-check-target-6kkbm" (UID: "ab2a66ee-852a-4d34-88fc-1000fef6b92c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:09:05.254175 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:05.253911 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:09:05.254175 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:05.253963 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0af3fecb-e79a-4b70-ba3a-158278da3aaa-original-pull-secret podName:0af3fecb-e79a-4b70-ba3a-158278da3aaa nodeName:}" failed. No retries permitted until 2026-07-28 17:09:09.25394662 +0000 UTC m=+9.973774312 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0af3fecb-e79a-4b70-ba3a-158278da3aaa-original-pull-secret") pod "global-pull-secret-syncer-w95vd" (UID: "0af3fecb-e79a-4b70-ba3a-158278da3aaa") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:09:05.254175 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:05.254019 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:09:05.254175 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:05.254049 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs podName:c0d05255-0573-4806-84b3-c03ff36cea82 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:09.254039435 +0000 UTC m=+9.973867127 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs") pod "network-metrics-daemon-w2qls" (UID: "c0d05255-0573-4806-84b3-c03ff36cea82") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:09:05.783097 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:05.782190 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:05.783097 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:05.782326 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w95vd" podUID="0af3fecb-e79a-4b70-ba3a-158278da3aaa" Jul 28 17:09:05.783097 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:05.782747 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:05.783097 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:05.782867 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2qls" podUID="c0d05255-0573-4806-84b3-c03ff36cea82" Jul 28 17:09:05.783097 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:05.782928 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:05.783097 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:05.782993 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kkbm" podUID="ab2a66ee-852a-4d34-88fc-1000fef6b92c" Jul 28 17:09:07.780320 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:07.779822 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:07.780320 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:07.779855 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:07.780320 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:07.779822 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:07.780320 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:07.779950 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kkbm" podUID="ab2a66ee-852a-4d34-88fc-1000fef6b92c" Jul 28 17:09:07.780320 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:07.780059 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2qls" podUID="c0d05255-0573-4806-84b3-c03ff36cea82" Jul 28 17:09:07.780320 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:07.780142 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w95vd" podUID="0af3fecb-e79a-4b70-ba3a-158278da3aaa" Jul 28 17:09:09.290365 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:09.290302 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv27t\" (UniqueName: \"kubernetes.io/projected/ab2a66ee-852a-4d34-88fc-1000fef6b92c-kube-api-access-rv27t\") pod \"network-check-target-6kkbm\" (UID: \"ab2a66ee-852a-4d34-88fc-1000fef6b92c\") " pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:09.290869 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:09.290390 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0af3fecb-e79a-4b70-ba3a-158278da3aaa-original-pull-secret\") pod \"global-pull-secret-syncer-w95vd\" (UID: \"0af3fecb-e79a-4b70-ba3a-158278da3aaa\") " pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:09.290869 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:09.290421 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs\") pod \"network-metrics-daemon-w2qls\" (UID: \"c0d05255-0573-4806-84b3-c03ff36cea82\") " pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:09.290869 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:09.290593 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:09:09.290869 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:09.290620 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:09:09.290869 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:09.290632 2571 projected.go:196] Error preparing data for projected volume kube-api-access-rv27t for pod openshift-network-diagnostics/network-check-target-6kkbm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:09:09.290869 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:09.290640 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:09:09.290869 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:09.290602 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:09:09.290869 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:09.290696 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs podName:c0d05255-0573-4806-84b3-c03ff36cea82 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:17.290676602 +0000 UTC m=+18.010504299 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs") pod "network-metrics-daemon-w2qls" (UID: "c0d05255-0573-4806-84b3-c03ff36cea82") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:09:09.290869 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:09.290718 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab2a66ee-852a-4d34-88fc-1000fef6b92c-kube-api-access-rv27t podName:ab2a66ee-852a-4d34-88fc-1000fef6b92c nodeName:}" failed. No retries permitted until 2026-07-28 17:09:17.290707947 +0000 UTC m=+18.010535638 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rv27t" (UniqueName: "kubernetes.io/projected/ab2a66ee-852a-4d34-88fc-1000fef6b92c-kube-api-access-rv27t") pod "network-check-target-6kkbm" (UID: "ab2a66ee-852a-4d34-88fc-1000fef6b92c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:09:09.290869 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:09.290743 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0af3fecb-e79a-4b70-ba3a-158278da3aaa-original-pull-secret podName:0af3fecb-e79a-4b70-ba3a-158278da3aaa nodeName:}" failed. No retries permitted until 2026-07-28 17:09:17.290734171 +0000 UTC m=+18.010561862 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0af3fecb-e79a-4b70-ba3a-158278da3aaa-original-pull-secret") pod "global-pull-secret-syncer-w95vd" (UID: "0af3fecb-e79a-4b70-ba3a-158278da3aaa") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:09:09.783524 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:09.783070 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:09.783524 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:09.783204 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:09.783524 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:09.783254 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:09.783524 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:09.783243 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2qls" podUID="c0d05255-0573-4806-84b3-c03ff36cea82" Jul 28 17:09:09.783524 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:09.783359 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kkbm" podUID="ab2a66ee-852a-4d34-88fc-1000fef6b92c" Jul 28 17:09:09.783524 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:09.783437 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w95vd" podUID="0af3fecb-e79a-4b70-ba3a-158278da3aaa" Jul 28 17:09:11.780066 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:11.780027 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:11.780550 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:11.780030 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:11.780550 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:11.780193 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2qls" podUID="c0d05255-0573-4806-84b3-c03ff36cea82" Jul 28 17:09:11.780550 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:11.780290 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:11.780550 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:11.780290 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kkbm" podUID="ab2a66ee-852a-4d34-88fc-1000fef6b92c" Jul 28 17:09:11.780550 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:11.780390 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w95vd" podUID="0af3fecb-e79a-4b70-ba3a-158278da3aaa" Jul 28 17:09:13.780275 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:13.780234 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:13.780708 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:13.780244 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:13.780708 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:13.780359 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:13.780708 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:13.780376 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w95vd" podUID="0af3fecb-e79a-4b70-ba3a-158278da3aaa" Jul 28 17:09:13.780708 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:13.780451 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kkbm" podUID="ab2a66ee-852a-4d34-88fc-1000fef6b92c" Jul 28 17:09:13.780708 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:13.780538 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2qls" podUID="c0d05255-0573-4806-84b3-c03ff36cea82" Jul 28 17:09:15.779729 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:15.779693 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:15.780242 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:15.779693 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:15.780242 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:15.779851 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w95vd" podUID="0af3fecb-e79a-4b70-ba3a-158278da3aaa" Jul 28 17:09:15.780242 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:15.779695 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:15.780242 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:15.779922 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2qls" podUID="c0d05255-0573-4806-84b3-c03ff36cea82" Jul 28 17:09:15.780242 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:15.779954 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kkbm" podUID="ab2a66ee-852a-4d34-88fc-1000fef6b92c" Jul 28 17:09:17.351809 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:17.351753 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0af3fecb-e79a-4b70-ba3a-158278da3aaa-original-pull-secret\") pod \"global-pull-secret-syncer-w95vd\" (UID: \"0af3fecb-e79a-4b70-ba3a-158278da3aaa\") " pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:17.351809 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:17.351814 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs\") pod \"network-metrics-daemon-w2qls\" (UID: \"c0d05255-0573-4806-84b3-c03ff36cea82\") " pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:17.352487 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:17.351869 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv27t\" (UniqueName: \"kubernetes.io/projected/ab2a66ee-852a-4d34-88fc-1000fef6b92c-kube-api-access-rv27t\") pod \"network-check-target-6kkbm\" (UID: \"ab2a66ee-852a-4d34-88fc-1000fef6b92c\") " pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:17.352487 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:17.351940 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:09:17.352487 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:17.351978 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:09:17.352487 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:17.351991 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:09:17.352487 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:17.352006 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:09:17.352487 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:17.352019 2571 projected.go:196] Error preparing data for projected volume kube-api-access-rv27t for pod openshift-network-diagnostics/network-check-target-6kkbm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:09:17.352487 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:17.352023 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0af3fecb-e79a-4b70-ba3a-158278da3aaa-original-pull-secret podName:0af3fecb-e79a-4b70-ba3a-158278da3aaa nodeName:}" failed. No retries permitted until 2026-07-28 17:09:33.352002287 +0000 UTC m=+34.071829984 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0af3fecb-e79a-4b70-ba3a-158278da3aaa-original-pull-secret") pod "global-pull-secret-syncer-w95vd" (UID: "0af3fecb-e79a-4b70-ba3a-158278da3aaa") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:09:17.352487 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:17.352046 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs podName:c0d05255-0573-4806-84b3-c03ff36cea82 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:33.352034894 +0000 UTC m=+34.071862584 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs") pod "network-metrics-daemon-w2qls" (UID: "c0d05255-0573-4806-84b3-c03ff36cea82") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:09:17.352487 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:17.352062 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab2a66ee-852a-4d34-88fc-1000fef6b92c-kube-api-access-rv27t podName:ab2a66ee-852a-4d34-88fc-1000fef6b92c nodeName:}" failed. No retries permitted until 2026-07-28 17:09:33.352054044 +0000 UTC m=+34.071881731 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rv27t" (UniqueName: "kubernetes.io/projected/ab2a66ee-852a-4d34-88fc-1000fef6b92c-kube-api-access-rv27t") pod "network-check-target-6kkbm" (UID: "ab2a66ee-852a-4d34-88fc-1000fef6b92c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:09:17.779795 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:17.779699 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:17.779795 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:17.779699 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:17.780000 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:17.779849 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kkbm" podUID="ab2a66ee-852a-4d34-88fc-1000fef6b92c" Jul 28 17:09:17.780000 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:17.779699 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:17.780000 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:17.779909 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w95vd" podUID="0af3fecb-e79a-4b70-ba3a-158278da3aaa" Jul 28 17:09:17.780000 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:17.779974 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2qls" podUID="c0d05255-0573-4806-84b3-c03ff36cea82" Jul 28 17:09:19.781152 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:19.781108 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:19.781686 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:19.781238 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w95vd" podUID="0af3fecb-e79a-4b70-ba3a-158278da3aaa" Jul 28 17:09:19.781686 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:19.781280 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:19.781686 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:19.781316 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:19.781686 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:19.781400 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2qls" podUID="c0d05255-0573-4806-84b3-c03ff36cea82" Jul 28 17:09:19.781686 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:19.781474 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kkbm" podUID="ab2a66ee-852a-4d34-88fc-1000fef6b92c" Jul 28 17:09:20.880249 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:20.879740 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" event={"ID":"a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3","Type":"ContainerStarted","Data":"d891f9350dbc7f8ce96a05cda03033dd978701b6f483ee5a534ec3ff750a9c77"} Jul 28 17:09:20.884114 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:20.883510 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xwqnd" event={"ID":"50a88180-0620-4676-ac16-95806564a133","Type":"ContainerStarted","Data":"fc105fd9288527a431a43370dfcc7a7e2844045b59a3759916c25801e96390aa"} Jul 28 17:09:20.886580 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:20.886508 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nmkgg" event={"ID":"0cbdabd3-4530-4336-b84e-4b1bb7f75650","Type":"ContainerStarted","Data":"2c26f07fde01cea4e9ec321d4c090311ea9cffae1dc78f3f3debdb9d835c9e13"} Jul 28 17:09:20.889470 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:20.889121 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6vlbb" event={"ID":"7fc05f85-4048-4422-8800-59ca1e11e254","Type":"ContainerStarted","Data":"f0e06f3cc4fcc51a87763f7764c63806e40c9a38635a0d75db9a1ec3f3e0b527"} Jul 28 17:09:20.897232 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:20.897154 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qwhwb" podStartSLOduration=3.624132046 podStartE2EDuration="21.897134164s" totalImagesPullingTime="18.273002118s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:20.896558349 +0000 UTC m=+21.616386059" watchObservedRunningTime="2026-07-28 17:09:20.897134164 +0000 UTC m=+21.616961874" Jul 28 17:09:20.897399 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:20.897366 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-34.ec2.internal" podStartSLOduration=20.897353813 podStartE2EDuration="20.897353813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:09:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:03.861159469 +0000 UTC m=+4.580987177" watchObservedRunningTime="2026-07-28 17:09:20.897353813 +0000 UTC m=+21.617181527" Jul 28 17:09:20.916400 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:20.915461 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nmkgg" podStartSLOduration=11.708282239 podStartE2EDuration="21.915437345s" totalImagesPullingTime="10.207155106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:20.914200518 +0000 UTC m=+21.634028232" watchObservedRunningTime="2026-07-28 17:09:20.915437345 +0000 UTC m=+21.635265056" Jul 28 17:09:20.932059 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:20.932005 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-xwqnd" podStartSLOduration=3.652451521 podStartE2EDuration="21.931988702s" totalImagesPullingTime="18.279537181s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:20.931836493 +0000 UTC m=+21.651664198" watchObservedRunningTime="2026-07-28 17:09:20.931988702 +0000 UTC m=+21.651816405" Jul 28 17:09:20.951940 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:20.951892 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6vlbb" podStartSLOduration=3.689748635 podStartE2EDuration="21.951875557s" totalImagesPullingTime="18.262126922s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:20.951684085 +0000 UTC m=+21.671511797" watchObservedRunningTime="2026-07-28 17:09:20.951875557 +0000 UTC m=+21.671703267" Jul 28 17:09:21.780266 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:21.779989 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:21.780459 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:21.779995 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:21.780459 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:21.780402 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w95vd" podUID="0af3fecb-e79a-4b70-ba3a-158278da3aaa" Jul 28 17:09:21.780459 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:21.780438 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2qls" podUID="c0d05255-0573-4806-84b3-c03ff36cea82" Jul 28 17:09:21.780459 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:21.780042 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:21.780671 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:21.780507 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kkbm" podUID="ab2a66ee-852a-4d34-88fc-1000fef6b92c" Jul 28 17:09:21.892529 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:21.892493 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dthjv" event={"ID":"507a907d-af5f-4815-93c4-0efe8cc04000","Type":"ContainerStarted","Data":"625ef0e569a74fc1491014cef6fe307a0f0378f6be44c38cda3e5ea89a296318"} Jul 28 17:09:21.895577 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:21.895550 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" event={"ID":"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e","Type":"ContainerStarted","Data":"0bff7cca91efb8002a3f6da935557cb1ae6fd6f6d281b7f7889db8f6e090a2ab"} Jul 28 17:09:21.895731 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:21.895629 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" event={"ID":"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e","Type":"ContainerStarted","Data":"dba214953661ef5e768a4cfe8f946864b236263fa03f7335c8333bfb4daa45ad"} Jul 28 17:09:21.895731 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:21.895677 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" event={"ID":"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e","Type":"ContainerStarted","Data":"e0998f84f02db6cde54c8fe790695a5f278fced122b7154205b7f74c0630e81c"} Jul 28 17:09:21.895731 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:21.895720 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" event={"ID":"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e","Type":"ContainerStarted","Data":"c876925a9768914d62a6df331a41fe1fad2854b16e6a5e1aec48615e2a2a3258"} Jul 28 17:09:21.895915 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:21.895782 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" event={"ID":"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e","Type":"ContainerStarted","Data":"6e3eafecf8d9bfb8c27e781d9f93d06934e971cb372c3c6fdb065dcefbb6b665"} Jul 28 17:09:21.895915 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:21.895830 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" event={"ID":"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e","Type":"ContainerStarted","Data":"4cc6e1154f415d6274740c13ceea1a98a5e1aa5c89bdf5470d0724d414b76cd7"} Jul 28 17:09:21.897149 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:21.897127 2571 generic.go:356] "Generic (PLEG): container finished" podID="52ae3ebe-36b6-44ca-94aa-ee34533ca203" containerID="5dcd2f3e048dae781c949593714e446b7cc976c69863b5e01f0dc3780d50b340" exitCode=0 Jul 28 17:09:21.897241 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:21.897197 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v9cpp" event={"ID":"52ae3ebe-36b6-44ca-94aa-ee34533ca203","Type":"ContainerDied","Data":"5dcd2f3e048dae781c949593714e446b7cc976c69863b5e01f0dc3780d50b340"} Jul 28 17:09:21.898678 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:21.898654 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" event={"ID":"bd1b17ab-76be-4cfe-bab5-6bea59920888","Type":"ContainerStarted","Data":"9b0b3fa362b0cb4ebbe95b2509d0b96cab31fa6f245c54d746f80b48b7d08c25"} Jul 28 17:09:21.908978 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:21.908930 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dthjv" podStartSLOduration=4.642678882 podStartE2EDuration="22.908913268s" totalImagesPullingTime="18.266234386s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:21.908674163 +0000 UTC m=+22.628501875" watchObservedRunningTime="2026-07-28 17:09:21.908913268 +0000 UTC m=+22.628740978" Jul 28 17:09:22.310356 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:22.310327 2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 28 17:09:22.682685 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:22.682579 2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T17:09:22.310349238Z","UUID":"ce4c960d-7c09-44c2-95d2-c680b8c75bb6","Handler":null,"Name":"","Endpoint":""} Jul 28 17:09:22.684342 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:22.684309 2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 28 17:09:22.684546 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:22.684351 2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 28 17:09:22.903281 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:22.903238 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" event={"ID":"bd1b17ab-76be-4cfe-bab5-6bea59920888","Type":"ContainerStarted","Data":"76810565bd15d1eecfc5f3ec2ee20427e66c783b04d949d6813cf6495f989a6e"} Jul 28 17:09:22.904929 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:22.904889 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-knlrx" event={"ID":"8f0df316-8fe9-4dcf-ab0c-c502dd565001","Type":"ContainerStarted","Data":"7d001f9a9da64e01da7d4e58296b90ca9472a9c5b069891a637c36dcf7446571"} Jul 28 17:09:23.779791 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:23.779663 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:23.779791 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:23.779668 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:23.780080 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:23.779804 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kkbm" podUID="ab2a66ee-852a-4d34-88fc-1000fef6b92c" Jul 28 17:09:23.780080 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:23.779815 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:23.780080 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:23.779917 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w95vd" podUID="0af3fecb-e79a-4b70-ba3a-158278da3aaa" Jul 28 17:09:23.780080 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:23.780015 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2qls" podUID="c0d05255-0573-4806-84b3-c03ff36cea82" Jul 28 17:09:23.911142 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:23.911045 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" event={"ID":"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e","Type":"ContainerStarted","Data":"e1a11ccfa9ec38584b2bd155c30c757a23e2fa32b3e120eb669a89fad2d75cda"} Jul 28 17:09:24.864239 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:24.864198 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nmkgg" Jul 28 17:09:24.865109 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:24.865084 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nmkgg" Jul 28 17:09:24.884448 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:24.884394 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-knlrx" podStartSLOduration=7.615161111 podStartE2EDuration="25.884375146s" totalImagesPullingTime="18.269214035s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:22.921641612 +0000 UTC m=+23.641469324" watchObservedRunningTime="2026-07-28 17:09:24.884375146 +0000 UTC m=+25.604202855" Jul 28 17:09:24.915510 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:24.915472 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" event={"ID":"bd1b17ab-76be-4cfe-bab5-6bea59920888","Type":"ContainerStarted","Data":"b7cde2a9cff1db98c8fabec0b5583507002c01bd6995fabc193d2a6e58c581b8"} Jul 28 17:09:24.916005 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:24.915719 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nmkgg" Jul 28 17:09:24.916990 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:24.916967 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nmkgg" Jul 28 17:09:24.970541 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:24.970318 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pz4tb" podStartSLOduration=4.679216546 podStartE2EDuration="25.970296312s" totalImagesPullingTime="21.291079766s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:24.943656418 +0000 UTC m=+25.663484130" watchObservedRunningTime="2026-07-28 17:09:24.970296312 +0000 UTC m=+25.690124022" Jul 28 17:09:25.780331 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:25.780291 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:25.780511 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:25.780296 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:25.780511 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:25.780425 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2qls" podUID="c0d05255-0573-4806-84b3-c03ff36cea82" Jul 28 17:09:25.780605 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:25.780296 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:25.780605 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:25.780527 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w95vd" podUID="0af3fecb-e79a-4b70-ba3a-158278da3aaa" Jul 28 17:09:25.780692 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:25.780640 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kkbm" podUID="ab2a66ee-852a-4d34-88fc-1000fef6b92c" Jul 28 17:09:26.923812 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:26.923747 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" event={"ID":"bfc273ea-75a3-41b1-9158-6a8e00aeaa2e","Type":"ContainerStarted","Data":"1cc8a824ffaab99b9ab3ab7ace7a23e1f3c24135657507b26961991ece753133"} Jul 28 17:09:26.924740 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:26.924023 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:26.925924 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:26.925894 2571 generic.go:356] "Generic (PLEG): container finished" podID="52ae3ebe-36b6-44ca-94aa-ee34533ca203" containerID="38ac6f63ebf8f1cfd6010c80badcde994c7b135015244577a94b0e87a187b299" exitCode=0 Jul 28 17:09:26.926073 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:26.925947 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v9cpp" event={"ID":"52ae3ebe-36b6-44ca-94aa-ee34533ca203","Type":"ContainerDied","Data":"38ac6f63ebf8f1cfd6010c80badcde994c7b135015244577a94b0e87a187b299"} Jul 28 17:09:26.940997 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:26.940969 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:26.956826 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:26.956748 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" podStartSLOduration=9.532235359 podStartE2EDuration="27.95673067s" totalImagesPullingTime="18.424495311s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:26.956459611 +0000 UTC m=+27.676287322" watchObservedRunningTime="2026-07-28 17:09:26.95673067 +0000 UTC m=+27.676558379" Jul 28 17:09:27.779793 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:27.779691 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:27.779793 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:27.779735 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:27.779793 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:27.779691 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:27.780147 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:27.779871 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w95vd" podUID="0af3fecb-e79a-4b70-ba3a-158278da3aaa" Jul 28 17:09:27.780147 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:27.780005 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2qls" podUID="c0d05255-0573-4806-84b3-c03ff36cea82" Jul 28 17:09:27.780147 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:27.780086 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kkbm" podUID="ab2a66ee-852a-4d34-88fc-1000fef6b92c" Jul 28 17:09:27.929319 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:27.929245 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:27.929319 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:27.929313 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:27.944989 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:27.944958 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:09:28.353539 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:28.353507 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6kkbm"] Jul 28 17:09:28.353705 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:28.353606 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:28.353705 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:28.353686 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kkbm" podUID="ab2a66ee-852a-4d34-88fc-1000fef6b92c" Jul 28 17:09:28.357090 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:28.357060 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w2qls"] Jul 28 17:09:28.357253 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:28.357191 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:28.357338 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:28.357314 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2qls" podUID="c0d05255-0573-4806-84b3-c03ff36cea82" Jul 28 17:09:28.357800 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:28.357752 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w95vd"] Jul 28 17:09:28.357871 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:28.357857 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:28.357948 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:28.357931 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w95vd" podUID="0af3fecb-e79a-4b70-ba3a-158278da3aaa" Jul 28 17:09:28.933330 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:28.933295 2571 generic.go:356] "Generic (PLEG): container finished" podID="52ae3ebe-36b6-44ca-94aa-ee34533ca203" containerID="670254b142395eff262f2284f9afd0d91acf1c5fce713ac025e836803a5841b9" exitCode=0 Jul 28 17:09:28.933877 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:28.933381 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v9cpp" event={"ID":"52ae3ebe-36b6-44ca-94aa-ee34533ca203","Type":"ContainerDied","Data":"670254b142395eff262f2284f9afd0d91acf1c5fce713ac025e836803a5841b9"} Jul 28 17:09:29.780872 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:29.780834 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:29.781068 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:29.780937 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:29.781128 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:29.780940 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w95vd" podUID="0af3fecb-e79a-4b70-ba3a-158278da3aaa" Jul 28 17:09:29.781128 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:29.781058 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2qls" podUID="c0d05255-0573-4806-84b3-c03ff36cea82" Jul 28 17:09:29.781202 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:29.781134 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:29.781280 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:29.781256 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kkbm" podUID="ab2a66ee-852a-4d34-88fc-1000fef6b92c" Jul 28 17:09:30.940975 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:30.940757 2571 generic.go:356] "Generic (PLEG): container finished" podID="52ae3ebe-36b6-44ca-94aa-ee34533ca203" containerID="a7f27b246a4ffa05ad8f7090ab7e7e7d1417ff8f9f98239408739d3cc6b915be" exitCode=0 Jul 28 17:09:30.940975 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:30.940800 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v9cpp" event={"ID":"52ae3ebe-36b6-44ca-94aa-ee34533ca203","Type":"ContainerDied","Data":"a7f27b246a4ffa05ad8f7090ab7e7e7d1417ff8f9f98239408739d3cc6b915be"} Jul 28 17:09:31.779859 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:31.779824 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:31.780051 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:31.779838 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:31.780051 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:31.779951 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w95vd" podUID="0af3fecb-e79a-4b70-ba3a-158278da3aaa" Jul 28 17:09:31.780160 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:31.780119 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w2qls" podUID="c0d05255-0573-4806-84b3-c03ff36cea82" Jul 28 17:09:31.780223 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:31.780196 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:31.780356 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:31.780332 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6kkbm" podUID="ab2a66ee-852a-4d34-88fc-1000fef6b92c" Jul 28 17:09:33.376914 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.376880 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv27t\" (UniqueName: \"kubernetes.io/projected/ab2a66ee-852a-4d34-88fc-1000fef6b92c-kube-api-access-rv27t\") pod \"network-check-target-6kkbm\" (UID: \"ab2a66ee-852a-4d34-88fc-1000fef6b92c\") " pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:33.377494 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.376934 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0af3fecb-e79a-4b70-ba3a-158278da3aaa-original-pull-secret\") pod \"global-pull-secret-syncer-w95vd\" (UID: \"0af3fecb-e79a-4b70-ba3a-158278da3aaa\") " pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:33.377494 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:33.377045 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:09:33.377494 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:33.377073 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:09:33.377494 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:33.377098 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:09:33.377494 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:33.377105 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0af3fecb-e79a-4b70-ba3a-158278da3aaa-original-pull-secret podName:0af3fecb-e79a-4b70-ba3a-158278da3aaa nodeName:}" failed. No retries permitted until 2026-07-28 17:10:05.377087073 +0000 UTC m=+66.096914777 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0af3fecb-e79a-4b70-ba3a-158278da3aaa-original-pull-secret") pod "global-pull-secret-syncer-w95vd" (UID: "0af3fecb-e79a-4b70-ba3a-158278da3aaa") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:09:33.377494 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:33.377112 2571 projected.go:196] Error preparing data for projected volume kube-api-access-rv27t for pod openshift-network-diagnostics/network-check-target-6kkbm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:09:33.377494 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.377124 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs\") pod \"network-metrics-daemon-w2qls\" (UID: \"c0d05255-0573-4806-84b3-c03ff36cea82\") " pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:33.377494 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:33.377217 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab2a66ee-852a-4d34-88fc-1000fef6b92c-kube-api-access-rv27t podName:ab2a66ee-852a-4d34-88fc-1000fef6b92c nodeName:}" failed. No retries permitted until 2026-07-28 17:10:05.377200651 +0000 UTC m=+66.097028354 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rv27t" (UniqueName: "kubernetes.io/projected/ab2a66ee-852a-4d34-88fc-1000fef6b92c-kube-api-access-rv27t") pod "network-check-target-6kkbm" (UID: "ab2a66ee-852a-4d34-88fc-1000fef6b92c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:09:33.377494 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:33.377224 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:09:33.377494 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:33.377266 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs podName:c0d05255-0573-4806-84b3-c03ff36cea82 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:05.377254152 +0000 UTC m=+66.097081841 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs") pod "network-metrics-daemon-w2qls" (UID: "c0d05255-0573-4806-84b3-c03ff36cea82") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:09:33.650667 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.650634 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-34.ec2.internal" event="NodeReady" Jul 28 17:09:33.650875 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.650785 2571 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 28 17:09:33.700218 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.700185 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qjmfr"] Jul 28 17:09:33.703202 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.703168 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qjmfr" Jul 28 17:09:33.703544 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.703508 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4nnr6"] Jul 28 17:09:33.705209 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.705187 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hwxn2\"" Jul 28 17:09:33.705442 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.705420 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 28 17:09:33.705442 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.705436 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4nnr6" Jul 28 17:09:33.705612 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.705460 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 28 17:09:33.711214 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.711173 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 28 17:09:33.711214 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.711214 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jfwmx\"" Jul 28 17:09:33.711416 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.711191 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 28 17:09:33.711746 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.711474 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 28 17:09:33.716655 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.716427 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qjmfr"] Jul 28 17:09:33.717532 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.717503 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4nnr6"] Jul 28 17:09:33.779168 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.779122 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51754089-3d17-471f-85f1-5b493bed2336-tmp-dir\") pod \"dns-default-qjmfr\" (UID: \"51754089-3d17-471f-85f1-5b493bed2336\") " pod="openshift-dns/dns-default-qjmfr" Jul 28 17:09:33.779353 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.779179 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jhmf\" (UniqueName: \"kubernetes.io/projected/91c276cf-bc64-466b-a82d-3cdbf4ccc495-kube-api-access-5jhmf\") pod \"ingress-canary-4nnr6\" (UID: \"91c276cf-bc64-466b-a82d-3cdbf4ccc495\") " pod="openshift-ingress-canary/ingress-canary-4nnr6" Jul 28 17:09:33.779353 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.779213 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/51754089-3d17-471f-85f1-5b493bed2336-config-volume\") pod \"dns-default-qjmfr\" (UID: \"51754089-3d17-471f-85f1-5b493bed2336\") " pod="openshift-dns/dns-default-qjmfr" Jul 28 17:09:33.779353 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.779324 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:09:33.779353 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.779324 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91c276cf-bc64-466b-a82d-3cdbf4ccc495-cert\") pod \"ingress-canary-4nnr6\" (UID: \"91c276cf-bc64-466b-a82d-3cdbf4ccc495\") " pod="openshift-ingress-canary/ingress-canary-4nnr6" Jul 28 17:09:33.779513 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.779385 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls\") pod \"dns-default-qjmfr\" (UID: \"51754089-3d17-471f-85f1-5b493bed2336\") " pod="openshift-dns/dns-default-qjmfr" Jul 28 17:09:33.779513 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.779423 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbb8q\" (UniqueName: \"kubernetes.io/projected/51754089-3d17-471f-85f1-5b493bed2336-kube-api-access-sbb8q\") pod \"dns-default-qjmfr\" (UID: \"51754089-3d17-471f-85f1-5b493bed2336\") " pod="openshift-dns/dns-default-qjmfr" Jul 28 17:09:33.779513 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.779471 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:09:33.779631 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.779589 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:09:33.787216 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.787187 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 17:09:33.787333 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.787315 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 17:09:33.790753 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.790715 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-99hnw\"" Jul 28 17:09:33.790912 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.790799 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 17:09:33.790912 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.790837 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 17:09:33.791074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.791030 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8h5hq\"" Jul 28 17:09:33.880397 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.880362 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51754089-3d17-471f-85f1-5b493bed2336-tmp-dir\") pod \"dns-default-qjmfr\" (UID: \"51754089-3d17-471f-85f1-5b493bed2336\") " pod="openshift-dns/dns-default-qjmfr" Jul 28 17:09:33.880397 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.880398 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jhmf\" (UniqueName: \"kubernetes.io/projected/91c276cf-bc64-466b-a82d-3cdbf4ccc495-kube-api-access-5jhmf\") pod \"ingress-canary-4nnr6\" (UID: \"91c276cf-bc64-466b-a82d-3cdbf4ccc495\") " pod="openshift-ingress-canary/ingress-canary-4nnr6" Jul 28 17:09:33.880636 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.880420 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/51754089-3d17-471f-85f1-5b493bed2336-config-volume\") pod \"dns-default-qjmfr\" (UID: \"51754089-3d17-471f-85f1-5b493bed2336\") " pod="openshift-dns/dns-default-qjmfr" Jul 28 17:09:33.880636 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.880441 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91c276cf-bc64-466b-a82d-3cdbf4ccc495-cert\") pod \"ingress-canary-4nnr6\" (UID: \"91c276cf-bc64-466b-a82d-3cdbf4ccc495\") " pod="openshift-ingress-canary/ingress-canary-4nnr6" Jul 28 17:09:33.880636 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.880483 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls\") pod \"dns-default-qjmfr\" (UID: \"51754089-3d17-471f-85f1-5b493bed2336\") " pod="openshift-dns/dns-default-qjmfr" Jul 28 17:09:33.880636 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.880508 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbb8q\" (UniqueName: \"kubernetes.io/projected/51754089-3d17-471f-85f1-5b493bed2336-kube-api-access-sbb8q\") pod \"dns-default-qjmfr\" (UID: \"51754089-3d17-471f-85f1-5b493bed2336\") " pod="openshift-dns/dns-default-qjmfr" Jul 28 17:09:33.880855 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:33.880630 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:09:33.880855 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:33.880703 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls podName:51754089-3d17-471f-85f1-5b493bed2336 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:34.380681294 +0000 UTC m=+35.100508996 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls") pod "dns-default-qjmfr" (UID: "51754089-3d17-471f-85f1-5b493bed2336") : secret "dns-default-metrics-tls" not found Jul 28 17:09:33.880855 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.880804 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51754089-3d17-471f-85f1-5b493bed2336-tmp-dir\") pod \"dns-default-qjmfr\" (UID: \"51754089-3d17-471f-85f1-5b493bed2336\") " pod="openshift-dns/dns-default-qjmfr" Jul 28 17:09:33.881206 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.881186 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/51754089-3d17-471f-85f1-5b493bed2336-config-volume\") pod \"dns-default-qjmfr\" (UID: \"51754089-3d17-471f-85f1-5b493bed2336\") " pod="openshift-dns/dns-default-qjmfr" Jul 28 17:09:33.885284 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.885259 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91c276cf-bc64-466b-a82d-3cdbf4ccc495-cert\") pod \"ingress-canary-4nnr6\" (UID: \"91c276cf-bc64-466b-a82d-3cdbf4ccc495\") " pod="openshift-ingress-canary/ingress-canary-4nnr6" Jul 28 17:09:33.892293 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.892265 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbb8q\" (UniqueName: \"kubernetes.io/projected/51754089-3d17-471f-85f1-5b493bed2336-kube-api-access-sbb8q\") pod \"dns-default-qjmfr\" (UID: \"51754089-3d17-471f-85f1-5b493bed2336\") " pod="openshift-dns/dns-default-qjmfr" Jul 28 17:09:33.892440 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:33.892342 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jhmf\" (UniqueName: \"kubernetes.io/projected/91c276cf-bc64-466b-a82d-3cdbf4ccc495-kube-api-access-5jhmf\") pod \"ingress-canary-4nnr6\" (UID: \"91c276cf-bc64-466b-a82d-3cdbf4ccc495\") " pod="openshift-ingress-canary/ingress-canary-4nnr6" Jul 28 17:09:34.023284 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:34.023199 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4nnr6" Jul 28 17:09:34.385572 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:34.385535 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls\") pod \"dns-default-qjmfr\" (UID: \"51754089-3d17-471f-85f1-5b493bed2336\") " pod="openshift-dns/dns-default-qjmfr" Jul 28 17:09:34.386375 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:34.385707 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:09:34.386375 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:34.385862 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls podName:51754089-3d17-471f-85f1-5b493bed2336 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:35.385835192 +0000 UTC m=+36.105662884 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls") pod "dns-default-qjmfr" (UID: "51754089-3d17-471f-85f1-5b493bed2336") : secret "dns-default-metrics-tls" not found Jul 28 17:09:35.097500 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:35.097294 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4nnr6"] Jul 28 17:09:35.101722 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:09:35.101698 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91c276cf_bc64_466b_a82d_3cdbf4ccc495.slice/crio-5ab4b27c5e9a54d3ca167f1c8f7a1f334556e3df3578dd9a84227477f0886dfc WatchSource:0}: Error finding container 5ab4b27c5e9a54d3ca167f1c8f7a1f334556e3df3578dd9a84227477f0886dfc: Status 404 returned error can't find the container with id 5ab4b27c5e9a54d3ca167f1c8f7a1f334556e3df3578dd9a84227477f0886dfc Jul 28 17:09:35.393068 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:35.393029 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls\") pod \"dns-default-qjmfr\" (UID: \"51754089-3d17-471f-85f1-5b493bed2336\") " pod="openshift-dns/dns-default-qjmfr" Jul 28 17:09:35.393446 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:35.393196 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:09:35.393446 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:35.393265 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls podName:51754089-3d17-471f-85f1-5b493bed2336 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:37.393250279 +0000 UTC m=+38.113077967 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls") pod "dns-default-qjmfr" (UID: "51754089-3d17-471f-85f1-5b493bed2336") : secret "dns-default-metrics-tls" not found Jul 28 17:09:35.955880 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:35.955842 2571 generic.go:356] "Generic (PLEG): container finished" podID="52ae3ebe-36b6-44ca-94aa-ee34533ca203" containerID="0a47d69bf36dfac65e8738f95652e7d71f8e1288674f72bb9732867d72b25210" exitCode=0 Jul 28 17:09:35.956047 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:35.955919 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v9cpp" event={"ID":"52ae3ebe-36b6-44ca-94aa-ee34533ca203","Type":"ContainerDied","Data":"0a47d69bf36dfac65e8738f95652e7d71f8e1288674f72bb9732867d72b25210"} Jul 28 17:09:35.957288 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:35.956974 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4nnr6" event={"ID":"91c276cf-bc64-466b-a82d-3cdbf4ccc495","Type":"ContainerStarted","Data":"5ab4b27c5e9a54d3ca167f1c8f7a1f334556e3df3578dd9a84227477f0886dfc"} Jul 28 17:09:36.963422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:36.963384 2571 generic.go:356] "Generic (PLEG): container finished" podID="52ae3ebe-36b6-44ca-94aa-ee34533ca203" containerID="f8844c3d249cff4680aab0f0a361ea0f96ea43f547320c6aad90c6de84204f65" exitCode=0 Jul 28 17:09:36.963910 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:36.963449 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v9cpp" event={"ID":"52ae3ebe-36b6-44ca-94aa-ee34533ca203","Type":"ContainerDied","Data":"f8844c3d249cff4680aab0f0a361ea0f96ea43f547320c6aad90c6de84204f65"} Jul 28 17:09:37.408741 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:37.408698 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls\") pod \"dns-default-qjmfr\" (UID: \"51754089-3d17-471f-85f1-5b493bed2336\") " pod="openshift-dns/dns-default-qjmfr" Jul 28 17:09:37.408931 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:37.408898 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:09:37.408994 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:37.408984 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls podName:51754089-3d17-471f-85f1-5b493bed2336 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:41.408962143 +0000 UTC m=+42.128789838 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls") pod "dns-default-qjmfr" (UID: "51754089-3d17-471f-85f1-5b493bed2336") : secret "dns-default-metrics-tls" not found Jul 28 17:09:37.970730 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:37.970463 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v9cpp" event={"ID":"52ae3ebe-36b6-44ca-94aa-ee34533ca203","Type":"ContainerStarted","Data":"353b4c733d18b50882383327b089540a3ff1bf8a1c0c265d19979052131746f4"} Jul 28 17:09:38.006892 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:38.006782 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-v9cpp" podStartSLOduration=10.105834753 podStartE2EDuration="39.006748098s" totalImagesPullingTime="28.900913345s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:59 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:38.006637155 +0000 UTC m=+38.726464866" watchObservedRunningTime="2026-07-28 17:09:38.006748098 +0000 UTC m=+38.726575807" Jul 28 17:09:38.974938 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:38.974901 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4nnr6" event={"ID":"91c276cf-bc64-466b-a82d-3cdbf4ccc495","Type":"ContainerStarted","Data":"e9fb2c26becac62edcae4d4fa48c43be253931dfd47089c6c0cc2ddb9103c081"} Jul 28 17:09:38.997821 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:38.997747 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4nnr6" podStartSLOduration=2.5549846929999998 podStartE2EDuration="5.997718472s" totalImagesPullingTime="3.442733779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:09:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:38.996947608 +0000 UTC m=+39.716775322" watchObservedRunningTime="2026-07-28 17:09:38.997718472 +0000 UTC m=+39.717546182" Jul 28 17:09:41.437774 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:41.437726 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls\") pod \"dns-default-qjmfr\" (UID: \"51754089-3d17-471f-85f1-5b493bed2336\") " pod="openshift-dns/dns-default-qjmfr" Jul 28 17:09:41.438144 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:41.437893 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:09:41.438144 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:41.437963 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls podName:51754089-3d17-471f-85f1-5b493bed2336 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:49.437945214 +0000 UTC m=+50.157772905 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls") pod "dns-default-qjmfr" (UID: "51754089-3d17-471f-85f1-5b493bed2336") : secret "dns-default-metrics-tls" not found Jul 28 17:09:49.496242 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:49.496201 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls\") pod \"dns-default-qjmfr\" (UID: \"51754089-3d17-471f-85f1-5b493bed2336\") " pod="openshift-dns/dns-default-qjmfr" Jul 28 17:09:49.496714 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:49.496320 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:09:49.496714 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:09:49.496380 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls podName:51754089-3d17-471f-85f1-5b493bed2336 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:05.496363032 +0000 UTC m=+66.216190740 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls") pod "dns-default-qjmfr" (UID: "51754089-3d17-471f-85f1-5b493bed2336") : secret "dns-default-metrics-tls" not found Jul 28 17:09:59.946855 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:09:59.946820 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dlxqx" Jul 28 17:10:05.411049 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:05.410997 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv27t\" (UniqueName: \"kubernetes.io/projected/ab2a66ee-852a-4d34-88fc-1000fef6b92c-kube-api-access-rv27t\") pod \"network-check-target-6kkbm\" (UID: \"ab2a66ee-852a-4d34-88fc-1000fef6b92c\") " pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:10:05.411049 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:05.411055 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0af3fecb-e79a-4b70-ba3a-158278da3aaa-original-pull-secret\") pod \"global-pull-secret-syncer-w95vd\" (UID: \"0af3fecb-e79a-4b70-ba3a-158278da3aaa\") " pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:10:05.411492 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:05.411082 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs\") pod \"network-metrics-daemon-w2qls\" (UID: \"c0d05255-0573-4806-84b3-c03ff36cea82\") " pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:10:05.413774 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:05.413732 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 17:10:05.413879 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:05.413740 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 17:10:05.413879 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:05.413742 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 17:10:05.421695 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:10:05.421672 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 28 17:10:05.421827 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:10:05.421775 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs podName:c0d05255-0573-4806-84b3-c03ff36cea82 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:09.421732419 +0000 UTC m=+130.141560119 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs") pod "network-metrics-daemon-w2qls" (UID: "c0d05255-0573-4806-84b3-c03ff36cea82") : secret "metrics-daemon-secret" not found Jul 28 17:10:05.424110 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:05.424091 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 17:10:05.425682 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:05.425661 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0af3fecb-e79a-4b70-ba3a-158278da3aaa-original-pull-secret\") pod \"global-pull-secret-syncer-w95vd\" (UID: \"0af3fecb-e79a-4b70-ba3a-158278da3aaa\") " pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:10:05.434701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:05.434671 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv27t\" (UniqueName: \"kubernetes.io/projected/ab2a66ee-852a-4d34-88fc-1000fef6b92c-kube-api-access-rv27t\") pod \"network-check-target-6kkbm\" (UID: \"ab2a66ee-852a-4d34-88fc-1000fef6b92c\") " pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:10:05.512205 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:05.512171 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls\") pod \"dns-default-qjmfr\" (UID: \"51754089-3d17-471f-85f1-5b493bed2336\") " pod="openshift-dns/dns-default-qjmfr" Jul 28 17:10:05.512384 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:10:05.512304 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:10:05.512384 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:10:05.512361 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls podName:51754089-3d17-471f-85f1-5b493bed2336 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:37.512343634 +0000 UTC m=+98.232171329 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls") pod "dns-default-qjmfr" (UID: "51754089-3d17-471f-85f1-5b493bed2336") : secret "dns-default-metrics-tls" not found Jul 28 17:10:05.591381 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:05.591342 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w95vd" Jul 28 17:10:05.608093 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:05.608059 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8h5hq\"" Jul 28 17:10:05.616810 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:05.616780 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:10:05.748946 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:05.748886 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w95vd"] Jul 28 17:10:05.752741 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:10:05.752713 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0af3fecb_e79a_4b70_ba3a_158278da3aaa.slice/crio-f81b6e475036f0711dfdb072a02e09ec7a9a23cb894967639af33dcc81164c52 WatchSource:0}: Error finding container f81b6e475036f0711dfdb072a02e09ec7a9a23cb894967639af33dcc81164c52: Status 404 returned error can't find the container with id f81b6e475036f0711dfdb072a02e09ec7a9a23cb894967639af33dcc81164c52 Jul 28 17:10:05.766754 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:05.766705 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6kkbm"] Jul 28 17:10:05.770385 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:10:05.770358 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab2a66ee_852a_4d34_88fc_1000fef6b92c.slice/crio-5cfad5d1613e57d06ccffbc7c8299d39a05ef7a349a50d183a2007511076d0a6 WatchSource:0}: Error finding container 5cfad5d1613e57d06ccffbc7c8299d39a05ef7a349a50d183a2007511076d0a6: Status 404 returned error can't find the container with id 5cfad5d1613e57d06ccffbc7c8299d39a05ef7a349a50d183a2007511076d0a6 Jul 28 17:10:06.040601 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:06.040511 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6kkbm" event={"ID":"ab2a66ee-852a-4d34-88fc-1000fef6b92c","Type":"ContainerStarted","Data":"5cfad5d1613e57d06ccffbc7c8299d39a05ef7a349a50d183a2007511076d0a6"} Jul 28 17:10:06.041555 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:06.041531 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w95vd" event={"ID":"0af3fecb-e79a-4b70-ba3a-158278da3aaa","Type":"ContainerStarted","Data":"f81b6e475036f0711dfdb072a02e09ec7a9a23cb894967639af33dcc81164c52"} Jul 28 17:10:11.057603 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:11.057562 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6kkbm" event={"ID":"ab2a66ee-852a-4d34-88fc-1000fef6b92c","Type":"ContainerStarted","Data":"e0ad136d8c8ddaa082072840d2af7ec08874291060a82634bd2fc0dafe4e80ed"} Jul 28 17:10:11.058072 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:11.057660 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:10:11.059043 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:11.059022 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w95vd" event={"ID":"0af3fecb-e79a-4b70-ba3a-158278da3aaa","Type":"ContainerStarted","Data":"4d7db36a4758948991193f176fc67abb1178b6213c82dbd88eec8c2e40b83adb"} Jul 28 17:10:11.075221 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:11.075164 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6kkbm" podStartSLOduration=67.682225621 podStartE2EDuration="1m12.075145396s" totalImagesPullingTime="4.392919775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:11.074134965 +0000 UTC m=+71.793962675" watchObservedRunningTime="2026-07-28 17:10:11.075145396 +0000 UTC m=+71.794973107" Jul 28 17:10:11.091225 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:11.091172 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-w95vd" podStartSLOduration=67.67612645 podStartE2EDuration="1m12.091156387s" totalImagesPullingTime="4.415029937s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:11.090181073 +0000 UTC m=+71.810008784" watchObservedRunningTime="2026-07-28 17:10:11.091156387 +0000 UTC m=+71.810984087" Jul 28 17:10:25.196199 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.196078 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-785fd5f646-bk8mk"] Jul 28 17:10:25.198848 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.198829 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-785fd5f646-bk8mk" Jul 28 17:10:25.202200 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.202170 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 28 17:10:25.202360 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.202210 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 28 17:10:25.202360 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.202173 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 28 17:10:25.202454 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.202356 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 28 17:10:25.203063 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.203046 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx"] Jul 28 17:10:25.204941 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.204917 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" Jul 28 17:10:25.207229 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.207208 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 28 17:10:25.207629 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.207603 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 28 17:10:25.207629 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.207624 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 28 17:10:25.207801 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.207624 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 28 17:10:25.210586 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.210565 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-785fd5f646-bk8mk"] Jul 28 17:10:25.218944 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.218919 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx"] Jul 28 17:10:25.251754 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.251720 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af193b3e-ec26-4d39-b4e0-8c1bec3eea4b-tmp\") pod \"klusterlet-addon-workmgr-785fd5f646-bk8mk\" (UID: \"af193b3e-ec26-4d39-b4e0-8c1bec3eea4b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-785fd5f646-bk8mk" Jul 28 17:10:25.251956 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.251801 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/74113348-a310-4e1c-af96-465a020890d8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-54b45ffdd4-6jflx\" (UID: \"74113348-a310-4e1c-af96-465a020890d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" Jul 28 17:10:25.251956 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.251823 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/af193b3e-ec26-4d39-b4e0-8c1bec3eea4b-klusterlet-config\") pod \"klusterlet-addon-workmgr-785fd5f646-bk8mk\" (UID: \"af193b3e-ec26-4d39-b4e0-8c1bec3eea4b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-785fd5f646-bk8mk" Jul 28 17:10:25.251956 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.251839 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/74113348-a310-4e1c-af96-465a020890d8-ca\") pod \"cluster-proxy-proxy-agent-54b45ffdd4-6jflx\" (UID: \"74113348-a310-4e1c-af96-465a020890d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" Jul 28 17:10:25.251956 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.251855 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/74113348-a310-4e1c-af96-465a020890d8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-54b45ffdd4-6jflx\" (UID: \"74113348-a310-4e1c-af96-465a020890d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" Jul 28 17:10:25.251956 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.251872 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chvn8\" (UniqueName: \"kubernetes.io/projected/af193b3e-ec26-4d39-b4e0-8c1bec3eea4b-kube-api-access-chvn8\") pod \"klusterlet-addon-workmgr-785fd5f646-bk8mk\" (UID: \"af193b3e-ec26-4d39-b4e0-8c1bec3eea4b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-785fd5f646-bk8mk" Jul 28 17:10:25.251956 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.251915 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2hs2\" (UniqueName: \"kubernetes.io/projected/74113348-a310-4e1c-af96-465a020890d8-kube-api-access-k2hs2\") pod \"cluster-proxy-proxy-agent-54b45ffdd4-6jflx\" (UID: \"74113348-a310-4e1c-af96-465a020890d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" Jul 28 17:10:25.251956 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.251953 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/74113348-a310-4e1c-af96-465a020890d8-hub\") pod \"cluster-proxy-proxy-agent-54b45ffdd4-6jflx\" (UID: \"74113348-a310-4e1c-af96-465a020890d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" Jul 28 17:10:25.252199 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.251996 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/74113348-a310-4e1c-af96-465a020890d8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-54b45ffdd4-6jflx\" (UID: \"74113348-a310-4e1c-af96-465a020890d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" Jul 28 17:10:25.353215 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.353175 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/74113348-a310-4e1c-af96-465a020890d8-hub\") pod \"cluster-proxy-proxy-agent-54b45ffdd4-6jflx\" (UID: \"74113348-a310-4e1c-af96-465a020890d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" Jul 28 17:10:25.353215 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.353218 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/74113348-a310-4e1c-af96-465a020890d8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-54b45ffdd4-6jflx\" (UID: \"74113348-a310-4e1c-af96-465a020890d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" Jul 28 17:10:25.353381 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.353352 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af193b3e-ec26-4d39-b4e0-8c1bec3eea4b-tmp\") pod \"klusterlet-addon-workmgr-785fd5f646-bk8mk\" (UID: \"af193b3e-ec26-4d39-b4e0-8c1bec3eea4b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-785fd5f646-bk8mk" Jul 28 17:10:25.353472 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.353457 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/74113348-a310-4e1c-af96-465a020890d8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-54b45ffdd4-6jflx\" (UID: \"74113348-a310-4e1c-af96-465a020890d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" Jul 28 17:10:25.353507 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.353492 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/af193b3e-ec26-4d39-b4e0-8c1bec3eea4b-klusterlet-config\") pod \"klusterlet-addon-workmgr-785fd5f646-bk8mk\" (UID: \"af193b3e-ec26-4d39-b4e0-8c1bec3eea4b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-785fd5f646-bk8mk" Jul 28 17:10:25.353540 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.353516 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/74113348-a310-4e1c-af96-465a020890d8-ca\") pod \"cluster-proxy-proxy-agent-54b45ffdd4-6jflx\" (UID: \"74113348-a310-4e1c-af96-465a020890d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" Jul 28 17:10:25.353584 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.353539 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/74113348-a310-4e1c-af96-465a020890d8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-54b45ffdd4-6jflx\" (UID: \"74113348-a310-4e1c-af96-465a020890d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" Jul 28 17:10:25.353584 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.353570 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chvn8\" (UniqueName: \"kubernetes.io/projected/af193b3e-ec26-4d39-b4e0-8c1bec3eea4b-kube-api-access-chvn8\") pod \"klusterlet-addon-workmgr-785fd5f646-bk8mk\" (UID: \"af193b3e-ec26-4d39-b4e0-8c1bec3eea4b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-785fd5f646-bk8mk" Jul 28 17:10:25.353837 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.353811 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af193b3e-ec26-4d39-b4e0-8c1bec3eea4b-tmp\") pod \"klusterlet-addon-workmgr-785fd5f646-bk8mk\" (UID: \"af193b3e-ec26-4d39-b4e0-8c1bec3eea4b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-785fd5f646-bk8mk" Jul 28 17:10:25.353965 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.353944 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2hs2\" (UniqueName: \"kubernetes.io/projected/74113348-a310-4e1c-af96-465a020890d8-kube-api-access-k2hs2\") pod \"cluster-proxy-proxy-agent-54b45ffdd4-6jflx\" (UID: \"74113348-a310-4e1c-af96-465a020890d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" Jul 28 17:10:25.354258 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.354234 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/74113348-a310-4e1c-af96-465a020890d8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-54b45ffdd4-6jflx\" (UID: \"74113348-a310-4e1c-af96-465a020890d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" Jul 28 17:10:25.355938 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.355910 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/74113348-a310-4e1c-af96-465a020890d8-ca\") pod \"cluster-proxy-proxy-agent-54b45ffdd4-6jflx\" (UID: \"74113348-a310-4e1c-af96-465a020890d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" Jul 28 17:10:25.356094 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.356007 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/74113348-a310-4e1c-af96-465a020890d8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-54b45ffdd4-6jflx\" (UID: \"74113348-a310-4e1c-af96-465a020890d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" Jul 28 17:10:25.356094 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.356016 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/74113348-a310-4e1c-af96-465a020890d8-hub\") pod \"cluster-proxy-proxy-agent-54b45ffdd4-6jflx\" (UID: \"74113348-a310-4e1c-af96-465a020890d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" Jul 28 17:10:25.356534 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.356509 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/74113348-a310-4e1c-af96-465a020890d8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-54b45ffdd4-6jflx\" (UID: \"74113348-a310-4e1c-af96-465a020890d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" Jul 28 17:10:25.356597 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.356518 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/af193b3e-ec26-4d39-b4e0-8c1bec3eea4b-klusterlet-config\") pod \"klusterlet-addon-workmgr-785fd5f646-bk8mk\" (UID: \"af193b3e-ec26-4d39-b4e0-8c1bec3eea4b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-785fd5f646-bk8mk" Jul 28 17:10:25.362690 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.362657 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chvn8\" (UniqueName: \"kubernetes.io/projected/af193b3e-ec26-4d39-b4e0-8c1bec3eea4b-kube-api-access-chvn8\") pod \"klusterlet-addon-workmgr-785fd5f646-bk8mk\" (UID: \"af193b3e-ec26-4d39-b4e0-8c1bec3eea4b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-785fd5f646-bk8mk" Jul 28 17:10:25.362914 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.362742 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2hs2\" (UniqueName: \"kubernetes.io/projected/74113348-a310-4e1c-af96-465a020890d8-kube-api-access-k2hs2\") pod \"cluster-proxy-proxy-agent-54b45ffdd4-6jflx\" (UID: \"74113348-a310-4e1c-af96-465a020890d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" Jul 28 17:10:25.508065 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.507959 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-785fd5f646-bk8mk" Jul 28 17:10:25.527994 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.527966 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" Jul 28 17:10:25.645276 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.645229 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-785fd5f646-bk8mk"] Jul 28 17:10:25.647932 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:10:25.647898 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf193b3e_ec26_4d39_b4e0_8c1bec3eea4b.slice/crio-92813d923cd99a45b99f04e7a7ae98b430203e06e377b1f7baac79c91e1df0d6 WatchSource:0}: Error finding container 92813d923cd99a45b99f04e7a7ae98b430203e06e377b1f7baac79c91e1df0d6: Status 404 returned error can't find the container with id 92813d923cd99a45b99f04e7a7ae98b430203e06e377b1f7baac79c91e1df0d6 Jul 28 17:10:25.665358 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:25.665297 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx"] Jul 28 17:10:25.668959 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:10:25.668931 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod74113348_a310_4e1c_af96_465a020890d8.slice/crio-6a2c3927e36967679b6ad09879de71aba39f8743f7ef66617d20987942fe68b5 WatchSource:0}: Error finding container 6a2c3927e36967679b6ad09879de71aba39f8743f7ef66617d20987942fe68b5: Status 404 returned error can't find the container with id 6a2c3927e36967679b6ad09879de71aba39f8743f7ef66617d20987942fe68b5 Jul 28 17:10:26.096844 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:26.096806 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" event={"ID":"74113348-a310-4e1c-af96-465a020890d8","Type":"ContainerStarted","Data":"6a2c3927e36967679b6ad09879de71aba39f8743f7ef66617d20987942fe68b5"} Jul 28 17:10:26.097699 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:26.097677 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-785fd5f646-bk8mk" event={"ID":"af193b3e-ec26-4d39-b4e0-8c1bec3eea4b","Type":"ContainerStarted","Data":"92813d923cd99a45b99f04e7a7ae98b430203e06e377b1f7baac79c91e1df0d6"} Jul 28 17:10:30.110078 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:30.110041 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-785fd5f646-bk8mk" event={"ID":"af193b3e-ec26-4d39-b4e0-8c1bec3eea4b","Type":"ContainerStarted","Data":"73d4c72aeffa4b04d5f662f0f8bd60e7cb9ef550d83b3a9a43fe6ce96d4229f8"} Jul 28 17:10:30.110594 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:30.110250 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-785fd5f646-bk8mk" Jul 28 17:10:30.111893 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:30.111871 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-785fd5f646-bk8mk" Jul 28 17:10:30.128417 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:30.128359 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-785fd5f646-bk8mk" podStartSLOduration=1.339453305 podStartE2EDuration="5.128342651s" totalImagesPullingTime="3.788889346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:10:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:30.127494036 +0000 UTC m=+90.847321746" watchObservedRunningTime="2026-07-28 17:10:30.128342651 +0000 UTC m=+90.848170339" Jul 28 17:10:37.131314 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:37.131278 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" event={"ID":"74113348-a310-4e1c-af96-465a020890d8","Type":"ContainerStarted","Data":"bc46aa919d3737ee976da7df7b8726048998660c5fe59994ec7f43a0039e119e"} Jul 28 17:10:37.131739 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:37.131324 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" event={"ID":"74113348-a310-4e1c-af96-465a020890d8","Type":"ContainerStarted","Data":"9faabe282a1e202c1d6da130e3596077a5cf8af1d7eb61cc1b19cadee0027a92"} Jul 28 17:10:37.131739 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:37.131339 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" event={"ID":"74113348-a310-4e1c-af96-465a020890d8","Type":"ContainerStarted","Data":"eeeb2dcc5915cbfca8380a364db751ec8649dc754f08740adc0d289bfd894507"} Jul 28 17:10:37.157459 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:37.157405 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" podStartSLOduration=1.701786086 podStartE2EDuration="12.157390545s" totalImagesPullingTime="10.455604459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:10:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:37.156171257 +0000 UTC m=+97.875999285" watchObservedRunningTime="2026-07-28 17:10:37.157390545 +0000 UTC m=+97.877218254" Jul 28 17:10:37.543990 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:37.543885 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls\") pod \"dns-default-qjmfr\" (UID: \"51754089-3d17-471f-85f1-5b493bed2336\") " pod="openshift-dns/dns-default-qjmfr" Jul 28 17:10:37.544267 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:10:37.544075 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:10:37.544267 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:10:37.544158 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls podName:51754089-3d17-471f-85f1-5b493bed2336 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:41.544138412 +0000 UTC m=+162.263966108 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls") pod "dns-default-qjmfr" (UID: "51754089-3d17-471f-85f1-5b493bed2336") : secret "dns-default-metrics-tls" not found Jul 28 17:10:42.064886 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:10:42.064853 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6kkbm" Jul 28 17:11:09.479448 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:11:09.479386 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs\") pod \"network-metrics-daemon-w2qls\" (UID: \"c0d05255-0573-4806-84b3-c03ff36cea82\") " pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:11:09.479994 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:11:09.479550 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 28 17:11:09.479994 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:11:09.479618 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs podName:c0d05255-0573-4806-84b3-c03ff36cea82 nodeName:}" failed. No retries permitted until 2026-07-28 17:13:11.479602227 +0000 UTC m=+252.199429914 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs") pod "network-metrics-daemon-w2qls" (UID: "c0d05255-0573-4806-84b3-c03ff36cea82") : secret "metrics-daemon-secret" not found Jul 28 17:11:36.715975 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:11:36.715927 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-qjmfr" podUID="51754089-3d17-471f-85f1-5b493bed2336" Jul 28 17:11:36.799201 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:11:36.799159 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-w2qls" podUID="c0d05255-0573-4806-84b3-c03ff36cea82" Jul 28 17:11:37.303701 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:11:37.303672 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qjmfr" Jul 28 17:11:41.601957 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:11:41.601894 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls\") pod \"dns-default-qjmfr\" (UID: \"51754089-3d17-471f-85f1-5b493bed2336\") " pod="openshift-dns/dns-default-qjmfr" Jul 28 17:11:41.602448 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:11:41.602046 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:11:41.602448 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:11:41.602133 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls podName:51754089-3d17-471f-85f1-5b493bed2336 nodeName:}" failed. No retries permitted until 2026-07-28 17:13:43.602115849 +0000 UTC m=+284.321943547 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls") pod "dns-default-qjmfr" (UID: "51754089-3d17-471f-85f1-5b493bed2336") : secret "dns-default-metrics-tls" not found Jul 28 17:11:48.779809 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:11:48.779740 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:12:30.110529 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:30.110476 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-785fd5f646-bk8mk" podUID="af193b3e-ec26-4d39-b4e0-8c1bec3eea4b" containerName="acm-agent" probeResult="failure" output="Get \"http://10.133.0.8:8000/readyz\": dial tcp 10.133.0.8:8000: connect: connection refused" Jul 28 17:12:30.461132 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:30.461094 2571 generic.go:356] "Generic (PLEG): container finished" podID="af193b3e-ec26-4d39-b4e0-8c1bec3eea4b" containerID="73d4c72aeffa4b04d5f662f0f8bd60e7cb9ef550d83b3a9a43fe6ce96d4229f8" exitCode=1 Jul 28 17:12:30.461317 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:30.461168 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-785fd5f646-bk8mk" event={"ID":"af193b3e-ec26-4d39-b4e0-8c1bec3eea4b","Type":"ContainerDied","Data":"73d4c72aeffa4b04d5f662f0f8bd60e7cb9ef550d83b3a9a43fe6ce96d4229f8"} Jul 28 17:12:30.461492 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:30.461479 2571 scope.go:117] "RemoveContainer" containerID="73d4c72aeffa4b04d5f662f0f8bd60e7cb9ef550d83b3a9a43fe6ce96d4229f8" Jul 28 17:12:31.465485 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:31.465442 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-785fd5f646-bk8mk" event={"ID":"af193b3e-ec26-4d39-b4e0-8c1bec3eea4b","Type":"ContainerStarted","Data":"2934155ccc0e1bdace1e84a0a5d8c91f721cbbd2b505d52e989d0ddc2ebeed75"} Jul 28 17:12:31.466000 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:31.465829 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-785fd5f646-bk8mk" Jul 28 17:12:31.466451 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:31.466430 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-785fd5f646-bk8mk" Jul 28 17:12:58.044585 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:58.044550 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ps8hz"] Jul 28 17:12:58.046485 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:58.046468 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:12:58.054626 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:58.054594 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 28 17:12:58.056564 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:58.056544 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 28 17:12:58.056663 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:58.056632 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 28 17:12:58.056875 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:58.056856 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 28 17:12:58.056937 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:58.056859 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8tp7z\"" Jul 28 17:12:58.111281 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:58.111245 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ps8hz"] Jul 28 17:12:58.136502 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:58.136465 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:12:58.136502 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:58.136502 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lfkj\" (UniqueName: \"kubernetes.io/projected/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-kube-api-access-7lfkj\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:12:58.136725 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:58.136529 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-crio-socket\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:12:58.136725 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:58.136616 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:12:58.136725 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:58.136644 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-data-volume\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:12:58.237820 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:58.237780 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:12:58.237820 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:58.237826 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-data-volume\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:12:58.238080 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:58.237865 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:12:58.238080 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:58.237891 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lfkj\" (UniqueName: \"kubernetes.io/projected/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-kube-api-access-7lfkj\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:12:58.238080 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:12:58.237898 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:12:58.238080 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:58.237922 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-crio-socket\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:12:58.238080 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:12:58.237985 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-kube-rbac-proxy-cm podName:b9cd99dc-8b97-4a3e-8eba-9d55efda9909 nodeName:}" failed. No retries permitted until 2026-07-28 17:12:58.737963452 +0000 UTC m=+239.457791160 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ps8hz" (UID: "b9cd99dc-8b97-4a3e-8eba-9d55efda9909") : configmap "kube-rbac-proxy" not found Jul 28 17:12:58.238080 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:12:58.238016 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:12:58.238080 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:58.238023 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-crio-socket\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:12:58.238080 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:12:58.238065 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-insights-runtime-extractor-tls podName:b9cd99dc-8b97-4a3e-8eba-9d55efda9909 nodeName:}" failed. No retries permitted until 2026-07-28 17:12:58.738054198 +0000 UTC m=+239.457881903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ps8hz" (UID: "b9cd99dc-8b97-4a3e-8eba-9d55efda9909") : secret "insights-runtime-extractor-tls" not found Jul 28 17:12:58.238336 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:58.238193 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-data-volume\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:12:58.251255 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:58.251225 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lfkj\" (UniqueName: \"kubernetes.io/projected/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-kube-api-access-7lfkj\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:12:58.741893 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:58.741853 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:12:58.742082 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:58.741905 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:12:58.742082 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:12:58.741974 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:12:58.742082 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:12:58.742009 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:12:58.742082 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:12:58.742038 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-kube-rbac-proxy-cm podName:b9cd99dc-8b97-4a3e-8eba-9d55efda9909 nodeName:}" failed. No retries permitted until 2026-07-28 17:12:59.74202377 +0000 UTC m=+240.461851458 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ps8hz" (UID: "b9cd99dc-8b97-4a3e-8eba-9d55efda9909") : configmap "kube-rbac-proxy" not found Jul 28 17:12:58.742082 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:12:58.742052 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-insights-runtime-extractor-tls podName:b9cd99dc-8b97-4a3e-8eba-9d55efda9909 nodeName:}" failed. No retries permitted until 2026-07-28 17:12:59.74204543 +0000 UTC m=+240.461873117 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ps8hz" (UID: "b9cd99dc-8b97-4a3e-8eba-9d55efda9909") : secret "insights-runtime-extractor-tls" not found Jul 28 17:12:59.710081 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:59.710054 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6vlbb_7fc05f85-4048-4422-8800-59ca1e11e254/dns-node-resolver/0.log" Jul 28 17:12:59.751155 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:59.751122 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:12:59.751326 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:12:59.751175 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:12:59.751326 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:12:59.751239 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:12:59.751326 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:12:59.751288 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:12:59.751326 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:12:59.751318 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-kube-rbac-proxy-cm podName:b9cd99dc-8b97-4a3e-8eba-9d55efda9909 nodeName:}" failed. No retries permitted until 2026-07-28 17:13:01.751299105 +0000 UTC m=+242.471126817 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ps8hz" (UID: "b9cd99dc-8b97-4a3e-8eba-9d55efda9909") : configmap "kube-rbac-proxy" not found Jul 28 17:12:59.751467 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:12:59.751336 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-insights-runtime-extractor-tls podName:b9cd99dc-8b97-4a3e-8eba-9d55efda9909 nodeName:}" failed. No retries permitted until 2026-07-28 17:13:01.751328085 +0000 UTC m=+242.471155773 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ps8hz" (UID: "b9cd99dc-8b97-4a3e-8eba-9d55efda9909") : secret "insights-runtime-extractor-tls" not found Jul 28 17:13:00.709806 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:00.709776 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dthjv_507a907d-af5f-4815-93c4-0efe8cc04000/node-ca/0.log" Jul 28 17:13:01.510454 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:01.510423 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4nnr6_91c276cf-bc64-466b-a82d-3cdbf4ccc495/serve-healthcheck-canary/0.log" Jul 28 17:13:01.766013 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:01.765923 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:13:01.766013 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:01.765988 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:13:01.766227 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:13:01.766053 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:13:01.766227 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:13:01.766115 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:13:01.766227 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:13:01.766129 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-kube-rbac-proxy-cm podName:b9cd99dc-8b97-4a3e-8eba-9d55efda9909 nodeName:}" failed. No retries permitted until 2026-07-28 17:13:05.766108982 +0000 UTC m=+246.485936685 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ps8hz" (UID: "b9cd99dc-8b97-4a3e-8eba-9d55efda9909") : configmap "kube-rbac-proxy" not found Jul 28 17:13:01.766227 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:13:01.766150 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-insights-runtime-extractor-tls podName:b9cd99dc-8b97-4a3e-8eba-9d55efda9909 nodeName:}" failed. No retries permitted until 2026-07-28 17:13:05.766140236 +0000 UTC m=+246.485967924 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ps8hz" (UID: "b9cd99dc-8b97-4a3e-8eba-9d55efda9909") : secret "insights-runtime-extractor-tls" not found Jul 28 17:13:05.796382 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:05.796349 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:13:05.796755 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:05.796429 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:13:05.796755 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:13:05.796487 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:13:05.796755 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:13:05.796512 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:13:05.796755 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:13:05.796544 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-insights-runtime-extractor-tls podName:b9cd99dc-8b97-4a3e-8eba-9d55efda9909 nodeName:}" failed. No retries permitted until 2026-07-28 17:13:13.796530393 +0000 UTC m=+254.516358081 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ps8hz" (UID: "b9cd99dc-8b97-4a3e-8eba-9d55efda9909") : secret "insights-runtime-extractor-tls" not found Jul 28 17:13:05.796755 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:13:05.796559 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-kube-rbac-proxy-cm podName:b9cd99dc-8b97-4a3e-8eba-9d55efda9909 nodeName:}" failed. No retries permitted until 2026-07-28 17:13:13.796551924 +0000 UTC m=+254.516379612 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ps8hz" (UID: "b9cd99dc-8b97-4a3e-8eba-9d55efda9909") : configmap "kube-rbac-proxy" not found Jul 28 17:13:11.541069 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:11.541029 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs\") pod \"network-metrics-daemon-w2qls\" (UID: \"c0d05255-0573-4806-84b3-c03ff36cea82\") " pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:13:11.541482 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:13:11.541193 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 28 17:13:11.541482 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:13:11.541266 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs podName:c0d05255-0573-4806-84b3-c03ff36cea82 nodeName:}" failed. No retries permitted until 2026-07-28 17:15:13.541247797 +0000 UTC m=+374.261075486 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs") pod "network-metrics-daemon-w2qls" (UID: "c0d05255-0573-4806-84b3-c03ff36cea82") : secret "metrics-daemon-secret" not found Jul 28 17:13:13.857214 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:13.857181 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:13:13.857690 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:13.857241 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:13:13.857690 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:13:13.857324 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:13:13.857690 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:13:13.857380 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-insights-runtime-extractor-tls podName:b9cd99dc-8b97-4a3e-8eba-9d55efda9909 nodeName:}" failed. No retries permitted until 2026-07-28 17:13:29.8573662 +0000 UTC m=+270.577193888 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ps8hz" (UID: "b9cd99dc-8b97-4a3e-8eba-9d55efda9909") : secret "insights-runtime-extractor-tls" not found Jul 28 17:13:13.857911 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:13.857702 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:13:15.530035 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:15.529999 2571 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" podUID="74113348-a310-4e1c-af96-465a020890d8" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 28 17:13:25.529784 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:25.529682 2571 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" podUID="74113348-a310-4e1c-af96-465a020890d8" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 28 17:13:29.869228 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:29.869187 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:13:29.871561 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:29.871540 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b9cd99dc-8b97-4a3e-8eba-9d55efda9909-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps8hz\" (UID: \"b9cd99dc-8b97-4a3e-8eba-9d55efda9909\") " pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:13:30.157728 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:30.157642 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8tp7z\"" Jul 28 17:13:30.165638 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:30.165606 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ps8hz" Jul 28 17:13:30.295034 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:30.295005 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ps8hz"] Jul 28 17:13:30.298409 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:13:30.298381 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9cd99dc_8b97_4a3e_8eba_9d55efda9909.slice/crio-632abd0a3f73faab7342e7f8399790ac92893ef465d287bd90b45746e7414362 WatchSource:0}: Error finding container 632abd0a3f73faab7342e7f8399790ac92893ef465d287bd90b45746e7414362: Status 404 returned error can't find the container with id 632abd0a3f73faab7342e7f8399790ac92893ef465d287bd90b45746e7414362 Jul 28 17:13:30.627543 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:30.627508 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ps8hz" event={"ID":"b9cd99dc-8b97-4a3e-8eba-9d55efda9909","Type":"ContainerStarted","Data":"632abd0a3f73faab7342e7f8399790ac92893ef465d287bd90b45746e7414362"} Jul 28 17:13:35.529887 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:35.529807 2571 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" podUID="74113348-a310-4e1c-af96-465a020890d8" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 28 17:13:35.529887 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:35.529873 2571 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" Jul 28 17:13:35.530358 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:35.530331 2571 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"bc46aa919d3737ee976da7df7b8726048998660c5fe59994ec7f43a0039e119e"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 28 17:13:35.530401 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:35.530383 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" podUID="74113348-a310-4e1c-af96-465a020890d8" containerName="service-proxy" containerID="cri-o://bc46aa919d3737ee976da7df7b8726048998660c5fe59994ec7f43a0039e119e" gracePeriod=30 Jul 28 17:13:35.645854 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:35.645794 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" event={"ID":"74113348-a310-4e1c-af96-465a020890d8","Type":"ContainerDied","Data":"bc46aa919d3737ee976da7df7b8726048998660c5fe59994ec7f43a0039e119e"} Jul 28 17:13:35.645854 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:35.645753 2571 generic.go:356] "Generic (PLEG): container finished" podID="74113348-a310-4e1c-af96-465a020890d8" containerID="bc46aa919d3737ee976da7df7b8726048998660c5fe59994ec7f43a0039e119e" exitCode=2 Jul 28 17:13:35.647277 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:35.647260 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ps8hz" event={"ID":"b9cd99dc-8b97-4a3e-8eba-9d55efda9909","Type":"ContainerStarted","Data":"55122653b9641a45fcbce3078f0ea3f23a188d63a30abe4d7d0e49cec230d4a1"} Jul 28 17:13:36.653361 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:36.653318 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b45ffdd4-6jflx" event={"ID":"74113348-a310-4e1c-af96-465a020890d8","Type":"ContainerStarted","Data":"4a4c0fa597d99dd8d2e2684958b545edcb1cc23752eb2578d8de21fbcc037e01"} Jul 28 17:13:39.460240 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.460204 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-45v8q"] Jul 28 17:13:39.462628 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.462603 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-45v8q" Jul 28 17:13:39.465471 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.465443 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 28 17:13:39.465622 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.465443 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 28 17:13:39.466306 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.466285 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 28 17:13:39.466409 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.466355 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-p5r9c\"" Jul 28 17:13:39.466409 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.466358 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 28 17:13:39.466527 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.466291 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 28 17:13:39.484923 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.484892 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-45v8q"] Jul 28 17:13:39.504035 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.503997 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dksmk"] Jul 28 17:13:39.506385 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.506359 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.510211 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.510183 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 28 17:13:39.510352 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.510220 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m2slg\"" Jul 28 17:13:39.510551 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.510535 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 28 17:13:39.511032 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.511014 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 28 17:13:39.544396 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.544366 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be9b4bb7-3adb-43b4-9eeb-a82ac76481ba-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-45v8q\" (UID: \"be9b4bb7-3adb-43b4-9eeb-a82ac76481ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-45v8q" Jul 28 17:13:39.544599 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.544409 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e14a8715-b4ff-48e3-82f6-b725bacaa253-node-exporter-accelerators-collector-config\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.544599 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.544437 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hjq9\" (UniqueName: \"kubernetes.io/projected/be9b4bb7-3adb-43b4-9eeb-a82ac76481ba-kube-api-access-2hjq9\") pod \"openshift-state-metrics-b8f49f5c6-45v8q\" (UID: \"be9b4bb7-3adb-43b4-9eeb-a82ac76481ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-45v8q" Jul 28 17:13:39.544599 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.544480 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e14a8715-b4ff-48e3-82f6-b725bacaa253-node-exporter-textfile\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.544599 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.544504 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e14a8715-b4ff-48e3-82f6-b725bacaa253-node-exporter-wtmp\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.544849 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.544594 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e14a8715-b4ff-48e3-82f6-b725bacaa253-root\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.544849 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.544657 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e14a8715-b4ff-48e3-82f6-b725bacaa253-metrics-client-ca\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.544849 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.544697 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/be9b4bb7-3adb-43b4-9eeb-a82ac76481ba-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-45v8q\" (UID: \"be9b4bb7-3adb-43b4-9eeb-a82ac76481ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-45v8q" Jul 28 17:13:39.544849 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.544717 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e14a8715-b4ff-48e3-82f6-b725bacaa253-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.544849 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.544737 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e14a8715-b4ff-48e3-82f6-b725bacaa253-sys\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.544849 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.544806 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e14a8715-b4ff-48e3-82f6-b725bacaa253-node-exporter-tls\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.544849 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.544850 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/be9b4bb7-3adb-43b4-9eeb-a82ac76481ba-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-45v8q\" (UID: \"be9b4bb7-3adb-43b4-9eeb-a82ac76481ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-45v8q" Jul 28 17:13:39.545110 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.544878 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlgj8\" (UniqueName: \"kubernetes.io/projected/e14a8715-b4ff-48e3-82f6-b725bacaa253-kube-api-access-nlgj8\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.646123 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.646087 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/be9b4bb7-3adb-43b4-9eeb-a82ac76481ba-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-45v8q\" (UID: \"be9b4bb7-3adb-43b4-9eeb-a82ac76481ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-45v8q" Jul 28 17:13:39.646123 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.646128 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlgj8\" (UniqueName: \"kubernetes.io/projected/e14a8715-b4ff-48e3-82f6-b725bacaa253-kube-api-access-nlgj8\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.646395 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.646147 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be9b4bb7-3adb-43b4-9eeb-a82ac76481ba-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-45v8q\" (UID: \"be9b4bb7-3adb-43b4-9eeb-a82ac76481ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-45v8q" Jul 28 17:13:39.646395 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.646171 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e14a8715-b4ff-48e3-82f6-b725bacaa253-node-exporter-accelerators-collector-config\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.646395 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.646197 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hjq9\" (UniqueName: \"kubernetes.io/projected/be9b4bb7-3adb-43b4-9eeb-a82ac76481ba-kube-api-access-2hjq9\") pod \"openshift-state-metrics-b8f49f5c6-45v8q\" (UID: \"be9b4bb7-3adb-43b4-9eeb-a82ac76481ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-45v8q" Jul 28 17:13:39.646395 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.646233 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e14a8715-b4ff-48e3-82f6-b725bacaa253-node-exporter-textfile\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.646395 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.646259 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e14a8715-b4ff-48e3-82f6-b725bacaa253-node-exporter-wtmp\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.646395 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.646301 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e14a8715-b4ff-48e3-82f6-b725bacaa253-root\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.646395 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.646347 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e14a8715-b4ff-48e3-82f6-b725bacaa253-root\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.646731 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.646699 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e14a8715-b4ff-48e3-82f6-b725bacaa253-node-exporter-textfile\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.646731 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.646699 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e14a8715-b4ff-48e3-82f6-b725bacaa253-node-exporter-wtmp\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.646872 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.646796 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e14a8715-b4ff-48e3-82f6-b725bacaa253-metrics-client-ca\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.646872 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.646824 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/be9b4bb7-3adb-43b4-9eeb-a82ac76481ba-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-45v8q\" (UID: \"be9b4bb7-3adb-43b4-9eeb-a82ac76481ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-45v8q" Jul 28 17:13:39.646872 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.646844 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e14a8715-b4ff-48e3-82f6-b725bacaa253-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.646872 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.646869 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e14a8715-b4ff-48e3-82f6-b725bacaa253-sys\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.647058 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.646894 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e14a8715-b4ff-48e3-82f6-b725bacaa253-node-exporter-tls\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.647058 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.646961 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e14a8715-b4ff-48e3-82f6-b725bacaa253-node-exporter-accelerators-collector-config\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.647058 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:13:39.647003 2571 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 28 17:13:39.647058 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:13:39.647057 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e14a8715-b4ff-48e3-82f6-b725bacaa253-node-exporter-tls podName:e14a8715-b4ff-48e3-82f6-b725bacaa253 nodeName:}" failed. No retries permitted until 2026-07-28 17:13:40.147039021 +0000 UTC m=+280.866866721 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e14a8715-b4ff-48e3-82f6-b725bacaa253-node-exporter-tls") pod "node-exporter-dksmk" (UID: "e14a8715-b4ff-48e3-82f6-b725bacaa253") : secret "node-exporter-tls" not found Jul 28 17:13:39.647323 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.647288 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e14a8715-b4ff-48e3-82f6-b725bacaa253-sys\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.647405 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.647353 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e14a8715-b4ff-48e3-82f6-b725bacaa253-metrics-client-ca\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.647455 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.647411 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be9b4bb7-3adb-43b4-9eeb-a82ac76481ba-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-45v8q\" (UID: \"be9b4bb7-3adb-43b4-9eeb-a82ac76481ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-45v8q" Jul 28 17:13:39.649201 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.649163 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/be9b4bb7-3adb-43b4-9eeb-a82ac76481ba-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-45v8q\" (UID: \"be9b4bb7-3adb-43b4-9eeb-a82ac76481ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-45v8q" Jul 28 17:13:39.649473 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.649453 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e14a8715-b4ff-48e3-82f6-b725bacaa253-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.649605 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.649584 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/be9b4bb7-3adb-43b4-9eeb-a82ac76481ba-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-45v8q\" (UID: \"be9b4bb7-3adb-43b4-9eeb-a82ac76481ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-45v8q" Jul 28 17:13:39.656702 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.656662 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlgj8\" (UniqueName: \"kubernetes.io/projected/e14a8715-b4ff-48e3-82f6-b725bacaa253-kube-api-access-nlgj8\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:39.656991 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.656946 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hjq9\" (UniqueName: \"kubernetes.io/projected/be9b4bb7-3adb-43b4-9eeb-a82ac76481ba-kube-api-access-2hjq9\") pod \"openshift-state-metrics-b8f49f5c6-45v8q\" (UID: \"be9b4bb7-3adb-43b4-9eeb-a82ac76481ba\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-45v8q" Jul 28 17:13:39.666516 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.666477 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ps8hz" event={"ID":"b9cd99dc-8b97-4a3e-8eba-9d55efda9909","Type":"ContainerStarted","Data":"efd9641b3353dbf1903033f339c3e4c8efec9b2ee1ddc665463d316f9b231680"} Jul 28 17:13:39.773429 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.773343 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-45v8q" Jul 28 17:13:39.929495 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:39.929458 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-45v8q"] Jul 28 17:13:39.932564 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:13:39.932528 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe9b4bb7_3adb_43b4_9eeb_a82ac76481ba.slice/crio-11f0968bceafef01bb3093555fcc08d11c2b3e1a9084581e43806d9a6257058e WatchSource:0}: Error finding container 11f0968bceafef01bb3093555fcc08d11c2b3e1a9084581e43806d9a6257058e: Status 404 returned error can't find the container with id 11f0968bceafef01bb3093555fcc08d11c2b3e1a9084581e43806d9a6257058e Jul 28 17:13:40.151376 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:40.151340 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e14a8715-b4ff-48e3-82f6-b725bacaa253-node-exporter-tls\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:40.154143 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:40.154115 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e14a8715-b4ff-48e3-82f6-b725bacaa253-node-exporter-tls\") pod \"node-exporter-dksmk\" (UID: \"e14a8715-b4ff-48e3-82f6-b725bacaa253\") " pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:40.305472 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:13:40.305355 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-qjmfr" podUID="51754089-3d17-471f-85f1-5b493bed2336" Jul 28 17:13:40.417672 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:40.417617 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dksmk" Jul 28 17:13:40.428661 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:13:40.428627 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode14a8715_b4ff_48e3_82f6_b725bacaa253.slice/crio-68e772b46bd03b85b92feb506174855545e36fb087bc7635f9c114749184384f WatchSource:0}: Error finding container 68e772b46bd03b85b92feb506174855545e36fb087bc7635f9c114749184384f: Status 404 returned error can't find the container with id 68e772b46bd03b85b92feb506174855545e36fb087bc7635f9c114749184384f Jul 28 17:13:40.673513 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:40.673470 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dksmk" event={"ID":"e14a8715-b4ff-48e3-82f6-b725bacaa253","Type":"ContainerStarted","Data":"68e772b46bd03b85b92feb506174855545e36fb087bc7635f9c114749184384f"} Jul 28 17:13:40.676603 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:40.676193 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qjmfr" Jul 28 17:13:40.676603 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:40.676460 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-45v8q" event={"ID":"be9b4bb7-3adb-43b4-9eeb-a82ac76481ba","Type":"ContainerStarted","Data":"ddbc61ff6e0b2b46e2802b123dd4f5566ea56c2d81780fc8e309bd3489fd65e0"} Jul 28 17:13:40.676603 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:40.676540 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-45v8q" event={"ID":"be9b4bb7-3adb-43b4-9eeb-a82ac76481ba","Type":"ContainerStarted","Data":"304b594c7564d43cb2b4758d3271a2e3aa4dae758d4b2dc088f88454abf6db51"} Jul 28 17:13:40.676603 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:40.676571 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-45v8q" event={"ID":"be9b4bb7-3adb-43b4-9eeb-a82ac76481ba","Type":"ContainerStarted","Data":"11f0968bceafef01bb3093555fcc08d11c2b3e1a9084581e43806d9a6257058e"} Jul 28 17:13:43.682700 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:43.682654 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls\") pod \"dns-default-qjmfr\" (UID: \"51754089-3d17-471f-85f1-5b493bed2336\") " pod="openshift-dns/dns-default-qjmfr" Jul 28 17:13:43.685436 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:43.685404 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/51754089-3d17-471f-85f1-5b493bed2336-metrics-tls\") pod \"dns-default-qjmfr\" (UID: \"51754089-3d17-471f-85f1-5b493bed2336\") " pod="openshift-dns/dns-default-qjmfr" Jul 28 17:13:43.688296 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:43.688269 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ps8hz" event={"ID":"b9cd99dc-8b97-4a3e-8eba-9d55efda9909","Type":"ContainerStarted","Data":"0fb64c029eb7d1d49e48461e15b4dcb1d46ca36b6e46023dee6e3db074e2acfc"} Jul 28 17:13:43.690007 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:43.689973 2571 generic.go:356] "Generic (PLEG): container finished" podID="e14a8715-b4ff-48e3-82f6-b725bacaa253" containerID="3de6ec50160cb61158adec6469b7112535ac067e00f2fe7efd6dae9f123d5ce2" exitCode=0 Jul 28 17:13:43.690123 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:43.690071 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dksmk" event={"ID":"e14a8715-b4ff-48e3-82f6-b725bacaa253","Type":"ContainerDied","Data":"3de6ec50160cb61158adec6469b7112535ac067e00f2fe7efd6dae9f123d5ce2"} Jul 28 17:13:43.692165 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:43.692140 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-45v8q" event={"ID":"be9b4bb7-3adb-43b4-9eeb-a82ac76481ba","Type":"ContainerStarted","Data":"7f1ee511f7f4256fcfd5757a0cd7804397579cb5c972ec66274586f72872cf8c"} Jul 28 17:13:43.710835 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:43.710782 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ps8hz" podStartSLOduration=34.16976287 podStartE2EDuration="46.710746328s" totalImagesPullingTime="12.540983458s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:12:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:13:43.709941416 +0000 UTC m=+284.429769136" watchObservedRunningTime="2026-07-28 17:13:43.710746328 +0000 UTC m=+284.430574042" Jul 28 17:13:43.747813 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:43.747746 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-45v8q" podStartSLOduration=1.874684835 podStartE2EDuration="4.747727076s" totalImagesPullingTime="2.873042241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:13:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:13:43.74628552 +0000 UTC m=+284.466113231" watchObservedRunningTime="2026-07-28 17:13:43.747727076 +0000 UTC m=+284.467554785" Jul 28 17:13:43.979175 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:43.979093 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hwxn2\"" Jul 28 17:13:43.986912 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:43.986887 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qjmfr" Jul 28 17:13:44.119651 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:44.119622 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qjmfr"] Jul 28 17:13:44.123930 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:13:44.123899 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51754089_3d17_471f_85f1_5b493bed2336.slice/crio-b2795e71a165c007dd8982cfcaccbf1e02b3a766fca4173db74c28817aa9bd9a WatchSource:0}: Error finding container b2795e71a165c007dd8982cfcaccbf1e02b3a766fca4173db74c28817aa9bd9a: Status 404 returned error can't find the container with id b2795e71a165c007dd8982cfcaccbf1e02b3a766fca4173db74c28817aa9bd9a Jul 28 17:13:44.697235 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:44.697195 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qjmfr" event={"ID":"51754089-3d17-471f-85f1-5b493bed2336","Type":"ContainerStarted","Data":"b2795e71a165c007dd8982cfcaccbf1e02b3a766fca4173db74c28817aa9bd9a"} Jul 28 17:13:44.699825 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:44.699787 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dksmk" event={"ID":"e14a8715-b4ff-48e3-82f6-b725bacaa253","Type":"ContainerStarted","Data":"d47a3aa06b8f581b2db42cd65a7fdabddfcfc3cb375b53e029711baff6a80e2c"} Jul 28 17:13:44.699997 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:44.699847 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dksmk" event={"ID":"e14a8715-b4ff-48e3-82f6-b725bacaa253","Type":"ContainerStarted","Data":"8608d7f2dccf1ece5fbcd60cc1247daf695067267986cebbd99ac719bf1a372c"} Jul 28 17:13:44.723238 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:44.723182 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dksmk" podStartSLOduration=3.140403985 podStartE2EDuration="5.723160982s" totalImagesPullingTime="2.513148434s" totalInitContainerRuntime="69.608563ms" isStatefulPod=true podCreationTimestamp="2026-07-28 17:13:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:13:44.721681636 +0000 UTC m=+285.441509346" watchObservedRunningTime="2026-07-28 17:13:44.723160982 +0000 UTC m=+285.442988692" Jul 28 17:13:45.704702 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:45.704671 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qjmfr" event={"ID":"51754089-3d17-471f-85f1-5b493bed2336","Type":"ContainerStarted","Data":"e903910a3582349bf66e1aa4a1d3959039a25fe98fb5b1192a488884e7040378"} Jul 28 17:13:46.709804 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:46.709741 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qjmfr" event={"ID":"51754089-3d17-471f-85f1-5b493bed2336","Type":"ContainerStarted","Data":"5dc35bcc0a88186bb58e63fbf457ea8bb47fc45ea9b00071231841bf919ed8d9"} Jul 28 17:13:46.710202 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:46.709961 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qjmfr" Jul 28 17:13:51.781095 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:13:51.781046 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-w2qls" podUID="c0d05255-0573-4806-84b3-c03ff36cea82" Jul 28 17:13:53.091694 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:53.091637 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qjmfr" podStartSLOduration=258.663704931 podStartE2EDuration="4m20.091619495s" totalImagesPullingTime="1.427914564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:09:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:13:46.730582046 +0000 UTC m=+287.450409757" watchObservedRunningTime="2026-07-28 17:13:53.091619495 +0000 UTC m=+293.811447227" Jul 28 17:13:53.092080 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:53.091781 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-4q56h"] Jul 28 17:13:53.094850 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:53.094831 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-4q56h" Jul 28 17:13:53.097223 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:53.097197 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 28 17:13:53.097648 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:53.097631 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-bxpk5\"" Jul 28 17:13:53.097708 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:53.097631 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 28 17:13:53.117638 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:53.117610 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-4q56h"] Jul 28 17:13:53.152129 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:53.152088 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hqtp\" (UniqueName: \"kubernetes.io/projected/714fbd64-f864-4c39-8e79-1ab21e7c1121-kube-api-access-7hqtp\") pod \"downloads-7c97dc4555-4q56h\" (UID: \"714fbd64-f864-4c39-8e79-1ab21e7c1121\") " pod="openshift-console/downloads-7c97dc4555-4q56h" Jul 28 17:13:53.253377 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:53.253341 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hqtp\" (UniqueName: \"kubernetes.io/projected/714fbd64-f864-4c39-8e79-1ab21e7c1121-kube-api-access-7hqtp\") pod \"downloads-7c97dc4555-4q56h\" (UID: \"714fbd64-f864-4c39-8e79-1ab21e7c1121\") " pod="openshift-console/downloads-7c97dc4555-4q56h" Jul 28 17:13:53.262055 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:53.262033 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hqtp\" (UniqueName: \"kubernetes.io/projected/714fbd64-f864-4c39-8e79-1ab21e7c1121-kube-api-access-7hqtp\") pod \"downloads-7c97dc4555-4q56h\" (UID: \"714fbd64-f864-4c39-8e79-1ab21e7c1121\") " pod="openshift-console/downloads-7c97dc4555-4q56h" Jul 28 17:13:53.404366 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:53.404336 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-4q56h" Jul 28 17:13:53.530731 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:53.530690 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-4q56h"] Jul 28 17:13:53.535097 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:13:53.535056 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod714fbd64_f864_4c39_8e79_1ab21e7c1121.slice/crio-d4739bafa7e91852a0fb8e402ec5e3b89d4bad6387681cb36a15fd293c06bdbf WatchSource:0}: Error finding container d4739bafa7e91852a0fb8e402ec5e3b89d4bad6387681cb36a15fd293c06bdbf: Status 404 returned error can't find the container with id d4739bafa7e91852a0fb8e402ec5e3b89d4bad6387681cb36a15fd293c06bdbf Jul 28 17:13:53.733010 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:53.732925 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-4q56h" event={"ID":"714fbd64-f864-4c39-8e79-1ab21e7c1121","Type":"ContainerStarted","Data":"d4739bafa7e91852a0fb8e402ec5e3b89d4bad6387681cb36a15fd293c06bdbf"} Jul 28 17:13:56.715684 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:56.715654 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qjmfr" Jul 28 17:13:59.639074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:13:59.639045 2571 kubelet.go:1648] "Image garbage collection succeeded" Jul 28 17:14:03.555665 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.555625 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6db9f6d998-frd6c"] Jul 28 17:14:03.559545 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.559522 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:03.562845 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.562817 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 28 17:14:03.563343 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.563320 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 28 17:14:03.563452 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.563430 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 28 17:14:03.563517 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.563478 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 28 17:14:03.563574 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.563537 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wfgr6\"" Jul 28 17:14:03.563574 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.563554 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 28 17:14:03.587020 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.586987 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6db9f6d998-frd6c"] Jul 28 17:14:03.651878 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.651840 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqwn7\" (UniqueName: \"kubernetes.io/projected/8da193b0-2020-4d3f-b726-7229395c3db9-kube-api-access-wqwn7\") pod \"console-6db9f6d998-frd6c\" (UID: \"8da193b0-2020-4d3f-b726-7229395c3db9\") " pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:03.652055 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.651896 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8da193b0-2020-4d3f-b726-7229395c3db9-service-ca\") pod \"console-6db9f6d998-frd6c\" (UID: \"8da193b0-2020-4d3f-b726-7229395c3db9\") " pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:03.652055 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.651960 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8da193b0-2020-4d3f-b726-7229395c3db9-oauth-serving-cert\") pod \"console-6db9f6d998-frd6c\" (UID: \"8da193b0-2020-4d3f-b726-7229395c3db9\") " pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:03.652055 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.652036 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8da193b0-2020-4d3f-b726-7229395c3db9-console-config\") pod \"console-6db9f6d998-frd6c\" (UID: \"8da193b0-2020-4d3f-b726-7229395c3db9\") " pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:03.652189 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.652067 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8da193b0-2020-4d3f-b726-7229395c3db9-console-serving-cert\") pod \"console-6db9f6d998-frd6c\" (UID: \"8da193b0-2020-4d3f-b726-7229395c3db9\") " pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:03.652189 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.652131 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8da193b0-2020-4d3f-b726-7229395c3db9-console-oauth-config\") pod \"console-6db9f6d998-frd6c\" (UID: \"8da193b0-2020-4d3f-b726-7229395c3db9\") " pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:03.752614 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.752576 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqwn7\" (UniqueName: \"kubernetes.io/projected/8da193b0-2020-4d3f-b726-7229395c3db9-kube-api-access-wqwn7\") pod \"console-6db9f6d998-frd6c\" (UID: \"8da193b0-2020-4d3f-b726-7229395c3db9\") " pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:03.752847 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.752626 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8da193b0-2020-4d3f-b726-7229395c3db9-service-ca\") pod \"console-6db9f6d998-frd6c\" (UID: \"8da193b0-2020-4d3f-b726-7229395c3db9\") " pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:03.752847 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.752786 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8da193b0-2020-4d3f-b726-7229395c3db9-oauth-serving-cert\") pod \"console-6db9f6d998-frd6c\" (UID: \"8da193b0-2020-4d3f-b726-7229395c3db9\") " pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:03.752994 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.752884 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8da193b0-2020-4d3f-b726-7229395c3db9-console-config\") pod \"console-6db9f6d998-frd6c\" (UID: \"8da193b0-2020-4d3f-b726-7229395c3db9\") " pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:03.752994 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.752919 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8da193b0-2020-4d3f-b726-7229395c3db9-console-serving-cert\") pod \"console-6db9f6d998-frd6c\" (UID: \"8da193b0-2020-4d3f-b726-7229395c3db9\") " pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:03.752994 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.752948 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8da193b0-2020-4d3f-b726-7229395c3db9-console-oauth-config\") pod \"console-6db9f6d998-frd6c\" (UID: \"8da193b0-2020-4d3f-b726-7229395c3db9\") " pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:03.753585 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.753536 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8da193b0-2020-4d3f-b726-7229395c3db9-oauth-serving-cert\") pod \"console-6db9f6d998-frd6c\" (UID: \"8da193b0-2020-4d3f-b726-7229395c3db9\") " pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:03.753585 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.753539 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8da193b0-2020-4d3f-b726-7229395c3db9-service-ca\") pod \"console-6db9f6d998-frd6c\" (UID: \"8da193b0-2020-4d3f-b726-7229395c3db9\") " pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:03.753798 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.753634 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8da193b0-2020-4d3f-b726-7229395c3db9-console-config\") pod \"console-6db9f6d998-frd6c\" (UID: \"8da193b0-2020-4d3f-b726-7229395c3db9\") " pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:03.755664 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.755639 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8da193b0-2020-4d3f-b726-7229395c3db9-console-oauth-config\") pod \"console-6db9f6d998-frd6c\" (UID: \"8da193b0-2020-4d3f-b726-7229395c3db9\") " pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:03.755807 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.755773 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8da193b0-2020-4d3f-b726-7229395c3db9-console-serving-cert\") pod \"console-6db9f6d998-frd6c\" (UID: \"8da193b0-2020-4d3f-b726-7229395c3db9\") " pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:03.763060 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.763028 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqwn7\" (UniqueName: \"kubernetes.io/projected/8da193b0-2020-4d3f-b726-7229395c3db9-kube-api-access-wqwn7\") pod \"console-6db9f6d998-frd6c\" (UID: \"8da193b0-2020-4d3f-b726-7229395c3db9\") " pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:03.870633 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:03.870592 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:04.779685 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:04.779656 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:14:10.326665 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:10.326495 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6db9f6d998-frd6c"] Jul 28 17:14:10.329287 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:14:10.329258 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8da193b0_2020_4d3f_b726_7229395c3db9.slice/crio-fca3506f63964ddcdce5f929fcf9d0cc5d2b56d5015b111e0945c1339010087f WatchSource:0}: Error finding container fca3506f63964ddcdce5f929fcf9d0cc5d2b56d5015b111e0945c1339010087f: Status 404 returned error can't find the container with id fca3506f63964ddcdce5f929fcf9d0cc5d2b56d5015b111e0945c1339010087f Jul 28 17:14:10.524785 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:10.524737 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 17:14:10.795629 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:10.795546 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-4q56h" event={"ID":"714fbd64-f864-4c39-8e79-1ab21e7c1121","Type":"ContainerStarted","Data":"4e8cb6337d737934ca7684d906120fdd7eabdc790e9edce44492ac2edfb91371"} Jul 28 17:14:10.795808 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:10.795695 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-4q56h" Jul 28 17:14:10.796811 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:10.796785 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db9f6d998-frd6c" event={"ID":"8da193b0-2020-4d3f-b726-7229395c3db9","Type":"ContainerStarted","Data":"fca3506f63964ddcdce5f929fcf9d0cc5d2b56d5015b111e0945c1339010087f"} Jul 28 17:14:10.797155 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:10.797130 2571 patch_prober.go:28] interesting pod/downloads-7c97dc4555-4q56h container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.12:8080/\": dial tcp 10.133.0.12:8080: connect: connection refused" start-of-body= Jul 28 17:14:10.797253 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:10.797176 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-7c97dc4555-4q56h" podUID="714fbd64-f864-4c39-8e79-1ab21e7c1121" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.12:8080/\": dial tcp 10.133.0.12:8080: connect: connection refused" Jul 28 17:14:10.820616 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:10.820563 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-4q56h" podStartSLOduration=0.783082855 podStartE2EDuration="17.820545849s" totalImagesPullingTime="17.037462994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:13:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:14:10.819493573 +0000 UTC m=+311.539321282" watchObservedRunningTime="2026-07-28 17:14:10.820545849 +0000 UTC m=+311.540373558" Jul 28 17:14:11.812391 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:11.812310 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-4q56h" Jul 28 17:14:14.815803 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:14.815701 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db9f6d998-frd6c" event={"ID":"8da193b0-2020-4d3f-b726-7229395c3db9","Type":"ContainerStarted","Data":"c03de1a7febbb1a99505dff2ccaf4874d7413c6ab087be6ad20e133ea26ba99f"} Jul 28 17:14:14.840358 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:14.837218 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6db9f6d998-frd6c" podStartSLOduration=7.828987402 podStartE2EDuration="11.837198147s" totalImagesPullingTime="4.008210745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:14:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:14:14.835046025 +0000 UTC m=+315.554873739" watchObservedRunningTime="2026-07-28 17:14:14.837198147 +0000 UTC m=+315.557025858" Jul 28 17:14:23.871099 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:23.871057 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:23.871571 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:23.871124 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:23.875885 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:23.875860 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:23.947935 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:23.947898 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6db9f6d998-frd6c"] Jul 28 17:14:24.861882 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:24.861854 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:50.882240 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:50.882198 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6db9f6d998-frd6c" podUID="8da193b0-2020-4d3f-b726-7229395c3db9" containerName="console" containerID="cri-o://c03de1a7febbb1a99505dff2ccaf4874d7413c6ab087be6ad20e133ea26ba99f" gracePeriod=15 Jul 28 17:14:51.158388 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.158310 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:51.252580 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.252547 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8da193b0-2020-4d3f-b726-7229395c3db9-console-serving-cert\") pod \"8da193b0-2020-4d3f-b726-7229395c3db9\" (UID: \"8da193b0-2020-4d3f-b726-7229395c3db9\") " Jul 28 17:14:51.252580 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.252589 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8da193b0-2020-4d3f-b726-7229395c3db9-console-config\") pod \"8da193b0-2020-4d3f-b726-7229395c3db9\" (UID: \"8da193b0-2020-4d3f-b726-7229395c3db9\") " Jul 28 17:14:51.252881 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.252611 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8da193b0-2020-4d3f-b726-7229395c3db9-console-oauth-config\") pod \"8da193b0-2020-4d3f-b726-7229395c3db9\" (UID: \"8da193b0-2020-4d3f-b726-7229395c3db9\") " Jul 28 17:14:51.252881 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.252646 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqwn7\" (UniqueName: \"kubernetes.io/projected/8da193b0-2020-4d3f-b726-7229395c3db9-kube-api-access-wqwn7\") pod \"8da193b0-2020-4d3f-b726-7229395c3db9\" (UID: \"8da193b0-2020-4d3f-b726-7229395c3db9\") " Jul 28 17:14:51.252881 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.252799 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8da193b0-2020-4d3f-b726-7229395c3db9-oauth-serving-cert\") pod \"8da193b0-2020-4d3f-b726-7229395c3db9\" (UID: \"8da193b0-2020-4d3f-b726-7229395c3db9\") " Jul 28 17:14:51.252881 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.252854 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8da193b0-2020-4d3f-b726-7229395c3db9-service-ca\") pod \"8da193b0-2020-4d3f-b726-7229395c3db9\" (UID: \"8da193b0-2020-4d3f-b726-7229395c3db9\") " Jul 28 17:14:51.253142 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.253111 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8da193b0-2020-4d3f-b726-7229395c3db9-console-config" (OuterVolumeSpecName: "console-config") pod "8da193b0-2020-4d3f-b726-7229395c3db9" (UID: "8da193b0-2020-4d3f-b726-7229395c3db9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:14:51.253286 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.253259 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8da193b0-2020-4d3f-b726-7229395c3db9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8da193b0-2020-4d3f-b726-7229395c3db9" (UID: "8da193b0-2020-4d3f-b726-7229395c3db9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:14:51.253390 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.253361 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8da193b0-2020-4d3f-b726-7229395c3db9-service-ca" (OuterVolumeSpecName: "service-ca") pod "8da193b0-2020-4d3f-b726-7229395c3db9" (UID: "8da193b0-2020-4d3f-b726-7229395c3db9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:14:51.255046 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.255015 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8da193b0-2020-4d3f-b726-7229395c3db9-kube-api-access-wqwn7" (OuterVolumeSpecName: "kube-api-access-wqwn7") pod "8da193b0-2020-4d3f-b726-7229395c3db9" (UID: "8da193b0-2020-4d3f-b726-7229395c3db9"). InnerVolumeSpecName "kube-api-access-wqwn7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:14:51.255141 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.255087 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8da193b0-2020-4d3f-b726-7229395c3db9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8da193b0-2020-4d3f-b726-7229395c3db9" (UID: "8da193b0-2020-4d3f-b726-7229395c3db9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:14:51.255141 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.255099 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8da193b0-2020-4d3f-b726-7229395c3db9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8da193b0-2020-4d3f-b726-7229395c3db9" (UID: "8da193b0-2020-4d3f-b726-7229395c3db9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:14:51.354023 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.353981 2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8da193b0-2020-4d3f-b726-7229395c3db9-console-config\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:14:51.354023 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.354014 2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8da193b0-2020-4d3f-b726-7229395c3db9-console-oauth-config\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:14:51.354023 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.354028 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqwn7\" (UniqueName: \"kubernetes.io/projected/8da193b0-2020-4d3f-b726-7229395c3db9-kube-api-access-wqwn7\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:14:51.354264 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.354039 2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8da193b0-2020-4d3f-b726-7229395c3db9-oauth-serving-cert\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:14:51.354264 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.354052 2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8da193b0-2020-4d3f-b726-7229395c3db9-service-ca\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:14:51.354264 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.354063 2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8da193b0-2020-4d3f-b726-7229395c3db9-console-serving-cert\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:14:51.948978 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.948946 2571 generic.go:356] "Generic (PLEG): container finished" podID="8da193b0-2020-4d3f-b726-7229395c3db9" containerID="c03de1a7febbb1a99505dff2ccaf4874d7413c6ab087be6ad20e133ea26ba99f" exitCode=0 Jul 28 17:14:51.949441 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.948987 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db9f6d998-frd6c" event={"ID":"8da193b0-2020-4d3f-b726-7229395c3db9","Type":"ContainerDied","Data":"c03de1a7febbb1a99505dff2ccaf4874d7413c6ab087be6ad20e133ea26ba99f"} Jul 28 17:14:51.949441 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.949023 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db9f6d998-frd6c" event={"ID":"8da193b0-2020-4d3f-b726-7229395c3db9","Type":"ContainerDied","Data":"fca3506f63964ddcdce5f929fcf9d0cc5d2b56d5015b111e0945c1339010087f"} Jul 28 17:14:51.949441 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.949055 2571 scope.go:117] "RemoveContainer" containerID="c03de1a7febbb1a99505dff2ccaf4874d7413c6ab087be6ad20e133ea26ba99f" Jul 28 17:14:51.949441 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.948989 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db9f6d998-frd6c" Jul 28 17:14:51.957082 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.957062 2571 scope.go:117] "RemoveContainer" containerID="c03de1a7febbb1a99505dff2ccaf4874d7413c6ab087be6ad20e133ea26ba99f" Jul 28 17:14:51.957400 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:14:51.957378 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c03de1a7febbb1a99505dff2ccaf4874d7413c6ab087be6ad20e133ea26ba99f\": container with ID starting with c03de1a7febbb1a99505dff2ccaf4874d7413c6ab087be6ad20e133ea26ba99f not found: ID does not exist" containerID="c03de1a7febbb1a99505dff2ccaf4874d7413c6ab087be6ad20e133ea26ba99f" Jul 28 17:14:51.957448 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.957413 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c03de1a7febbb1a99505dff2ccaf4874d7413c6ab087be6ad20e133ea26ba99f"} err="failed to get container status \"c03de1a7febbb1a99505dff2ccaf4874d7413c6ab087be6ad20e133ea26ba99f\": rpc error: code = NotFound desc = could not find container \"c03de1a7febbb1a99505dff2ccaf4874d7413c6ab087be6ad20e133ea26ba99f\": container with ID starting with c03de1a7febbb1a99505dff2ccaf4874d7413c6ab087be6ad20e133ea26ba99f not found: ID does not exist" Jul 28 17:14:51.964640 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.964611 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6db9f6d998-frd6c"] Jul 28 17:14:51.968637 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:51.968613 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6db9f6d998-frd6c"] Jul 28 17:14:53.784357 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:14:53.784317 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8da193b0-2020-4d3f-b726-7229395c3db9" path="/var/lib/kubelet/pods/8da193b0-2020-4d3f-b726-7229395c3db9/volumes" Jul 28 17:15:13.621917 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:15:13.621860 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs\") pod \"network-metrics-daemon-w2qls\" (UID: \"c0d05255-0573-4806-84b3-c03ff36cea82\") " pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:15:13.624186 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:15:13.624163 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c0d05255-0573-4806-84b3-c03ff36cea82-metrics-certs\") pod \"network-metrics-daemon-w2qls\" (UID: \"c0d05255-0573-4806-84b3-c03ff36cea82\") " pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:15:13.782799 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:15:13.782753 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-99hnw\"" Jul 28 17:15:13.791509 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:15:13.791482 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w2qls" Jul 28 17:15:13.925146 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:15:13.925059 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w2qls"] Jul 28 17:15:13.928285 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:15:13.928256 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0d05255_0573_4806_84b3_c03ff36cea82.slice/crio-fdf804dfcd7a0fa1dc226d80bfadaa2d2b5c0720d4e8be659ccd258aef1ab962 WatchSource:0}: Error finding container fdf804dfcd7a0fa1dc226d80bfadaa2d2b5c0720d4e8be659ccd258aef1ab962: Status 404 returned error can't find the container with id fdf804dfcd7a0fa1dc226d80bfadaa2d2b5c0720d4e8be659ccd258aef1ab962 Jul 28 17:15:14.020287 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:15:14.020251 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w2qls" event={"ID":"c0d05255-0573-4806-84b3-c03ff36cea82","Type":"ContainerStarted","Data":"fdf804dfcd7a0fa1dc226d80bfadaa2d2b5c0720d4e8be659ccd258aef1ab962"} Jul 28 17:15:16.028522 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:15:16.028481 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w2qls" event={"ID":"c0d05255-0573-4806-84b3-c03ff36cea82","Type":"ContainerStarted","Data":"c2de6517bbcf046b2e4efd631e568dd9646cdfa80a8e46d91b5b579f2d37d82b"} Jul 28 17:15:16.028945 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:15:16.028545 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w2qls" event={"ID":"c0d05255-0573-4806-84b3-c03ff36cea82","Type":"ContainerStarted","Data":"fa5a903df607282e63e074584b55641e940a7def39aa8f6026b113f14537cb49"} Jul 28 17:15:16.055956 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:15:16.055899 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-w2qls" podStartSLOduration=376.016641454 podStartE2EDuration="6m17.055881669s" totalImagesPullingTime="1.039240215s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:15:16.055004586 +0000 UTC m=+376.774832322" watchObservedRunningTime="2026-07-28 17:15:16.055881669 +0000 UTC m=+376.775709378" Jul 28 17:19:23.304128 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.304045 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b75845fc7-h87lt"] Jul 28 17:19:23.304554 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.304285 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8da193b0-2020-4d3f-b726-7229395c3db9" containerName="console" Jul 28 17:19:23.304554 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.304296 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8da193b0-2020-4d3f-b726-7229395c3db9" containerName="console" Jul 28 17:19:23.304554 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.304339 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8da193b0-2020-4d3f-b726-7229395c3db9" containerName="console" Jul 28 17:19:23.305982 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.305962 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:23.308891 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.308865 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 28 17:19:23.309038 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.308925 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 28 17:19:23.309038 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.308939 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wfgr6\"" Jul 28 17:19:23.309038 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.308952 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 28 17:19:23.309038 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.308944 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 28 17:19:23.309361 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.309346 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 28 17:19:23.314167 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.314151 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 28 17:19:23.319674 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.319652 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b75845fc7-h87lt"] Jul 28 17:19:23.390634 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.390594 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ce8b2dc0-925e-470f-816a-cea0de9df4ec-oauth-serving-cert\") pod \"console-6b75845fc7-h87lt\" (UID: \"ce8b2dc0-925e-470f-816a-cea0de9df4ec\") " pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:23.390851 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.390652 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ce8b2dc0-925e-470f-816a-cea0de9df4ec-console-config\") pod \"console-6b75845fc7-h87lt\" (UID: \"ce8b2dc0-925e-470f-816a-cea0de9df4ec\") " pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:23.390851 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.390717 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ce8b2dc0-925e-470f-816a-cea0de9df4ec-service-ca\") pod \"console-6b75845fc7-h87lt\" (UID: \"ce8b2dc0-925e-470f-816a-cea0de9df4ec\") " pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:23.390851 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.390779 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce8b2dc0-925e-470f-816a-cea0de9df4ec-trusted-ca-bundle\") pod \"console-6b75845fc7-h87lt\" (UID: \"ce8b2dc0-925e-470f-816a-cea0de9df4ec\") " pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:23.390851 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.390797 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w98bf\" (UniqueName: \"kubernetes.io/projected/ce8b2dc0-925e-470f-816a-cea0de9df4ec-kube-api-access-w98bf\") pod \"console-6b75845fc7-h87lt\" (UID: \"ce8b2dc0-925e-470f-816a-cea0de9df4ec\") " pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:23.390851 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.390823 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ce8b2dc0-925e-470f-816a-cea0de9df4ec-console-serving-cert\") pod \"console-6b75845fc7-h87lt\" (UID: \"ce8b2dc0-925e-470f-816a-cea0de9df4ec\") " pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:23.391050 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.390855 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ce8b2dc0-925e-470f-816a-cea0de9df4ec-console-oauth-config\") pod \"console-6b75845fc7-h87lt\" (UID: \"ce8b2dc0-925e-470f-816a-cea0de9df4ec\") " pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:23.492208 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.492166 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce8b2dc0-925e-470f-816a-cea0de9df4ec-trusted-ca-bundle\") pod \"console-6b75845fc7-h87lt\" (UID: \"ce8b2dc0-925e-470f-816a-cea0de9df4ec\") " pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:23.492208 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.492206 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w98bf\" (UniqueName: \"kubernetes.io/projected/ce8b2dc0-925e-470f-816a-cea0de9df4ec-kube-api-access-w98bf\") pod \"console-6b75845fc7-h87lt\" (UID: \"ce8b2dc0-925e-470f-816a-cea0de9df4ec\") " pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:23.492469 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.492226 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ce8b2dc0-925e-470f-816a-cea0de9df4ec-console-serving-cert\") pod \"console-6b75845fc7-h87lt\" (UID: \"ce8b2dc0-925e-470f-816a-cea0de9df4ec\") " pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:23.492469 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.492243 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ce8b2dc0-925e-470f-816a-cea0de9df4ec-console-oauth-config\") pod \"console-6b75845fc7-h87lt\" (UID: \"ce8b2dc0-925e-470f-816a-cea0de9df4ec\") " pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:23.492469 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.492271 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ce8b2dc0-925e-470f-816a-cea0de9df4ec-oauth-serving-cert\") pod \"console-6b75845fc7-h87lt\" (UID: \"ce8b2dc0-925e-470f-816a-cea0de9df4ec\") " pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:23.492469 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.492307 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ce8b2dc0-925e-470f-816a-cea0de9df4ec-console-config\") pod \"console-6b75845fc7-h87lt\" (UID: \"ce8b2dc0-925e-470f-816a-cea0de9df4ec\") " pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:23.492469 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.492339 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ce8b2dc0-925e-470f-816a-cea0de9df4ec-service-ca\") pod \"console-6b75845fc7-h87lt\" (UID: \"ce8b2dc0-925e-470f-816a-cea0de9df4ec\") " pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:23.493191 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.493152 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ce8b2dc0-925e-470f-816a-cea0de9df4ec-oauth-serving-cert\") pod \"console-6b75845fc7-h87lt\" (UID: \"ce8b2dc0-925e-470f-816a-cea0de9df4ec\") " pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:23.493321 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.493192 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ce8b2dc0-925e-470f-816a-cea0de9df4ec-console-config\") pod \"console-6b75845fc7-h87lt\" (UID: \"ce8b2dc0-925e-470f-816a-cea0de9df4ec\") " pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:23.493321 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.493197 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ce8b2dc0-925e-470f-816a-cea0de9df4ec-service-ca\") pod \"console-6b75845fc7-h87lt\" (UID: \"ce8b2dc0-925e-470f-816a-cea0de9df4ec\") " pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:23.493441 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.493366 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce8b2dc0-925e-470f-816a-cea0de9df4ec-trusted-ca-bundle\") pod \"console-6b75845fc7-h87lt\" (UID: \"ce8b2dc0-925e-470f-816a-cea0de9df4ec\") " pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:23.494803 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.494774 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ce8b2dc0-925e-470f-816a-cea0de9df4ec-console-oauth-config\") pod \"console-6b75845fc7-h87lt\" (UID: \"ce8b2dc0-925e-470f-816a-cea0de9df4ec\") " pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:23.494893 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.494878 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ce8b2dc0-925e-470f-816a-cea0de9df4ec-console-serving-cert\") pod \"console-6b75845fc7-h87lt\" (UID: \"ce8b2dc0-925e-470f-816a-cea0de9df4ec\") " pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:23.501574 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.501548 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w98bf\" (UniqueName: \"kubernetes.io/projected/ce8b2dc0-925e-470f-816a-cea0de9df4ec-kube-api-access-w98bf\") pod \"console-6b75845fc7-h87lt\" (UID: \"ce8b2dc0-925e-470f-816a-cea0de9df4ec\") " pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:23.616170 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.616143 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:23.745217 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.745056 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b75845fc7-h87lt"] Jul 28 17:19:23.747715 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:19:23.747687 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce8b2dc0_925e_470f_816a_cea0de9df4ec.slice/crio-7dd47361fcccb7ae968146b00b2edc6b3ab3d9c2adaedf2f7685d5eb19a605d2 WatchSource:0}: Error finding container 7dd47361fcccb7ae968146b00b2edc6b3ab3d9c2adaedf2f7685d5eb19a605d2: Status 404 returned error can't find the container with id 7dd47361fcccb7ae968146b00b2edc6b3ab3d9c2adaedf2f7685d5eb19a605d2 Jul 28 17:19:23.860851 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.860814 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b75845fc7-h87lt" event={"ID":"ce8b2dc0-925e-470f-816a-cea0de9df4ec","Type":"ContainerStarted","Data":"fad8c2075a7c1b9be2d9b5b6fdba8af2247bc7c0aac0055014d29a8f54837585"} Jul 28 17:19:23.861040 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.860873 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b75845fc7-h87lt" event={"ID":"ce8b2dc0-925e-470f-816a-cea0de9df4ec","Type":"ContainerStarted","Data":"7dd47361fcccb7ae968146b00b2edc6b3ab3d9c2adaedf2f7685d5eb19a605d2"} Jul 28 17:19:23.880228 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:23.880110 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b75845fc7-h87lt" podStartSLOduration=0.880089313 podStartE2EDuration="880.089313ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:19:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:19:23.878796389 +0000 UTC m=+624.598624100" watchObservedRunningTime="2026-07-28 17:19:23.880089313 +0000 UTC m=+624.599917022" Jul 28 17:19:33.616608 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:33.616555 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:33.617037 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:33.616665 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:33.621468 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:33.621443 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:19:33.898448 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:19:33.898364 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b75845fc7-h87lt" Jul 28 17:23:22.725833 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:22.725792 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-69bbd798b6-bwj8k"] Jul 28 17:23:22.727932 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:22.727907 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-bwj8k" Jul 28 17:23:22.730141 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:22.730117 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jul 28 17:23:22.730696 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:22.730675 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-5jhsm\"" Jul 28 17:23:22.730889 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:22.730749 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jul 28 17:23:22.733784 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:22.733743 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-bwj8k"] Jul 28 17:23:22.823375 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:22.823331 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wpzh\" (UniqueName: \"kubernetes.io/projected/cee54858-6bc1-47a2-aef4-56c2a13838c3-kube-api-access-7wpzh\") pod \"redis-69bbd798b6-bwj8k\" (UID: \"cee54858-6bc1-47a2-aef4-56c2a13838c3\") " pod="test-ns-feast/redis-69bbd798b6-bwj8k" Jul 28 17:23:22.924708 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:22.924665 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wpzh\" (UniqueName: \"kubernetes.io/projected/cee54858-6bc1-47a2-aef4-56c2a13838c3-kube-api-access-7wpzh\") pod \"redis-69bbd798b6-bwj8k\" (UID: \"cee54858-6bc1-47a2-aef4-56c2a13838c3\") " pod="test-ns-feast/redis-69bbd798b6-bwj8k" Jul 28 17:23:22.933386 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:22.933357 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wpzh\" (UniqueName: \"kubernetes.io/projected/cee54858-6bc1-47a2-aef4-56c2a13838c3-kube-api-access-7wpzh\") pod \"redis-69bbd798b6-bwj8k\" (UID: \"cee54858-6bc1-47a2-aef4-56c2a13838c3\") " pod="test-ns-feast/redis-69bbd798b6-bwj8k" Jul 28 17:23:23.038598 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:23.038506 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-bwj8k" Jul 28 17:23:23.160628 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:23.160595 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-bwj8k"] Jul 28 17:23:23.164243 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:23:23.164214 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcee54858_6bc1_47a2_aef4_56c2a13838c3.slice/crio-0a14fb5563be43e8a87c65a02e239a55d90523f22a4c1de9d3e1a42de93fd9a2 WatchSource:0}: Error finding container 0a14fb5563be43e8a87c65a02e239a55d90523f22a4c1de9d3e1a42de93fd9a2: Status 404 returned error can't find the container with id 0a14fb5563be43e8a87c65a02e239a55d90523f22a4c1de9d3e1a42de93fd9a2 Jul 28 17:23:23.166166 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:23.166152 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 17:23:23.678217 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:23.678176 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-bwj8k" event={"ID":"cee54858-6bc1-47a2-aef4-56c2a13838c3","Type":"ContainerStarted","Data":"0a14fb5563be43e8a87c65a02e239a55d90523f22a4c1de9d3e1a42de93fd9a2"} Jul 28 17:23:25.688053 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:25.688020 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-bwj8k" event={"ID":"cee54858-6bc1-47a2-aef4-56c2a13838c3","Type":"ContainerStarted","Data":"2cd5696242603962917601dd4f9dd73a341fd5e3f334e44278f20cfdad7a3d78"} Jul 28 17:23:25.702331 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:25.702279 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/redis-69bbd798b6-bwj8k" podStartSLOduration=1.994481923 podStartE2EDuration="3.702260442s" totalImagesPullingTime="1.707778519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:23:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:23:25.701539985 +0000 UTC m=+866.421367696" watchObservedRunningTime="2026-07-28 17:23:25.702260442 +0000 UTC m=+866.422088153" Jul 28 17:23:58.316635 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:58.316467 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-test-apply-2sk6l"] Jul 28 17:23:58.319658 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:58.319634 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-2sk6l" Jul 28 17:23:58.322047 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:58.322017 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-tdzbp\"" Jul 28 17:23:58.322182 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:58.322052 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Jul 28 17:23:58.328014 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:58.327991 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-2sk6l"] Jul 28 17:23:58.370207 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:58.370176 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg4ng\" (UniqueName: \"kubernetes.io/projected/86fedd23-641b-495a-af95-be184636f095-kube-api-access-rg4ng\") pod \"feast-test-apply-2sk6l\" (UID: \"86fedd23-641b-495a-af95-be184636f095\") " pod="test-ns-feast/feast-test-apply-2sk6l" Jul 28 17:23:58.470579 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:58.470545 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg4ng\" (UniqueName: \"kubernetes.io/projected/86fedd23-641b-495a-af95-be184636f095-kube-api-access-rg4ng\") pod \"feast-test-apply-2sk6l\" (UID: \"86fedd23-641b-495a-af95-be184636f095\") " pod="test-ns-feast/feast-test-apply-2sk6l" Jul 28 17:23:58.479220 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:58.479185 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg4ng\" (UniqueName: \"kubernetes.io/projected/86fedd23-641b-495a-af95-be184636f095-kube-api-access-rg4ng\") pod \"feast-test-apply-2sk6l\" (UID: \"86fedd23-641b-495a-af95-be184636f095\") " pod="test-ns-feast/feast-test-apply-2sk6l" Jul 28 17:23:58.656587 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:58.656551 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-2sk6l" Jul 28 17:23:58.781441 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:58.781410 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-2sk6l"] Jul 28 17:23:58.784599 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:23:58.784574 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86fedd23_641b_495a_af95_be184636f095.slice/crio-7c8cd8c84be4d76c806ba0060d7e872f3371064ea7325f8c3f6d5b7d264b844f WatchSource:0}: Error finding container 7c8cd8c84be4d76c806ba0060d7e872f3371064ea7325f8c3f6d5b7d264b844f: Status 404 returned error can't find the container with id 7c8cd8c84be4d76c806ba0060d7e872f3371064ea7325f8c3f6d5b7d264b844f Jul 28 17:23:58.802750 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:23:58.802722 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-2sk6l" event={"ID":"86fedd23-641b-495a-af95-be184636f095","Type":"ContainerStarted","Data":"7c8cd8c84be4d76c806ba0060d7e872f3371064ea7325f8c3f6d5b7d264b844f"} Jul 28 17:24:04.384099 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:24:04.384074 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Jul 28 17:24:04.830711 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:24:04.830608 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-2sk6l" event={"ID":"86fedd23-641b-495a-af95-be184636f095","Type":"ContainerStarted","Data":"f28058bd95247056b5155e6380548a06e207dd17d8cec5a6d373c9479d1a2c68"} Jul 28 17:24:11.856330 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:24:11.856290 2571 generic.go:356] "Generic (PLEG): container finished" podID="86fedd23-641b-495a-af95-be184636f095" containerID="f28058bd95247056b5155e6380548a06e207dd17d8cec5a6d373c9479d1a2c68" exitCode=0 Jul 28 17:24:11.856839 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:24:11.856366 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-2sk6l" event={"ID":"86fedd23-641b-495a-af95-be184636f095","Type":"ContainerDied","Data":"f28058bd95247056b5155e6380548a06e207dd17d8cec5a6d373c9479d1a2c68"} Jul 28 17:24:12.861834 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:24:12.861801 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-2sk6l" event={"ID":"86fedd23-641b-495a-af95-be184636f095","Type":"ContainerStarted","Data":"56144495c5d9fe3d2293f3a29e4c77738ebf2a793fe12924e4efaff5f9baf38b"} Jul 28 17:24:12.877078 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:24:12.877022 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-test-apply-2sk6l" podStartSLOduration=2.667467621 podStartE2EDuration="14.87700322s" totalImagesPullingTime="5.595763877s" totalInitContainerRuntime="6.613771722s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:23:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:24:12.875977041 +0000 UTC m=+913.595804751" watchObservedRunningTime="2026-07-28 17:24:12.87700322 +0000 UTC m=+913.596830929" Jul 28 17:24:21.894576 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:24:21.894541 2571 generic.go:356] "Generic (PLEG): container finished" podID="86fedd23-641b-495a-af95-be184636f095" containerID="56144495c5d9fe3d2293f3a29e4c77738ebf2a793fe12924e4efaff5f9baf38b" exitCode=0 Jul 28 17:24:21.895015 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:24:21.894613 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-2sk6l" event={"ID":"86fedd23-641b-495a-af95-be184636f095","Type":"ContainerDied","Data":"56144495c5d9fe3d2293f3a29e4c77738ebf2a793fe12924e4efaff5f9baf38b"} Jul 28 17:24:23.029398 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:24:23.029373 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-2sk6l" Jul 28 17:24:23.151354 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:24:23.151258 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rg4ng\" (UniqueName: \"kubernetes.io/projected/86fedd23-641b-495a-af95-be184636f095-kube-api-access-rg4ng\") pod \"86fedd23-641b-495a-af95-be184636f095\" (UID: \"86fedd23-641b-495a-af95-be184636f095\") " Jul 28 17:24:23.156371 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:24:23.154582 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86fedd23-641b-495a-af95-be184636f095-kube-api-access-rg4ng" (OuterVolumeSpecName: "kube-api-access-rg4ng") pod "86fedd23-641b-495a-af95-be184636f095" (UID: "86fedd23-641b-495a-af95-be184636f095"). InnerVolumeSpecName "kube-api-access-rg4ng". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:24:23.252408 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:24:23.252365 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rg4ng\" (UniqueName: \"kubernetes.io/projected/86fedd23-641b-495a-af95-be184636f095-kube-api-access-rg4ng\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:24:23.905021 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:24:23.904984 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-2sk6l" event={"ID":"86fedd23-641b-495a-af95-be184636f095","Type":"ContainerDied","Data":"7c8cd8c84be4d76c806ba0060d7e872f3371064ea7325f8c3f6d5b7d264b844f"} Jul 28 17:24:23.905021 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:24:23.905032 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7c8cd8c84be4d76c806ba0060d7e872f3371064ea7325f8c3f6d5b7d264b844f" Jul 28 17:24:23.905237 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:24:23.905034 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-2sk6l" Jul 28 17:24:24.083468 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:24:24.083377 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-2sk6l_86fedd23-641b-495a-af95-be184636f095/feast-0/0.log" Jul 28 17:24:24.090337 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:24:24.090313 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-2sk6l_86fedd23-641b-495a-af95-be184636f095/feast-1/0.log" Jul 28 17:25:01.844507 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:25:01.844468 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-test-apply-2sk6l"] Jul 28 17:25:01.849138 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:25:01.849108 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-test-apply-2sk6l"] Jul 28 17:25:01.860597 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:25:01.860563 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-bwj8k"] Jul 28 17:25:01.860801 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:25:01.860742 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/redis-69bbd798b6-bwj8k" podUID="cee54858-6bc1-47a2-aef4-56c2a13838c3" containerName="redis" containerID="cri-o://2cd5696242603962917601dd4f9dd73a341fd5e3f334e44278f20cfdad7a3d78" gracePeriod=30 Jul 28 17:25:02.037347 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:25:02.037313 2571 generic.go:356] "Generic (PLEG): container finished" podID="cee54858-6bc1-47a2-aef4-56c2a13838c3" containerID="2cd5696242603962917601dd4f9dd73a341fd5e3f334e44278f20cfdad7a3d78" exitCode=0 Jul 28 17:25:02.037531 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:25:02.037380 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-bwj8k" event={"ID":"cee54858-6bc1-47a2-aef4-56c2a13838c3","Type":"ContainerDied","Data":"2cd5696242603962917601dd4f9dd73a341fd5e3f334e44278f20cfdad7a3d78"} Jul 28 17:25:02.099603 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:25:02.099545 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-bwj8k" Jul 28 17:25:02.246562 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:25:02.246523 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wpzh\" (UniqueName: \"kubernetes.io/projected/cee54858-6bc1-47a2-aef4-56c2a13838c3-kube-api-access-7wpzh\") pod \"cee54858-6bc1-47a2-aef4-56c2a13838c3\" (UID: \"cee54858-6bc1-47a2-aef4-56c2a13838c3\") " Jul 28 17:25:02.248568 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:25:02.248533 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cee54858-6bc1-47a2-aef4-56c2a13838c3-kube-api-access-7wpzh" (OuterVolumeSpecName: "kube-api-access-7wpzh") pod "cee54858-6bc1-47a2-aef4-56c2a13838c3" (UID: "cee54858-6bc1-47a2-aef4-56c2a13838c3"). InnerVolumeSpecName "kube-api-access-7wpzh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:25:02.347080 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:25:02.347042 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wpzh\" (UniqueName: \"kubernetes.io/projected/cee54858-6bc1-47a2-aef4-56c2a13838c3-kube-api-access-7wpzh\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:25:03.041867 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:25:03.041830 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-bwj8k" event={"ID":"cee54858-6bc1-47a2-aef4-56c2a13838c3","Type":"ContainerDied","Data":"0a14fb5563be43e8a87c65a02e239a55d90523f22a4c1de9d3e1a42de93fd9a2"} Jul 28 17:25:03.042330 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:25:03.041870 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-bwj8k" Jul 28 17:25:03.042330 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:25:03.041885 2571 scope.go:117] "RemoveContainer" containerID="2cd5696242603962917601dd4f9dd73a341fd5e3f334e44278f20cfdad7a3d78" Jul 28 17:25:03.076239 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:25:03.076203 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-bwj8k"] Jul 28 17:25:03.085185 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:25:03.085160 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-69bbd798b6-bwj8k"] Jul 28 17:25:03.784122 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:25:03.784085 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86fedd23-641b-495a-af95-be184636f095" path="/var/lib/kubelet/pods/86fedd23-641b-495a-af95-be184636f095/volumes" Jul 28 17:25:03.784431 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:25:03.784419 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cee54858-6bc1-47a2-aef4-56c2a13838c3" path="/var/lib/kubelet/pods/cee54858-6bc1-47a2-aef4-56c2a13838c3/volumes" Jul 28 17:30:16.363971 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:16.363937 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7495bcd-p8f5l"] Jul 28 17:30:16.364388 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:16.364167 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cee54858-6bc1-47a2-aef4-56c2a13838c3" containerName="redis" Jul 28 17:30:16.364388 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:16.364177 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="cee54858-6bc1-47a2-aef4-56c2a13838c3" containerName="redis" Jul 28 17:30:16.364388 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:16.364184 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86fedd23-641b-495a-af95-be184636f095" containerName="feast-1" Jul 28 17:30:16.364388 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:16.364190 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="86fedd23-641b-495a-af95-be184636f095" containerName="feast-1" Jul 28 17:30:16.364388 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:16.364198 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86fedd23-641b-495a-af95-be184636f095" containerName="feast-0" Jul 28 17:30:16.364388 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:16.364205 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="86fedd23-641b-495a-af95-be184636f095" containerName="feast-0" Jul 28 17:30:16.364388 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:16.364249 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="cee54858-6bc1-47a2-aef4-56c2a13838c3" containerName="redis" Jul 28 17:30:16.364388 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:16.364255 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="86fedd23-641b-495a-af95-be184636f095" containerName="feast-1" Jul 28 17:30:16.365839 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:16.365815 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-p8f5l" Jul 28 17:30:16.367981 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:16.367960 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-cxfgb\"" Jul 28 17:30:16.368104 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:16.368038 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Jul 28 17:30:16.368574 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:16.368559 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Jul 28 17:30:16.378954 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:16.378924 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7495bcd-p8f5l"] Jul 28 17:30:16.497925 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:16.497885 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppkpn\" (UniqueName: \"kubernetes.io/projected/87e50a68-be0f-4288-afa7-eee7d0bee4e5-kube-api-access-ppkpn\") pod \"feast-operator-controller-manager-7495bcd-p8f5l\" (UID: \"87e50a68-be0f-4288-afa7-eee7d0bee4e5\") " pod="feast-operator-system/feast-operator-controller-manager-7495bcd-p8f5l" Jul 28 17:30:16.598546 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:16.598503 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppkpn\" (UniqueName: \"kubernetes.io/projected/87e50a68-be0f-4288-afa7-eee7d0bee4e5-kube-api-access-ppkpn\") pod \"feast-operator-controller-manager-7495bcd-p8f5l\" (UID: \"87e50a68-be0f-4288-afa7-eee7d0bee4e5\") " pod="feast-operator-system/feast-operator-controller-manager-7495bcd-p8f5l" Jul 28 17:30:16.616870 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:16.616846 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppkpn\" (UniqueName: \"kubernetes.io/projected/87e50a68-be0f-4288-afa7-eee7d0bee4e5-kube-api-access-ppkpn\") pod \"feast-operator-controller-manager-7495bcd-p8f5l\" (UID: \"87e50a68-be0f-4288-afa7-eee7d0bee4e5\") " pod="feast-operator-system/feast-operator-controller-manager-7495bcd-p8f5l" Jul 28 17:30:16.675976 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:16.675934 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-p8f5l" Jul 28 17:30:16.812167 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:16.812132 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7495bcd-p8f5l"] Jul 28 17:30:16.815160 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:30:16.815134 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87e50a68_be0f_4288_afa7_eee7d0bee4e5.slice/crio-31fee98f92536c66ee2b9539debb7363d224937034d041eb1001efa31ed97dcf WatchSource:0}: Error finding container 31fee98f92536c66ee2b9539debb7363d224937034d041eb1001efa31ed97dcf: Status 404 returned error can't find the container with id 31fee98f92536c66ee2b9539debb7363d224937034d041eb1001efa31ed97dcf Jul 28 17:30:16.816939 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:16.816920 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 17:30:17.123545 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:17.123503 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-p8f5l" event={"ID":"87e50a68-be0f-4288-afa7-eee7d0bee4e5","Type":"ContainerStarted","Data":"31fee98f92536c66ee2b9539debb7363d224937034d041eb1001efa31ed97dcf"} Jul 28 17:30:20.136435 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:20.136395 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-p8f5l" event={"ID":"87e50a68-be0f-4288-afa7-eee7d0bee4e5","Type":"ContainerStarted","Data":"331850620a04614c058836393781a01291c02b5e049449c455f646834a5ce335"} Jul 28 17:30:20.136938 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:20.136465 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-p8f5l" Jul 28 17:30:20.153689 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:20.153633 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-p8f5l" podStartSLOduration=1.8276618930000001 podStartE2EDuration="4.153615249s" totalImagesPullingTime="2.325953356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:30:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:30:20.152887594 +0000 UTC m=+1280.872715315" watchObservedRunningTime="2026-07-28 17:30:20.153615249 +0000 UTC m=+1280.873442959" Jul 28 17:30:31.142754 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:31.142724 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-p8f5l" Jul 28 17:30:33.142520 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.142483 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf"] Jul 28 17:30:33.144926 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.144908 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:30:33.146973 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.146946 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Jul 28 17:30:33.147518 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.147498 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jul 28 17:30:33.147609 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.147497 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Jul 28 17:30:33.147609 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.147507 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-47kxd\"" Jul 28 17:30:33.147609 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.147591 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Jul 28 17:30:33.147781 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.147613 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jul 28 17:30:33.148286 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.148267 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Jul 28 17:30:33.159925 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.159897 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf"] Jul 28 17:30:33.225634 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.225602 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-2rfmf\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:30:33.225846 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.225645 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-2rfmf\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:30:33.225846 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.225672 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-2rfmf\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:30:33.225846 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.225744 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d2db5a2b-de98-4587-a8a1-8a4b90a72467-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-2rfmf\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:30:33.225846 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.225819 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndbrn\" (UniqueName: \"kubernetes.io/projected/d2db5a2b-de98-4587-a8a1-8a4b90a72467-kube-api-access-ndbrn\") pod \"feast-simple-feast-setup-777d6cc998-2rfmf\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:30:33.226015 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.225855 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-2rfmf\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:30:33.327023 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.326984 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-2rfmf\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:30:33.327195 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.327035 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d2db5a2b-de98-4587-a8a1-8a4b90a72467-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-2rfmf\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:30:33.327195 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.327062 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndbrn\" (UniqueName: \"kubernetes.io/projected/d2db5a2b-de98-4587-a8a1-8a4b90a72467-kube-api-access-ndbrn\") pod \"feast-simple-feast-setup-777d6cc998-2rfmf\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:30:33.327195 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.327099 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-2rfmf\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:30:33.327195 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.327131 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-2rfmf\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:30:33.327195 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.327169 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-2rfmf\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:30:33.327479 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:30:33.327217 2571 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Jul 28 17:30:33.327479 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:30:33.327299 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-offline-tls podName:d2db5a2b-de98-4587-a8a1-8a4b90a72467 nodeName:}" failed. No retries permitted until 2026-07-28 17:30:33.827277832 +0000 UTC m=+1294.547105527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-offline-tls") pod "feast-simple-feast-setup-777d6cc998-2rfmf" (UID: "d2db5a2b-de98-4587-a8a1-8a4b90a72467") : secret "feast-simple-feast-setup-offline-tls" not found Jul 28 17:30:33.327479 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:30:33.327314 2571 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Jul 28 17:30:33.327479 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:30:33.327396 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-registry-tls podName:d2db5a2b-de98-4587-a8a1-8a4b90a72467 nodeName:}" failed. No retries permitted until 2026-07-28 17:30:33.82737505 +0000 UTC m=+1294.547202738 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-registry-tls") pod "feast-simple-feast-setup-777d6cc998-2rfmf" (UID: "d2db5a2b-de98-4587-a8a1-8a4b90a72467") : secret "feast-simple-feast-setup-registry-tls" not found Jul 28 17:30:33.327631 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.327509 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d2db5a2b-de98-4587-a8a1-8a4b90a72467-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-2rfmf\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:30:33.329427 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.329409 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-2rfmf\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:30:33.329523 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.329450 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-2rfmf\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:30:33.338005 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.337979 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndbrn\" (UniqueName: \"kubernetes.io/projected/d2db5a2b-de98-4587-a8a1-8a4b90a72467-kube-api-access-ndbrn\") pod \"feast-simple-feast-setup-777d6cc998-2rfmf\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:30:33.830890 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.830848 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-2rfmf\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:30:33.830890 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.830896 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-2rfmf\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:30:33.833346 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.833314 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-2rfmf\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:30:33.833346 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:33.833326 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-2rfmf\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:30:34.056924 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:34.056744 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:30:34.190450 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:34.190334 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf"] Jul 28 17:30:34.192941 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:30:34.192909 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd2db5a2b_de98_4587_a8a1_8a4b90a72467.slice/crio-258beed2bd7020110c3d8f8341cb1a3abab9d55f9e760edee12d797c1fdee012 WatchSource:0}: Error finding container 258beed2bd7020110c3d8f8341cb1a3abab9d55f9e760edee12d797c1fdee012: Status 404 returned error can't find the container with id 258beed2bd7020110c3d8f8341cb1a3abab9d55f9e760edee12d797c1fdee012 Jul 28 17:30:35.192933 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:35.192843 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" event={"ID":"d2db5a2b-de98-4587-a8a1-8a4b90a72467","Type":"ContainerStarted","Data":"258beed2bd7020110c3d8f8341cb1a3abab9d55f9e760edee12d797c1fdee012"} Jul 28 17:30:49.255664 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:49.255623 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" event={"ID":"d2db5a2b-de98-4587-a8a1-8a4b90a72467","Type":"ContainerStarted","Data":"2c18d7fbe59d6cf6a87db006732e8c46f4e439d79d088318fd5f03621799c996"} Jul 28 17:30:55.280228 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:55.280193 2571 generic.go:356] "Generic (PLEG): container finished" podID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerID="2c18d7fbe59d6cf6a87db006732e8c46f4e439d79d088318fd5f03621799c996" exitCode=0 Jul 28 17:30:55.280228 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:55.280238 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" event={"ID":"d2db5a2b-de98-4587-a8a1-8a4b90a72467","Type":"ContainerDied","Data":"2c18d7fbe59d6cf6a87db006732e8c46f4e439d79d088318fd5f03621799c996"} Jul 28 17:30:56.286137 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:56.286101 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" event={"ID":"d2db5a2b-de98-4587-a8a1-8a4b90a72467","Type":"ContainerStarted","Data":"98f0f98fd42c737237d6f91e018fbad896768093cc041afd4cc8e24e8a402448"} Jul 28 17:30:59.729776 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:59.729732 2571 scope.go:117] "RemoveContainer" containerID="f28058bd95247056b5155e6380548a06e207dd17d8cec5a6d373c9479d1a2c68" Jul 28 17:30:59.737786 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:30:59.737744 2571 scope.go:117] "RemoveContainer" containerID="56144495c5d9fe3d2293f3a29e4c77738ebf2a793fe12924e4efaff5f9baf38b" Jul 28 17:31:02.309551 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:02.309505 2571 generic.go:356] "Generic (PLEG): container finished" podID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerID="98f0f98fd42c737237d6f91e018fbad896768093cc041afd4cc8e24e8a402448" exitCode=0 Jul 28 17:31:02.309551 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:02.309551 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" event={"ID":"d2db5a2b-de98-4587-a8a1-8a4b90a72467","Type":"ContainerDied","Data":"98f0f98fd42c737237d6f91e018fbad896768093cc041afd4cc8e24e8a402448"} Jul 28 17:31:03.319276 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:03.319230 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" event={"ID":"d2db5a2b-de98-4587-a8a1-8a4b90a72467","Type":"ContainerStarted","Data":"edbbd633196b7a4c1767a3c9b14642e9df009ceb345389437cd7d00468589f63"} Jul 28 17:31:03.319794 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:03.319302 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" event={"ID":"d2db5a2b-de98-4587-a8a1-8a4b90a72467","Type":"ContainerStarted","Data":"e8b90f45e7be37e2a33bf77ad3a63ba3727478f5bfc576d5ffc95f414791441a"} Jul 28 17:31:03.319794 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:03.319325 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" event={"ID":"d2db5a2b-de98-4587-a8a1-8a4b90a72467","Type":"ContainerStarted","Data":"c9b5b9b786975b8a3c4197761e26944fdb94bd40fb88623729c60664d63f5c18"} Jul 28 17:31:03.319794 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:03.319347 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" event={"ID":"d2db5a2b-de98-4587-a8a1-8a4b90a72467","Type":"ContainerStarted","Data":"23d24daae3c5474b2cf38c36737bb0e46416ba48eda03b14213490dbd111b038"} Jul 28 17:31:03.344971 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:03.344901 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" podStartSLOduration=15.398562032 podStartE2EDuration="30.344844655s" totalImagesPullingTime="14.946282623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:30:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:31:03.343132729 +0000 UTC m=+1324.062960441" watchObservedRunningTime="2026-07-28 17:31:03.344844655 +0000 UTC m=+1324.064672367" Jul 28 17:31:04.057641 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:04.057580 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:31:04.057884 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:04.057655 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:31:04.057884 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:04.057689 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:31:04.057884 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:04.057710 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:31:04.057884 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:04.057727 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:31:04.057884 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:04.057787 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:31:04.058161 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:04.057942 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:31:04.058161 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:04.057968 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:31:04.060469 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:04.060424 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.18:8816: connect: connection refused" Jul 28 17:31:04.060619 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:04.060519 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.18:6571: connect: connection refused" Jul 28 17:31:04.060619 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:04.060545 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="online" probeResult="failure" output="Get \"https://10.133.0.18:6567/health\": dial tcp 10.133.0.18:6567: connect: connection refused" Jul 28 17:31:04.060739 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:04.060618 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.18:8443: connect: connection refused" Jul 28 17:31:07.058189 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:07.058131 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.18:8816: connect: connection refused" Jul 28 17:31:07.058672 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:07.058131 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.18:8443: connect: connection refused" Jul 28 17:31:07.058672 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:07.058149 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="online" probeResult="failure" output="Get \"https://10.133.0.18:6567/health\": dial tcp 10.133.0.18:6567: connect: connection refused" Jul 28 17:31:07.058913 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:07.058878 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.18:6571: connect: connection refused" Jul 28 17:31:10.058141 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:10.058107 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:31:10.058614 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:10.058216 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:31:10.058614 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:10.058445 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:31:10.058927 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:10.058904 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:31:10.059223 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:10.059204 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:31:10.059331 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:10.059317 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:31:10.063566 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:10.063549 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:31:10.357979 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:31:10.357891 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:32:12.530345 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.530303 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf"] Jul 28 17:32:12.530994 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.530600 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="registry" containerID="cri-o://23d24daae3c5474b2cf38c36737bb0e46416ba48eda03b14213490dbd111b038" gracePeriod=30 Jul 28 17:32:12.530994 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.530668 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="online" containerID="cri-o://c9b5b9b786975b8a3c4197761e26944fdb94bd40fb88623729c60664d63f5c18" gracePeriod=30 Jul 28 17:32:12.530994 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.530709 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="offline" containerID="cri-o://e8b90f45e7be37e2a33bf77ad3a63ba3727478f5bfc576d5ffc95f414791441a" gracePeriod=30 Jul 28 17:32:12.530994 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.530690 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="ui" containerID="cri-o://edbbd633196b7a4c1767a3c9b14642e9df009ceb345389437cd7d00468589f63" gracePeriod=30 Jul 28 17:32:12.872343 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.872257 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt"] Jul 28 17:32:12.875945 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.875915 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:12.878255 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.878222 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-rrz4d\"" Jul 28 17:32:12.888249 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.888218 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxck7\" (UniqueName: \"kubernetes.io/projected/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-kube-api-access-sxck7\") pod \"feast-simple-feast-setup-777d6cc998-d8mpt\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:12.888249 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.888254 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-d8mpt\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:12.888446 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.888270 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-d8mpt\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:12.888446 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.888300 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-d8mpt\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:12.888446 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.888323 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-d8mpt\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:12.888446 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.888384 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-d8mpt\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:12.891729 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.891695 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt"] Jul 28 17:32:12.989548 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.989507 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxck7\" (UniqueName: \"kubernetes.io/projected/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-kube-api-access-sxck7\") pod \"feast-simple-feast-setup-777d6cc998-d8mpt\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:12.989548 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.989553 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-d8mpt\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:12.989874 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.989577 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-d8mpt\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:12.989874 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.989624 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-d8mpt\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:12.989874 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:32:12.989713 2571 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Jul 28 17:32:12.989874 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.989741 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-d8mpt\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:12.989874 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:32:12.989816 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-registry-tls podName:4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8 nodeName:}" failed. No retries permitted until 2026-07-28 17:32:13.489791387 +0000 UTC m=+1394.209619079 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-registry-tls") pod "feast-simple-feast-setup-777d6cc998-d8mpt" (UID: "4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8") : secret "feast-simple-feast-setup-registry-tls" not found Jul 28 17:32:12.989874 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.989849 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-d8mpt\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:12.990216 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:32:12.989978 2571 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Jul 28 17:32:12.990216 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:32:12.990017 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-ui-tls podName:4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8 nodeName:}" failed. No retries permitted until 2026-07-28 17:32:13.490004717 +0000 UTC m=+1394.209832406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-ui-tls") pod "feast-simple-feast-setup-777d6cc998-d8mpt" (UID: "4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8") : secret "feast-simple-feast-setup-ui-tls" not found Jul 28 17:32:12.990216 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.990148 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-d8mpt\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:12.992306 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.992266 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-d8mpt\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:12.992500 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.992479 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-d8mpt\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:12.998586 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:12.998552 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxck7\" (UniqueName: \"kubernetes.io/projected/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-kube-api-access-sxck7\") pod \"feast-simple-feast-setup-777d6cc998-d8mpt\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:13.494999 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:13.494918 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-d8mpt\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:13.494999 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:13.494976 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-d8mpt\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:13.497332 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:13.497306 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-d8mpt\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:13.497455 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:13.497343 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-d8mpt\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:13.577890 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:13.577856 2571 generic.go:356] "Generic (PLEG): container finished" podID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerID="edbbd633196b7a4c1767a3c9b14642e9df009ceb345389437cd7d00468589f63" exitCode=0 Jul 28 17:32:13.577890 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:13.577882 2571 generic.go:356] "Generic (PLEG): container finished" podID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerID="c9b5b9b786975b8a3c4197761e26944fdb94bd40fb88623729c60664d63f5c18" exitCode=0 Jul 28 17:32:13.577890 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:13.577902 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" event={"ID":"d2db5a2b-de98-4587-a8a1-8a4b90a72467","Type":"ContainerDied","Data":"edbbd633196b7a4c1767a3c9b14642e9df009ceb345389437cd7d00468589f63"} Jul 28 17:32:13.578337 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:13.577933 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" event={"ID":"d2db5a2b-de98-4587-a8a1-8a4b90a72467","Type":"ContainerDied","Data":"c9b5b9b786975b8a3c4197761e26944fdb94bd40fb88623729c60664d63f5c18"} Jul 28 17:32:13.785994 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:13.785914 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:13.922590 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:13.922555 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt"] Jul 28 17:32:13.926299 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:32:13.926271 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f5b5fe2_8f7d_4e30_9234_e4f1fe4712a8.slice/crio-3ab8dcc8d8004263fc6c40b898d26d12949cd67d76bf67c9a67e4bd36b77bd77 WatchSource:0}: Error finding container 3ab8dcc8d8004263fc6c40b898d26d12949cd67d76bf67c9a67e4bd36b77bd77: Status 404 returned error can't find the container with id 3ab8dcc8d8004263fc6c40b898d26d12949cd67d76bf67c9a67e4bd36b77bd77 Jul 28 17:32:14.582827 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:14.582790 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" event={"ID":"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8","Type":"ContainerStarted","Data":"6192c22e16db0ecf2eafb7580240e529db3e00766f2755fa311851d3298d664f"} Jul 28 17:32:14.582827 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:14.582834 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" event={"ID":"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8","Type":"ContainerStarted","Data":"3ab8dcc8d8004263fc6c40b898d26d12949cd67d76bf67c9a67e4bd36b77bd77"} Jul 28 17:32:19.605493 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:19.605455 2571 generic.go:356] "Generic (PLEG): container finished" podID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerID="6192c22e16db0ecf2eafb7580240e529db3e00766f2755fa311851d3298d664f" exitCode=0 Jul 28 17:32:19.605918 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:19.605502 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" event={"ID":"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8","Type":"ContainerDied","Data":"6192c22e16db0ecf2eafb7580240e529db3e00766f2755fa311851d3298d664f"} Jul 28 17:32:20.059442 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:20.059394 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.18:8443: connect: connection refused" Jul 28 17:32:20.354920 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:20.354826 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="online" probeResult="failure" output="Get \"https://10.133.0.18:6567/health\": dial tcp 10.133.0.18:6567: connect: connection refused" Jul 28 17:32:20.611568 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:20.611483 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" event={"ID":"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8","Type":"ContainerStarted","Data":"58cc507ee793ce1aaa9bd4e5011c2920ca617a248a600b85e8eeae678a99251d"} Jul 28 17:32:25.632048 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:25.632017 2571 generic.go:356] "Generic (PLEG): container finished" podID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerID="58cc507ee793ce1aaa9bd4e5011c2920ca617a248a600b85e8eeae678a99251d" exitCode=0 Jul 28 17:32:25.632422 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:25.632093 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" event={"ID":"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8","Type":"ContainerDied","Data":"58cc507ee793ce1aaa9bd4e5011c2920ca617a248a600b85e8eeae678a99251d"} Jul 28 17:32:26.641603 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:26.641557 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" event={"ID":"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8","Type":"ContainerStarted","Data":"b679a866a8e857f9060fbd82387dcfc474e6e6ca157c7065aea149a2b092371b"} Jul 28 17:32:26.642098 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:26.641626 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" event={"ID":"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8","Type":"ContainerStarted","Data":"c04f9d44eb10565a8f5174bbad11db35ca340cf484ac2ead854d0820e5580cbb"} Jul 28 17:32:26.642098 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:26.641646 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" event={"ID":"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8","Type":"ContainerStarted","Data":"b81c1ebcb21ce0e24daf19ca8bf10bc4515b972c1b9c5c7cee9a15abf19d5791"} Jul 28 17:32:26.642098 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:26.641662 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" event={"ID":"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8","Type":"ContainerStarted","Data":"f679ee52eb662bdd4e3ff368e0899043618b352cc48ce272be59b6f13c9a897c"} Jul 28 17:32:26.669342 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:26.669281 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" podStartSLOduration=14.669258926 podStartE2EDuration="14.669258926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:32:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:32:26.667574118 +0000 UTC m=+1407.387401828" watchObservedRunningTime="2026-07-28 17:32:26.669258926 +0000 UTC m=+1407.389086638" Jul 28 17:32:28.787584 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:28.786916 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:28.787584 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:28.786990 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:28.787584 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:28.787011 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:28.787584 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:28.787472 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:28.789273 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:28.789240 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.19:8816: connect: connection refused" Jul 28 17:32:28.789499 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:28.789270 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.19:8443: connect: connection refused" Jul 28 17:32:28.789617 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:28.789303 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="online" probeResult="failure" output="Get \"https://10.133.0.19:6567/health\": dial tcp 10.133.0.19:6567: connect: connection refused" Jul 28 17:32:28.789617 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:28.789354 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.19:6571: connect: connection refused" Jul 28 17:32:30.059698 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:30.059615 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.18:8443: connect: connection refused" Jul 28 17:32:30.354929 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:30.354819 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="online" probeResult="failure" output="Get \"https://10.133.0.18:6567/health\": dial tcp 10.133.0.18:6567: connect: connection refused" Jul 28 17:32:31.787009 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:31.786740 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.19:8443: connect: connection refused" Jul 28 17:32:31.787695 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:31.787674 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:31.787879 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:31.787844 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:31.788325 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:31.788298 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:31.788442 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:31.788339 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:31.788717 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:31.788695 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:31.789935 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:31.789918 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:31.803671 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:31.803633 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:32.673154 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:32.673123 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:32.676399 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:32.676385 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:33.786099 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:33.786069 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:34.787077 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:34.787048 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:35.685266 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:35.685240 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:32:40.059261 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:40.059214 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.18:8443: connect: connection refused" Jul 28 17:32:40.059743 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:40.059364 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:32:40.355315 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:40.355216 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="online" probeResult="failure" output="Get \"https://10.133.0.18:6567/health\": dial tcp 10.133.0.18:6567: connect: connection refused" Jul 28 17:32:40.355460 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:40.355341 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:32:42.716002 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:42.715966 2571 generic.go:356] "Generic (PLEG): container finished" podID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerID="e8b90f45e7be37e2a33bf77ad3a63ba3727478f5bfc576d5ffc95f414791441a" exitCode=137 Jul 28 17:32:42.716002 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:42.715992 2571 generic.go:356] "Generic (PLEG): container finished" podID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerID="23d24daae3c5474b2cf38c36737bb0e46416ba48eda03b14213490dbd111b038" exitCode=137 Jul 28 17:32:42.716394 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:42.716002 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" event={"ID":"d2db5a2b-de98-4587-a8a1-8a4b90a72467","Type":"ContainerDied","Data":"e8b90f45e7be37e2a33bf77ad3a63ba3727478f5bfc576d5ffc95f414791441a"} Jul 28 17:32:42.716394 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:42.716053 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" event={"ID":"d2db5a2b-de98-4587-a8a1-8a4b90a72467","Type":"ContainerDied","Data":"23d24daae3c5474b2cf38c36737bb0e46416ba48eda03b14213490dbd111b038"} Jul 28 17:32:43.177641 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.177617 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:32:43.256836 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.256794 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-ui-tls\") pod \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " Jul 28 17:32:43.256836 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.256842 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-online-tls\") pod \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " Jul 28 17:32:43.257128 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.256861 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndbrn\" (UniqueName: \"kubernetes.io/projected/d2db5a2b-de98-4587-a8a1-8a4b90a72467-kube-api-access-ndbrn\") pod \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " Jul 28 17:32:43.257128 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.256877 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-offline-tls\") pod \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " Jul 28 17:32:43.257128 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.256936 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-registry-tls\") pod \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " Jul 28 17:32:43.257266 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.257175 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d2db5a2b-de98-4587-a8a1-8a4b90a72467-feast-data\") pod \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\" (UID: \"d2db5a2b-de98-4587-a8a1-8a4b90a72467\") " Jul 28 17:32:43.257899 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.257863 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2db5a2b-de98-4587-a8a1-8a4b90a72467-feast-data" (OuterVolumeSpecName: "feast-data") pod "d2db5a2b-de98-4587-a8a1-8a4b90a72467" (UID: "d2db5a2b-de98-4587-a8a1-8a4b90a72467"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:32:43.259222 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.259188 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d2db5a2b-de98-4587-a8a1-8a4b90a72467" (UID: "d2db5a2b-de98-4587-a8a1-8a4b90a72467"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:32:43.259222 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.259206 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "d2db5a2b-de98-4587-a8a1-8a4b90a72467" (UID: "d2db5a2b-de98-4587-a8a1-8a4b90a72467"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:32:43.259383 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.259223 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "d2db5a2b-de98-4587-a8a1-8a4b90a72467" (UID: "d2db5a2b-de98-4587-a8a1-8a4b90a72467"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:32:43.259383 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.259367 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2db5a2b-de98-4587-a8a1-8a4b90a72467-kube-api-access-ndbrn" (OuterVolumeSpecName: "kube-api-access-ndbrn") pod "d2db5a2b-de98-4587-a8a1-8a4b90a72467" (UID: "d2db5a2b-de98-4587-a8a1-8a4b90a72467"). InnerVolumeSpecName "kube-api-access-ndbrn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:32:43.259490 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.259381 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-online-tls" (OuterVolumeSpecName: "online-tls") pod "d2db5a2b-de98-4587-a8a1-8a4b90a72467" (UID: "d2db5a2b-de98-4587-a8a1-8a4b90a72467"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:32:43.358648 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.358559 2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-registry-tls\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:32:43.358648 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.358589 2571 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d2db5a2b-de98-4587-a8a1-8a4b90a72467-feast-data\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:32:43.358648 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.358600 2571 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-ui-tls\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:32:43.358648 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.358609 2571 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-online-tls\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:32:43.358648 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.358616 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndbrn\" (UniqueName: \"kubernetes.io/projected/d2db5a2b-de98-4587-a8a1-8a4b90a72467-kube-api-access-ndbrn\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:32:43.358648 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.358625 2571 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d2db5a2b-de98-4587-a8a1-8a4b90a72467-offline-tls\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:32:43.722404 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.722368 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" event={"ID":"d2db5a2b-de98-4587-a8a1-8a4b90a72467","Type":"ContainerDied","Data":"258beed2bd7020110c3d8f8341cb1a3abab9d55f9e760edee12d797c1fdee012"} Jul 28 17:32:43.722861 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.722410 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf" Jul 28 17:32:43.722861 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.722417 2571 scope.go:117] "RemoveContainer" containerID="edbbd633196b7a4c1767a3c9b14642e9df009ceb345389437cd7d00468589f63" Jul 28 17:32:43.730923 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.730898 2571 scope.go:117] "RemoveContainer" containerID="e8b90f45e7be37e2a33bf77ad3a63ba3727478f5bfc576d5ffc95f414791441a" Jul 28 17:32:43.738220 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.738200 2571 scope.go:117] "RemoveContainer" containerID="c9b5b9b786975b8a3c4197761e26944fdb94bd40fb88623729c60664d63f5c18" Jul 28 17:32:43.742992 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.742966 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf"] Jul 28 17:32:43.746188 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.746166 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-2rfmf"] Jul 28 17:32:43.746782 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.746748 2571 scope.go:117] "RemoveContainer" containerID="23d24daae3c5474b2cf38c36737bb0e46416ba48eda03b14213490dbd111b038" Jul 28 17:32:43.755792 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.755752 2571 scope.go:117] "RemoveContainer" containerID="98f0f98fd42c737237d6f91e018fbad896768093cc041afd4cc8e24e8a402448" Jul 28 17:32:43.763095 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.763072 2571 scope.go:117] "RemoveContainer" containerID="2c18d7fbe59d6cf6a87db006732e8c46f4e439d79d088318fd5f03621799c996" Jul 28 17:32:43.784166 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:32:43.784135 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" path="/var/lib/kubelet/pods/d2db5a2b-de98-4587-a8a1-8a4b90a72467/volumes" Jul 28 17:33:38.779619 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.779583 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r"] Jul 28 17:33:38.780122 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.779871 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="online" Jul 28 17:33:38.780122 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.779885 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="online" Jul 28 17:33:38.780122 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.779904 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="registry" Jul 28 17:33:38.780122 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.779909 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="registry" Jul 28 17:33:38.780122 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.779916 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="ui" Jul 28 17:33:38.780122 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.779923 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="ui" Jul 28 17:33:38.780122 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.779930 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="offline" Jul 28 17:33:38.780122 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.779936 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="offline" Jul 28 17:33:38.780122 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.779943 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="feast-init" Jul 28 17:33:38.780122 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.779948 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="feast-init" Jul 28 17:33:38.780122 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.779956 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="feast-apply" Jul 28 17:33:38.780122 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.779960 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="feast-apply" Jul 28 17:33:38.780122 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.780002 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="online" Jul 28 17:33:38.780122 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.780009 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="registry" Jul 28 17:33:38.780122 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.780016 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="offline" Jul 28 17:33:38.780122 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.780021 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2db5a2b-de98-4587-a8a1-8a4b90a72467" containerName="ui" Jul 28 17:33:38.783276 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.783253 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:38.785460 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.785439 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Jul 28 17:33:38.785634 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.785620 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Jul 28 17:33:38.785700 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.785681 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Jul 28 17:33:38.785840 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.785819 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Jul 28 17:33:38.785929 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.785910 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Jul 28 17:33:38.785991 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.785950 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Jul 28 17:33:38.788604 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.788587 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-swd9g\"" Jul 28 17:33:38.801782 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.799834 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r"] Jul 28 17:33:38.855678 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.855633 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qwt8r\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:38.855678 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.855673 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-registry-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qwt8r\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:38.855918 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.855692 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qwt8r\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:38.855918 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.855744 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qwt8r\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:38.855991 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.855921 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfhdn\" (UniqueName: \"kubernetes.io/projected/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-kube-api-access-gfhdn\") pod \"feast-simple-feast-remote-setup-db6955c6c-qwt8r\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:38.855991 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.855956 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-feast-data\") pod \"feast-simple-feast-remote-setup-db6955c6c-qwt8r\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:38.957049 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.957005 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfhdn\" (UniqueName: \"kubernetes.io/projected/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-kube-api-access-gfhdn\") pod \"feast-simple-feast-remote-setup-db6955c6c-qwt8r\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:38.957233 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.957053 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-feast-data\") pod \"feast-simple-feast-remote-setup-db6955c6c-qwt8r\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:38.957233 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.957101 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qwt8r\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:38.957303 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.957260 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-registry-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qwt8r\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:38.957336 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.957304 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qwt8r\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:38.957380 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.957336 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qwt8r\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:38.957506 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:33:38.957484 2571 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Jul 28 17:33:38.957566 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.957544 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-feast-data\") pod \"feast-simple-feast-remote-setup-db6955c6c-qwt8r\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:38.957617 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:33:38.957565 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-ui-tls podName:d9651ab6-3dc3-4955-bcc4-2df854c0b14b nodeName:}" failed. No retries permitted until 2026-07-28 17:33:39.457542373 +0000 UTC m=+1480.177370079 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-ui-tls") pod "feast-simple-feast-remote-setup-db6955c6c-qwt8r" (UID: "d9651ab6-3dc3-4955-bcc4-2df854c0b14b") : secret "feast-simple-feast-remote-setup-ui-tls" not found Jul 28 17:33:38.958020 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.957999 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-registry-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qwt8r\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:38.959619 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.959597 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qwt8r\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:38.959705 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.959661 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qwt8r\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:38.972127 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:38.972086 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfhdn\" (UniqueName: \"kubernetes.io/projected/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-kube-api-access-gfhdn\") pod \"feast-simple-feast-remote-setup-db6955c6c-qwt8r\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:39.461542 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:39.461507 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qwt8r\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:39.464047 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:39.464020 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qwt8r\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:39.693008 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:39.692953 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:39.835371 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:39.835332 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r"] Jul 28 17:33:39.839336 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:33:39.839302 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9651ab6_3dc3_4955_bcc4_2df854c0b14b.slice/crio-100723e0a10ba988f63bc1db27a4a112802befe1cf98d3e6cdf33fd546ab0c92 WatchSource:0}: Error finding container 100723e0a10ba988f63bc1db27a4a112802befe1cf98d3e6cdf33fd546ab0c92: Status 404 returned error can't find the container with id 100723e0a10ba988f63bc1db27a4a112802befe1cf98d3e6cdf33fd546ab0c92 Jul 28 17:33:39.918625 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:39.918589 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" event={"ID":"d9651ab6-3dc3-4955-bcc4-2df854c0b14b","Type":"ContainerStarted","Data":"8143061fef5466017a5d436d685753d3f379cc42ce93d23a5101afd06dac9089"} Jul 28 17:33:39.918825 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:39.918640 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" event={"ID":"d9651ab6-3dc3-4955-bcc4-2df854c0b14b","Type":"ContainerStarted","Data":"100723e0a10ba988f63bc1db27a4a112802befe1cf98d3e6cdf33fd546ab0c92"} Jul 28 17:33:45.941486 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:45.941451 2571 generic.go:356] "Generic (PLEG): container finished" podID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerID="8143061fef5466017a5d436d685753d3f379cc42ce93d23a5101afd06dac9089" exitCode=0 Jul 28 17:33:45.941951 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:45.941510 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" event={"ID":"d9651ab6-3dc3-4955-bcc4-2df854c0b14b","Type":"ContainerDied","Data":"8143061fef5466017a5d436d685753d3f379cc42ce93d23a5101afd06dac9089"} Jul 28 17:33:46.947180 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:46.947145 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" event={"ID":"d9651ab6-3dc3-4955-bcc4-2df854c0b14b","Type":"ContainerStarted","Data":"a5ab78b96d3cb9f55214c66858c92ebdcf582cb1b3b95260a173606ead806def"} Jul 28 17:33:52.973050 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:52.973012 2571 generic.go:356] "Generic (PLEG): container finished" podID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerID="a5ab78b96d3cb9f55214c66858c92ebdcf582cb1b3b95260a173606ead806def" exitCode=0 Jul 28 17:33:52.973490 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:52.973064 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" event={"ID":"d9651ab6-3dc3-4955-bcc4-2df854c0b14b","Type":"ContainerDied","Data":"a5ab78b96d3cb9f55214c66858c92ebdcf582cb1b3b95260a173606ead806def"} Jul 28 17:33:53.981980 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:53.981942 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" event={"ID":"d9651ab6-3dc3-4955-bcc4-2df854c0b14b","Type":"ContainerStarted","Data":"12754757bd33f1e75dbe1ee1f586517359f3cbef321ec1bcbc1e754101ca3dc7"} Jul 28 17:33:53.981980 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:53.981991 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" event={"ID":"d9651ab6-3dc3-4955-bcc4-2df854c0b14b","Type":"ContainerStarted","Data":"bdc9fffcff343c915f351691d77939ce38fa3ae2fee5c12bd21bb551663aaf4c"} Jul 28 17:33:53.982515 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:53.982004 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" event={"ID":"d9651ab6-3dc3-4955-bcc4-2df854c0b14b","Type":"ContainerStarted","Data":"4ec500ff8808fb810280f62fb3dfd8f797abfda931deb9d00d201fb39844f092"} Jul 28 17:33:54.006686 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:54.006610 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" podStartSLOduration=16.006587401 podStartE2EDuration="16.006587401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:33:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:33:54.00467866 +0000 UTC m=+1494.724506371" watchObservedRunningTime="2026-07-28 17:33:54.006587401 +0000 UTC m=+1494.726415110" Jul 28 17:33:54.693348 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:54.693309 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:54.693649 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:54.693620 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:54.693649 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:54.693652 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:54.695192 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:54.695156 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused" Jul 28 17:33:54.695338 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:54.695195 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused" Jul 28 17:33:54.695338 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:54.695173 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.20:8816: connect: connection refused" Jul 28 17:33:57.694255 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:57.694197 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.20:8816: connect: connection refused" Jul 28 17:33:57.694743 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:57.694206 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused" Jul 28 17:33:57.694743 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:57.694211 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused" Jul 28 17:33:59.693059 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:59.693022 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:59.693448 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:59.693081 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:33:59.693448 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:33:59.693095 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:34:00.694907 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:34:00.694868 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:34:00.695379 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:34:00.695017 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:34:00.695705 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:34:00.695681 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:34:00.696052 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:34:00.696033 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:34:00.701811 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:34:00.701773 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:34:01.016531 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:34:01.016450 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:35:02.182216 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:02.182178 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r"] Jul 28 17:35:02.182804 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:02.182442 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="online" containerID="cri-o://4ec500ff8808fb810280f62fb3dfd8f797abfda931deb9d00d201fb39844f092" gracePeriod=30 Jul 28 17:35:02.182804 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:02.182490 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="ui" containerID="cri-o://12754757bd33f1e75dbe1ee1f586517359f3cbef321ec1bcbc1e754101ca3dc7" gracePeriod=30 Jul 28 17:35:02.182804 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:02.182499 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="offline" containerID="cri-o://bdc9fffcff343c915f351691d77939ce38fa3ae2fee5c12bd21bb551663aaf4c" gracePeriod=30 Jul 28 17:35:02.302681 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:02.302637 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt"] Jul 28 17:35:02.303010 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:02.302955 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="registry" containerID="cri-o://f679ee52eb662bdd4e3ff368e0899043618b352cc48ce272be59b6f13c9a897c" gracePeriod=30 Jul 28 17:35:02.303160 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:02.303002 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="ui" containerID="cri-o://b679a866a8e857f9060fbd82387dcfc474e6e6ca157c7065aea149a2b092371b" gracePeriod=30 Jul 28 17:35:02.303160 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:02.303022 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="offline" containerID="cri-o://c04f9d44eb10565a8f5174bbad11db35ca340cf484ac2ead854d0820e5580cbb" gracePeriod=30 Jul 28 17:35:02.303160 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:02.303016 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="online" containerID="cri-o://b81c1ebcb21ce0e24daf19ca8bf10bc4515b972c1b9c5c7cee9a15abf19d5791" gracePeriod=30 Jul 28 17:35:02.673942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:02.673894 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="online" probeResult="failure" output="Get \"https://10.133.0.19:6567/health\": dial tcp 10.133.0.19:6567: connect: connection refused" Jul 28 17:35:03.247606 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:03.247510 2571 generic.go:356] "Generic (PLEG): container finished" podID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerID="b679a866a8e857f9060fbd82387dcfc474e6e6ca157c7065aea149a2b092371b" exitCode=0 Jul 28 17:35:03.247606 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:03.247537 2571 generic.go:356] "Generic (PLEG): container finished" podID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerID="b81c1ebcb21ce0e24daf19ca8bf10bc4515b972c1b9c5c7cee9a15abf19d5791" exitCode=0 Jul 28 17:35:03.247606 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:03.247578 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" event={"ID":"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8","Type":"ContainerDied","Data":"b679a866a8e857f9060fbd82387dcfc474e6e6ca157c7065aea149a2b092371b"} Jul 28 17:35:03.248079 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:03.247619 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" event={"ID":"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8","Type":"ContainerDied","Data":"b81c1ebcb21ce0e24daf19ca8bf10bc4515b972c1b9c5c7cee9a15abf19d5791"} Jul 28 17:35:03.250061 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:03.250039 2571 generic.go:356] "Generic (PLEG): container finished" podID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerID="12754757bd33f1e75dbe1ee1f586517359f3cbef321ec1bcbc1e754101ca3dc7" exitCode=0 Jul 28 17:35:03.250061 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:03.250060 2571 generic.go:356] "Generic (PLEG): container finished" podID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerID="4ec500ff8808fb810280f62fb3dfd8f797abfda931deb9d00d201fb39844f092" exitCode=0 Jul 28 17:35:03.250213 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:03.250110 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" event={"ID":"d9651ab6-3dc3-4955-bcc4-2df854c0b14b","Type":"ContainerDied","Data":"12754757bd33f1e75dbe1ee1f586517359f3cbef321ec1bcbc1e754101ca3dc7"} Jul 28 17:35:03.250213 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:03.250143 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" event={"ID":"d9651ab6-3dc3-4955-bcc4-2df854c0b14b","Type":"ContainerDied","Data":"4ec500ff8808fb810280f62fb3dfd8f797abfda931deb9d00d201fb39844f092"} Jul 28 17:35:05.685792 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:05.685706 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.19:8443: connect: connection refused" Jul 28 17:35:10.695741 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:10.695685 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused" Jul 28 17:35:11.012780 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:11.012656 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused" Jul 28 17:35:12.674175 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:12.674127 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="online" probeResult="failure" output="Get \"https://10.133.0.19:6567/health\": dial tcp 10.133.0.19:6567: connect: connection refused" Jul 28 17:35:15.685419 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:15.685372 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.19:8443: connect: connection refused" Jul 28 17:35:20.695947 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:20.695894 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused" Jul 28 17:35:21.012401 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:21.012302 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused" Jul 28 17:35:22.673959 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:22.673915 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="online" probeResult="failure" output="Get \"https://10.133.0.19:6567/health\": dial tcp 10.133.0.19:6567: connect: connection refused" Jul 28 17:35:22.674335 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:22.674049 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:35:25.684961 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:25.684921 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.19:8443: connect: connection refused" Jul 28 17:35:25.685354 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:25.685046 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:35:30.696539 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:30.696491 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused" Jul 28 17:35:30.697057 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:30.696654 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:35:31.012378 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:31.012274 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused" Jul 28 17:35:31.012564 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:31.012421 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:35:32.359335 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.359296 2571 generic.go:356] "Generic (PLEG): container finished" podID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerID="c04f9d44eb10565a8f5174bbad11db35ca340cf484ac2ead854d0820e5580cbb" exitCode=137 Jul 28 17:35:32.359335 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.359321 2571 generic.go:356] "Generic (PLEG): container finished" podID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerID="f679ee52eb662bdd4e3ff368e0899043618b352cc48ce272be59b6f13c9a897c" exitCode=137 Jul 28 17:35:32.359839 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.359372 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" event={"ID":"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8","Type":"ContainerDied","Data":"c04f9d44eb10565a8f5174bbad11db35ca340cf484ac2ead854d0820e5580cbb"} Jul 28 17:35:32.359839 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.359433 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" event={"ID":"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8","Type":"ContainerDied","Data":"f679ee52eb662bdd4e3ff368e0899043618b352cc48ce272be59b6f13c9a897c"} Jul 28 17:35:32.362665 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.362631 2571 generic.go:356] "Generic (PLEG): container finished" podID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerID="bdc9fffcff343c915f351691d77939ce38fa3ae2fee5c12bd21bb551663aaf4c" exitCode=137 Jul 28 17:35:32.362903 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.362707 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" event={"ID":"d9651ab6-3dc3-4955-bcc4-2df854c0b14b","Type":"ContainerDied","Data":"bdc9fffcff343c915f351691d77939ce38fa3ae2fee5c12bd21bb551663aaf4c"} Jul 28 17:35:32.394448 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.394422 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:35:32.503015 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.502912 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-registry-tls\") pod \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " Jul 28 17:35:32.503015 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.502955 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-offline-tls\") pod \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " Jul 28 17:35:32.503257 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.503019 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-online-tls\") pod \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " Jul 28 17:35:32.503257 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.503047 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfhdn\" (UniqueName: \"kubernetes.io/projected/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-kube-api-access-gfhdn\") pod \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " Jul 28 17:35:32.503257 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.503080 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-feast-data\") pod \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " Jul 28 17:35:32.503257 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.503118 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-ui-tls\") pod \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\" (UID: \"d9651ab6-3dc3-4955-bcc4-2df854c0b14b\") " Jul 28 17:35:32.503472 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.503374 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d9651ab6-3dc3-4955-bcc4-2df854c0b14b" (UID: "d9651ab6-3dc3-4955-bcc4-2df854c0b14b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:35:32.503741 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.503710 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-feast-data" (OuterVolumeSpecName: "feast-data") pod "d9651ab6-3dc3-4955-bcc4-2df854c0b14b" (UID: "d9651ab6-3dc3-4955-bcc4-2df854c0b14b"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:35:32.505511 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.505477 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-kube-api-access-gfhdn" (OuterVolumeSpecName: "kube-api-access-gfhdn") pod "d9651ab6-3dc3-4955-bcc4-2df854c0b14b" (UID: "d9651ab6-3dc3-4955-bcc4-2df854c0b14b"). InnerVolumeSpecName "kube-api-access-gfhdn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:35:32.505613 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.505507 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "d9651ab6-3dc3-4955-bcc4-2df854c0b14b" (UID: "d9651ab6-3dc3-4955-bcc4-2df854c0b14b"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:35:32.505613 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.505527 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "d9651ab6-3dc3-4955-bcc4-2df854c0b14b" (UID: "d9651ab6-3dc3-4955-bcc4-2df854c0b14b"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:35:32.505613 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.505571 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-online-tls" (OuterVolumeSpecName: "online-tls") pod "d9651ab6-3dc3-4955-bcc4-2df854c0b14b" (UID: "d9651ab6-3dc3-4955-bcc4-2df854c0b14b"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:35:32.604483 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.604440 2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-registry-tls\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:35:32.604483 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.604473 2571 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-offline-tls\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:35:32.604483 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.604482 2571 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-online-tls\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:35:32.604483 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.604490 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfhdn\" (UniqueName: \"kubernetes.io/projected/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-kube-api-access-gfhdn\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:35:32.604835 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.604500 2571 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-feast-data\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:35:32.604835 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.604509 2571 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d9651ab6-3dc3-4955-bcc4-2df854c0b14b-ui-tls\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:35:32.673358 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.673315 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="online" probeResult="failure" output="Get \"https://10.133.0.19:6567/health\": dial tcp 10.133.0.19:6567: connect: connection refused" Jul 28 17:35:32.959481 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:32.959440 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:35:33.007424 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.007385 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-online-tls\") pod \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " Jul 28 17:35:33.007617 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.007449 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-feast-data\") pod \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " Jul 28 17:35:33.007617 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.007486 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxck7\" (UniqueName: \"kubernetes.io/projected/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-kube-api-access-sxck7\") pod \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " Jul 28 17:35:33.007617 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.007534 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-offline-tls\") pod \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " Jul 28 17:35:33.007617 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.007558 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-ui-tls\") pod \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " Jul 28 17:35:33.007617 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.007584 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-registry-tls\") pod \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\" (UID: \"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8\") " Jul 28 17:35:33.008075 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.008041 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-feast-data" (OuterVolumeSpecName: "feast-data") pod "4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" (UID: "4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:35:33.009685 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.009652 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" (UID: "4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:35:33.010235 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.010149 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" (UID: "4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:35:33.010235 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.010201 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-online-tls" (OuterVolumeSpecName: "online-tls") pod "4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" (UID: "4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:35:33.010235 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.010216 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-kube-api-access-sxck7" (OuterVolumeSpecName: "kube-api-access-sxck7") pod "4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" (UID: "4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8"). InnerVolumeSpecName "kube-api-access-sxck7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:35:33.010376 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.010253 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" (UID: "4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:35:33.109159 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.109061 2571 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-online-tls\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:35:33.109159 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.109096 2571 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-feast-data\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:35:33.109159 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.109106 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxck7\" (UniqueName: \"kubernetes.io/projected/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-kube-api-access-sxck7\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:35:33.109159 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.109117 2571 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-offline-tls\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:35:33.109159 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.109125 2571 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-ui-tls\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:35:33.109159 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.109133 2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8-registry-tls\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:35:33.369210 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.369167 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" event={"ID":"4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8","Type":"ContainerDied","Data":"3ab8dcc8d8004263fc6c40b898d26d12949cd67d76bf67c9a67e4bd36b77bd77"} Jul 28 17:35:33.369210 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.369206 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt" Jul 28 17:35:33.369851 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.369247 2571 scope.go:117] "RemoveContainer" containerID="b679a866a8e857f9060fbd82387dcfc474e6e6ca157c7065aea149a2b092371b" Jul 28 17:35:33.371942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.371911 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" event={"ID":"d9651ab6-3dc3-4955-bcc4-2df854c0b14b","Type":"ContainerDied","Data":"100723e0a10ba988f63bc1db27a4a112802befe1cf98d3e6cdf33fd546ab0c92"} Jul 28 17:35:33.372069 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.371983 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r" Jul 28 17:35:33.379770 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.379732 2571 scope.go:117] "RemoveContainer" containerID="c04f9d44eb10565a8f5174bbad11db35ca340cf484ac2ead854d0820e5580cbb" Jul 28 17:35:33.389743 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.389715 2571 scope.go:117] "RemoveContainer" containerID="b81c1ebcb21ce0e24daf19ca8bf10bc4515b972c1b9c5c7cee9a15abf19d5791" Jul 28 17:35:33.398043 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.398010 2571 scope.go:117] "RemoveContainer" containerID="f679ee52eb662bdd4e3ff368e0899043618b352cc48ce272be59b6f13c9a897c" Jul 28 17:35:33.399027 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.399004 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt"] Jul 28 17:35:33.403387 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.403363 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-d8mpt"] Jul 28 17:35:33.406788 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.406753 2571 scope.go:117] "RemoveContainer" containerID="58cc507ee793ce1aaa9bd4e5011c2920ca617a248a600b85e8eeae678a99251d" Jul 28 17:35:33.414496 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.414473 2571 scope.go:117] "RemoveContainer" containerID="6192c22e16db0ecf2eafb7580240e529db3e00766f2755fa311851d3298d664f" Jul 28 17:35:33.416497 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.416474 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r"] Jul 28 17:35:33.419324 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.419298 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qwt8r"] Jul 28 17:35:33.423389 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.423367 2571 scope.go:117] "RemoveContainer" containerID="12754757bd33f1e75dbe1ee1f586517359f3cbef321ec1bcbc1e754101ca3dc7" Jul 28 17:35:33.431915 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.431888 2571 scope.go:117] "RemoveContainer" containerID="bdc9fffcff343c915f351691d77939ce38fa3ae2fee5c12bd21bb551663aaf4c" Jul 28 17:35:33.440053 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.440029 2571 scope.go:117] "RemoveContainer" containerID="4ec500ff8808fb810280f62fb3dfd8f797abfda931deb9d00d201fb39844f092" Jul 28 17:35:33.448173 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.448150 2571 scope.go:117] "RemoveContainer" containerID="a5ab78b96d3cb9f55214c66858c92ebdcf582cb1b3b95260a173606ead806def" Jul 28 17:35:33.455785 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.455744 2571 scope.go:117] "RemoveContainer" containerID="8143061fef5466017a5d436d685753d3f379cc42ce93d23a5101afd06dac9089" Jul 28 17:35:33.784079 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.783994 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" path="/var/lib/kubelet/pods/4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8/volumes" Jul 28 17:35:33.784612 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:33.784598 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" path="/var/lib/kubelet/pods/d9651ab6-3dc3-4955-bcc4-2df854c0b14b/volumes" Jul 28 17:35:51.595062 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:51.594978 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7495bcd-p8f5l"] Jul 28 17:35:51.595464 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:51.595183 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-p8f5l" podUID="87e50a68-be0f-4288-afa7-eee7d0bee4e5" containerName="manager" containerID="cri-o://331850620a04614c058836393781a01291c02b5e049449c455f646834a5ce335" gracePeriod=10 Jul 28 17:35:51.846339 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:51.846268 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-p8f5l" Jul 28 17:35:51.946351 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:51.946312 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppkpn\" (UniqueName: \"kubernetes.io/projected/87e50a68-be0f-4288-afa7-eee7d0bee4e5-kube-api-access-ppkpn\") pod \"87e50a68-be0f-4288-afa7-eee7d0bee4e5\" (UID: \"87e50a68-be0f-4288-afa7-eee7d0bee4e5\") " Jul 28 17:35:51.948581 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:51.948545 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87e50a68-be0f-4288-afa7-eee7d0bee4e5-kube-api-access-ppkpn" (OuterVolumeSpecName: "kube-api-access-ppkpn") pod "87e50a68-be0f-4288-afa7-eee7d0bee4e5" (UID: "87e50a68-be0f-4288-afa7-eee7d0bee4e5"). InnerVolumeSpecName "kube-api-access-ppkpn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:35:52.047615 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:52.047574 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppkpn\" (UniqueName: \"kubernetes.io/projected/87e50a68-be0f-4288-afa7-eee7d0bee4e5-kube-api-access-ppkpn\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:35:52.440561 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:52.440526 2571 generic.go:356] "Generic (PLEG): container finished" podID="87e50a68-be0f-4288-afa7-eee7d0bee4e5" containerID="331850620a04614c058836393781a01291c02b5e049449c455f646834a5ce335" exitCode=0 Jul 28 17:35:52.440814 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:52.440575 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-p8f5l" Jul 28 17:35:52.440814 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:52.440623 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-p8f5l" event={"ID":"87e50a68-be0f-4288-afa7-eee7d0bee4e5","Type":"ContainerDied","Data":"331850620a04614c058836393781a01291c02b5e049449c455f646834a5ce335"} Jul 28 17:35:52.440814 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:52.440687 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-p8f5l" event={"ID":"87e50a68-be0f-4288-afa7-eee7d0bee4e5","Type":"ContainerDied","Data":"31fee98f92536c66ee2b9539debb7363d224937034d041eb1001efa31ed97dcf"} Jul 28 17:35:52.440814 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:52.440719 2571 scope.go:117] "RemoveContainer" containerID="331850620a04614c058836393781a01291c02b5e049449c455f646834a5ce335" Jul 28 17:35:52.449050 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:52.449029 2571 scope.go:117] "RemoveContainer" containerID="331850620a04614c058836393781a01291c02b5e049449c455f646834a5ce335" Jul 28 17:35:52.449369 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:35:52.449350 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"331850620a04614c058836393781a01291c02b5e049449c455f646834a5ce335\": container with ID starting with 331850620a04614c058836393781a01291c02b5e049449c455f646834a5ce335 not found: ID does not exist" containerID="331850620a04614c058836393781a01291c02b5e049449c455f646834a5ce335" Jul 28 17:35:52.449424 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:52.449380 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"331850620a04614c058836393781a01291c02b5e049449c455f646834a5ce335"} err="failed to get container status \"331850620a04614c058836393781a01291c02b5e049449c455f646834a5ce335\": rpc error: code = NotFound desc = could not find container \"331850620a04614c058836393781a01291c02b5e049449c455f646834a5ce335\": container with ID starting with 331850620a04614c058836393781a01291c02b5e049449c455f646834a5ce335 not found: ID does not exist" Jul 28 17:35:52.459897 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:52.459861 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7495bcd-p8f5l"] Jul 28 17:35:52.463724 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:52.463696 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7495bcd-p8f5l"] Jul 28 17:35:53.783580 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:53.783539 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87e50a68-be0f-4288-afa7-eee7d0bee4e5" path="/var/lib/kubelet/pods/87e50a68-be0f-4288-afa7-eee7d0bee4e5/volumes" Jul 28 17:35:56.306531 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306494 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7495bcd-59g86"] Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306728 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87e50a68-be0f-4288-afa7-eee7d0bee4e5" containerName="manager" Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306739 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="87e50a68-be0f-4288-afa7-eee7d0bee4e5" containerName="manager" Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306748 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="ui" Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306754 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="ui" Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306781 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="online" Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306787 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="online" Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306793 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="feast-init" Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306798 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="feast-init" Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306804 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="online" Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306809 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="online" Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306815 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="offline" Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306842 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="offline" Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306850 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="feast-apply" Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306855 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="feast-apply" Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306861 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="offline" Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306867 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="offline" Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306880 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="feast-apply" Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306885 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="feast-apply" Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306892 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="registry" Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306897 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="registry" Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306903 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="feast-init" Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306909 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="feast-init" Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306915 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="ui" Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306920 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="ui" Jul 28 17:35:56.306942 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306955 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="87e50a68-be0f-4288-afa7-eee7d0bee4e5" containerName="manager" Jul 28 17:35:56.307657 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306962 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="online" Jul 28 17:35:56.307657 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306967 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="ui" Jul 28 17:35:56.307657 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306973 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="offline" Jul 28 17:35:56.307657 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306978 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="ui" Jul 28 17:35:56.307657 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306984 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9651ab6-3dc3-4955-bcc4-2df854c0b14b" containerName="online" Jul 28 17:35:56.307657 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306989 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="registry" Jul 28 17:35:56.307657 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.306993 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f5b5fe2-8f7d-4e30-9234-e4f1fe4712a8" containerName="offline" Jul 28 17:35:56.311079 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.311059 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-59g86" Jul 28 17:35:56.313173 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.313134 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Jul 28 17:35:56.313300 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.313194 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Jul 28 17:35:56.313706 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.313687 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-cxfgb\"" Jul 28 17:35:56.319385 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.319358 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7495bcd-59g86"] Jul 28 17:35:56.382420 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.382385 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj5xp\" (UniqueName: \"kubernetes.io/projected/a1b56e9f-20ac-450b-aa93-8c95daeeb6d2-kube-api-access-sj5xp\") pod \"feast-operator-controller-manager-7495bcd-59g86\" (UID: \"a1b56e9f-20ac-450b-aa93-8c95daeeb6d2\") " pod="feast-operator-system/feast-operator-controller-manager-7495bcd-59g86" Jul 28 17:35:56.482976 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.482935 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj5xp\" (UniqueName: \"kubernetes.io/projected/a1b56e9f-20ac-450b-aa93-8c95daeeb6d2-kube-api-access-sj5xp\") pod \"feast-operator-controller-manager-7495bcd-59g86\" (UID: \"a1b56e9f-20ac-450b-aa93-8c95daeeb6d2\") " pod="feast-operator-system/feast-operator-controller-manager-7495bcd-59g86" Jul 28 17:35:56.491451 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.491420 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj5xp\" (UniqueName: \"kubernetes.io/projected/a1b56e9f-20ac-450b-aa93-8c95daeeb6d2-kube-api-access-sj5xp\") pod \"feast-operator-controller-manager-7495bcd-59g86\" (UID: \"a1b56e9f-20ac-450b-aa93-8c95daeeb6d2\") " pod="feast-operator-system/feast-operator-controller-manager-7495bcd-59g86" Jul 28 17:35:56.621413 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.621376 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-59g86" Jul 28 17:35:56.762075 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:56.762040 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7495bcd-59g86"] Jul 28 17:35:56.765221 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:35:56.765185 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1b56e9f_20ac_450b_aa93_8c95daeeb6d2.slice/crio-ebeb62d7ffceee12616673b6cea92936dbf9fd2c23a6795afcdc29cd16ce3cb0 WatchSource:0}: Error finding container ebeb62d7ffceee12616673b6cea92936dbf9fd2c23a6795afcdc29cd16ce3cb0: Status 404 returned error can't find the container with id ebeb62d7ffceee12616673b6cea92936dbf9fd2c23a6795afcdc29cd16ce3cb0 Jul 28 17:35:57.460529 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:57.460485 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-59g86" event={"ID":"a1b56e9f-20ac-450b-aa93-8c95daeeb6d2","Type":"ContainerStarted","Data":"1ce333c3abdf56cf59859b9c420401b2a3f35841ad11c5aecb58c274a0affb99"} Jul 28 17:35:57.460960 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:57.460549 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-59g86" event={"ID":"a1b56e9f-20ac-450b-aa93-8c95daeeb6d2","Type":"ContainerStarted","Data":"ebeb62d7ffceee12616673b6cea92936dbf9fd2c23a6795afcdc29cd16ce3cb0"} Jul 28 17:35:57.460960 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:57.460610 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-59g86" Jul 28 17:35:57.476332 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:35:57.476278 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-59g86" podStartSLOduration=1.476259958 podStartE2EDuration="1.476259958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:35:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:35:57.475397198 +0000 UTC m=+1618.195224909" watchObservedRunningTime="2026-07-28 17:35:57.476259958 +0000 UTC m=+1618.196087667" Jul 28 17:36:08.466670 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:08.466632 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-59g86" Jul 28 17:36:15.436021 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.435983 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl"] Jul 28 17:36:15.439650 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.439627 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:15.442147 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.442122 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Jul 28 17:36:15.442315 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.442301 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Jul 28 17:36:15.442411 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.442316 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Jul 28 17:36:15.442411 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.442383 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jul 28 17:36:15.442411 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.442388 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-m2tsr\"" Jul 28 17:36:15.442644 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.442431 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jul 28 17:36:15.442696 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.442683 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Jul 28 17:36:15.453182 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.453152 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl"] Jul 28 17:36:15.532005 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.531970 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-j9fsl\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:15.532243 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.532032 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/81d11314-e263-4602-9b65-fdfc111fc130-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-j9fsl\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:15.532243 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.532084 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55n8z\" (UniqueName: \"kubernetes.io/projected/81d11314-e263-4602-9b65-fdfc111fc130-kube-api-access-55n8z\") pod \"feast-simple-feast-setup-777d6cc998-j9fsl\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:15.532243 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.532108 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-j9fsl\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:15.532243 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.532128 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-j9fsl\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:15.532243 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.532187 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-j9fsl\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:15.633417 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.633384 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-j9fsl\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:15.633417 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.633422 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-j9fsl\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:15.633679 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.633450 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-j9fsl\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:15.633679 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.633488 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-j9fsl\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:15.633679 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.633513 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/81d11314-e263-4602-9b65-fdfc111fc130-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-j9fsl\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:15.633679 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:36:15.633558 2571 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Jul 28 17:36:15.633679 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.633563 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55n8z\" (UniqueName: \"kubernetes.io/projected/81d11314-e263-4602-9b65-fdfc111fc130-kube-api-access-55n8z\") pod \"feast-simple-feast-setup-777d6cc998-j9fsl\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:15.633679 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:36:15.633634 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-online-tls podName:81d11314-e263-4602-9b65-fdfc111fc130 nodeName:}" failed. No retries permitted until 2026-07-28 17:36:16.133612981 +0000 UTC m=+1636.853440689 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-online-tls") pod "feast-simple-feast-setup-777d6cc998-j9fsl" (UID: "81d11314-e263-4602-9b65-fdfc111fc130") : secret "feast-simple-feast-setup-online-tls" not found Jul 28 17:36:15.634027 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.633935 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/81d11314-e263-4602-9b65-fdfc111fc130-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-j9fsl\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:15.636074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.636054 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-j9fsl\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:15.636277 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.636258 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-j9fsl\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:15.636318 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.636258 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-j9fsl\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:15.641509 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:15.641483 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55n8z\" (UniqueName: \"kubernetes.io/projected/81d11314-e263-4602-9b65-fdfc111fc130-kube-api-access-55n8z\") pod \"feast-simple-feast-setup-777d6cc998-j9fsl\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:16.138603 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:16.138561 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-j9fsl\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:16.141062 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:16.141035 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-j9fsl\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:16.348936 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:16.348899 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:16.488152 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:16.488123 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl"] Jul 28 17:36:16.490501 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:36:16.490473 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81d11314_e263_4602_9b65_fdfc111fc130.slice/crio-d1359dc2227333093410b260b3711e47014f8c348519710947299767efd48049 WatchSource:0}: Error finding container d1359dc2227333093410b260b3711e47014f8c348519710947299767efd48049: Status 404 returned error can't find the container with id d1359dc2227333093410b260b3711e47014f8c348519710947299767efd48049 Jul 28 17:36:16.527836 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:16.527802 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" event={"ID":"81d11314-e263-4602-9b65-fdfc111fc130","Type":"ContainerStarted","Data":"d1359dc2227333093410b260b3711e47014f8c348519710947299767efd48049"} Jul 28 17:36:17.533118 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:17.533073 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" event={"ID":"81d11314-e263-4602-9b65-fdfc111fc130","Type":"ContainerStarted","Data":"fe0853e0c33bc5c67c4cafdac253dc1da3f1e6511f565f1cf9aef443fa7d7be5"} Jul 28 17:36:22.553245 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:22.553209 2571 generic.go:356] "Generic (PLEG): container finished" podID="81d11314-e263-4602-9b65-fdfc111fc130" containerID="fe0853e0c33bc5c67c4cafdac253dc1da3f1e6511f565f1cf9aef443fa7d7be5" exitCode=0 Jul 28 17:36:22.553596 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:22.553285 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" event={"ID":"81d11314-e263-4602-9b65-fdfc111fc130","Type":"ContainerDied","Data":"fe0853e0c33bc5c67c4cafdac253dc1da3f1e6511f565f1cf9aef443fa7d7be5"} Jul 28 17:36:23.558796 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:23.558743 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" event={"ID":"81d11314-e263-4602-9b65-fdfc111fc130","Type":"ContainerStarted","Data":"ebb20729731e8d3f2b9b28ff8fdd00cdc26153d4d31cbedb736bdc0d60c2207c"} Jul 28 17:36:28.581743 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:28.581699 2571 generic.go:356] "Generic (PLEG): container finished" podID="81d11314-e263-4602-9b65-fdfc111fc130" containerID="ebb20729731e8d3f2b9b28ff8fdd00cdc26153d4d31cbedb736bdc0d60c2207c" exitCode=0 Jul 28 17:36:28.582221 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:28.581781 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" event={"ID":"81d11314-e263-4602-9b65-fdfc111fc130","Type":"ContainerDied","Data":"ebb20729731e8d3f2b9b28ff8fdd00cdc26153d4d31cbedb736bdc0d60c2207c"} Jul 28 17:36:29.591846 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:29.591801 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" event={"ID":"81d11314-e263-4602-9b65-fdfc111fc130","Type":"ContainerStarted","Data":"3e7e97db4c4ccbadc066a5d28c694454aa4846fbd59ed2d841e7afa2cb53e741"} Jul 28 17:36:29.592340 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:29.591867 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" event={"ID":"81d11314-e263-4602-9b65-fdfc111fc130","Type":"ContainerStarted","Data":"49fc6f002200a60cb16d217cb62e17f51d69ccb0817d562472ec59768b6a7ced"} Jul 28 17:36:29.592340 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:29.591892 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" event={"ID":"81d11314-e263-4602-9b65-fdfc111fc130","Type":"ContainerStarted","Data":"090b3f9fb2c613961b049facc6569f6a030f8ba3c75d519d5c310d9092d87b58"} Jul 28 17:36:29.592340 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:29.591912 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" event={"ID":"81d11314-e263-4602-9b65-fdfc111fc130","Type":"ContainerStarted","Data":"e3effb7c7dfe046ee4d5b92eaa625d347823927d9ea6c26ade9154da4c171e03"} Jul 28 17:36:29.617644 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:29.617525 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" podStartSLOduration=14.617502108 podStartE2EDuration="14.617502108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:36:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:36:29.615847219 +0000 UTC m=+1650.335674959" watchObservedRunningTime="2026-07-28 17:36:29.617502108 +0000 UTC m=+1650.337329817" Jul 28 17:36:31.349872 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:31.349832 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:31.350455 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:31.350430 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:31.350455 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:31.350463 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:31.350659 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:31.350480 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:31.352264 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:31.352200 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.22:6571: connect: connection refused" Jul 28 17:36:31.352264 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:31.352222 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.22:8816: connect: connection refused" Jul 28 17:36:31.352465 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:31.352282 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused" Jul 28 17:36:31.352465 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:31.352243 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": dial tcp 10.133.0.22:6567: connect: connection refused" Jul 28 17:36:34.349660 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:34.349611 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.22:8816: connect: connection refused" Jul 28 17:36:34.350198 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:34.349624 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.22:6571: connect: connection refused" Jul 28 17:36:34.350198 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:34.349616 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused" Jul 28 17:36:34.350198 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:34.349956 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": dial tcp 10.133.0.22:6567: connect: connection refused" Jul 28 17:36:36.349871 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:36.349833 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:36.350263 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:36.349914 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:36.350263 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:36.349952 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:36.350263 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:36.349964 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:37.350510 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:37.350481 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:37.350919 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:37.350595 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:37.350919 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:37.350898 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:37.355532 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:37.355515 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:37.634945 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:37.634912 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:37.635118 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:37.634985 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:37.635118 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:37.635014 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:36:37.637877 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:36:37.637853 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:37:38.425804 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.425748 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl"] Jul 28 17:37:38.426354 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.425992 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="registry" containerID="cri-o://e3effb7c7dfe046ee4d5b92eaa625d347823927d9ea6c26ade9154da4c171e03" gracePeriod=30 Jul 28 17:37:38.426354 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.426051 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="ui" containerID="cri-o://3e7e97db4c4ccbadc066a5d28c694454aa4846fbd59ed2d841e7afa2cb53e741" gracePeriod=30 Jul 28 17:37:38.426354 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.426086 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="offline" containerID="cri-o://49fc6f002200a60cb16d217cb62e17f51d69ccb0817d562472ec59768b6a7ced" gracePeriod=30 Jul 28 17:37:38.426354 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.426097 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="online" containerID="cri-o://090b3f9fb2c613961b049facc6569f6a030f8ba3c75d519d5c310d9092d87b58" gracePeriod=30 Jul 28 17:37:38.745356 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.745258 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln"] Jul 28 17:37:38.748745 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.748728 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:38.750872 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.750849 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-9c9kc\"" Jul 28 17:37:38.760893 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.760829 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln"] Jul 28 17:37:38.869823 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.869754 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-2jsln\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:38.869823 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.869824 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8701697a-9097-4a29-bc2c-53e0db69e3a3-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-2jsln\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:38.870062 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.869877 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-747ld\" (UniqueName: \"kubernetes.io/projected/8701697a-9097-4a29-bc2c-53e0db69e3a3-kube-api-access-747ld\") pod \"feast-simple-feast-setup-777d6cc998-2jsln\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:38.870062 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.869973 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-2jsln\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:38.870062 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.870026 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-2jsln\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:38.870206 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.870066 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-2jsln\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:38.970971 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.970936 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-2jsln\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:38.971193 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.970981 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-2jsln\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:38.971193 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.971008 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-2jsln\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:38.971193 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.971041 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-2jsln\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:38.971193 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.971095 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8701697a-9097-4a29-bc2c-53e0db69e3a3-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-2jsln\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:38.971193 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.971144 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-747ld\" (UniqueName: \"kubernetes.io/projected/8701697a-9097-4a29-bc2c-53e0db69e3a3-kube-api-access-747ld\") pod \"feast-simple-feast-setup-777d6cc998-2jsln\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:38.971193 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:37:38.971168 2571 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Jul 28 17:37:38.971521 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:37:38.971174 2571 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Jul 28 17:37:38.971521 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:37:38.971258 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-online-tls podName:8701697a-9097-4a29-bc2c-53e0db69e3a3 nodeName:}" failed. No retries permitted until 2026-07-28 17:37:39.471233319 +0000 UTC m=+1720.191061027 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-online-tls") pod "feast-simple-feast-setup-777d6cc998-2jsln" (UID: "8701697a-9097-4a29-bc2c-53e0db69e3a3") : secret "feast-simple-feast-setup-online-tls" not found Jul 28 17:37:38.971521 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:37:38.971286 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-offline-tls podName:8701697a-9097-4a29-bc2c-53e0db69e3a3 nodeName:}" failed. No retries permitted until 2026-07-28 17:37:39.471273986 +0000 UTC m=+1720.191101677 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-offline-tls") pod "feast-simple-feast-setup-777d6cc998-2jsln" (UID: "8701697a-9097-4a29-bc2c-53e0db69e3a3") : secret "feast-simple-feast-setup-offline-tls" not found Jul 28 17:37:38.971682 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.971548 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8701697a-9097-4a29-bc2c-53e0db69e3a3-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-2jsln\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:38.973897 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.973849 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-2jsln\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:38.974165 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.974119 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-2jsln\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:38.979595 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:38.979567 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-747ld\" (UniqueName: \"kubernetes.io/projected/8701697a-9097-4a29-bc2c-53e0db69e3a3-kube-api-access-747ld\") pod \"feast-simple-feast-setup-777d6cc998-2jsln\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:39.475880 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:39.475841 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-2jsln\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:39.476277 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:39.475907 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-2jsln\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:39.478325 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:39.478302 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-2jsln\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:39.478454 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:39.478427 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-2jsln\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:39.658229 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:39.658187 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:39.795304 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:39.795267 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln"] Jul 28 17:37:39.798073 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:37:39.798040 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8701697a_9097_4a29_bc2c_53e0db69e3a3.slice/crio-79d3d62655756a4250fb130864aca500c7513823c1ac6c7d163fb790bb0451fa WatchSource:0}: Error finding container 79d3d62655756a4250fb130864aca500c7513823c1ac6c7d163fb790bb0451fa: Status 404 returned error can't find the container with id 79d3d62655756a4250fb130864aca500c7513823c1ac6c7d163fb790bb0451fa Jul 28 17:37:39.856612 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:39.856581 2571 generic.go:356] "Generic (PLEG): container finished" podID="81d11314-e263-4602-9b65-fdfc111fc130" containerID="3e7e97db4c4ccbadc066a5d28c694454aa4846fbd59ed2d841e7afa2cb53e741" exitCode=0 Jul 28 17:37:39.856612 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:39.856607 2571 generic.go:356] "Generic (PLEG): container finished" podID="81d11314-e263-4602-9b65-fdfc111fc130" containerID="090b3f9fb2c613961b049facc6569f6a030f8ba3c75d519d5c310d9092d87b58" exitCode=0 Jul 28 17:37:39.856820 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:39.856650 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" event={"ID":"81d11314-e263-4602-9b65-fdfc111fc130","Type":"ContainerDied","Data":"3e7e97db4c4ccbadc066a5d28c694454aa4846fbd59ed2d841e7afa2cb53e741"} Jul 28 17:37:39.856820 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:39.856694 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" event={"ID":"81d11314-e263-4602-9b65-fdfc111fc130","Type":"ContainerDied","Data":"090b3f9fb2c613961b049facc6569f6a030f8ba3c75d519d5c310d9092d87b58"} Jul 28 17:37:39.858315 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:39.858292 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" event={"ID":"8701697a-9097-4a29-bc2c-53e0db69e3a3","Type":"ContainerStarted","Data":"79d3d62655756a4250fb130864aca500c7513823c1ac6c7d163fb790bb0451fa"} Jul 28 17:37:40.864019 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:40.863971 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" event={"ID":"8701697a-9097-4a29-bc2c-53e0db69e3a3","Type":"ContainerStarted","Data":"009352ad777b74e50b34d4f11fa6567bc54a204a05be069175eeeb711605fd3f"} Jul 28 17:37:44.879894 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:44.879858 2571 generic.go:356] "Generic (PLEG): container finished" podID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerID="009352ad777b74e50b34d4f11fa6567bc54a204a05be069175eeeb711605fd3f" exitCode=0 Jul 28 17:37:44.880333 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:44.879942 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" event={"ID":"8701697a-9097-4a29-bc2c-53e0db69e3a3","Type":"ContainerDied","Data":"009352ad777b74e50b34d4f11fa6567bc54a204a05be069175eeeb711605fd3f"} Jul 28 17:37:45.885457 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:45.885420 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" event={"ID":"8701697a-9097-4a29-bc2c-53e0db69e3a3","Type":"ContainerStarted","Data":"d05fdbab25b0f0b97f500cb4ef940f2e8040c4617462a0e2d222104cb1de412f"} Jul 28 17:37:47.634643 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:47.634599 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": dial tcp 10.133.0.22:6567: connect: connection refused" Jul 28 17:37:47.635115 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:47.634597 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused" Jul 28 17:37:50.904079 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:50.904044 2571 generic.go:356] "Generic (PLEG): container finished" podID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerID="d05fdbab25b0f0b97f500cb4ef940f2e8040c4617462a0e2d222104cb1de412f" exitCode=0 Jul 28 17:37:50.904447 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:50.904114 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" event={"ID":"8701697a-9097-4a29-bc2c-53e0db69e3a3","Type":"ContainerDied","Data":"d05fdbab25b0f0b97f500cb4ef940f2e8040c4617462a0e2d222104cb1de412f"} Jul 28 17:37:51.913799 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:51.913735 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" event={"ID":"8701697a-9097-4a29-bc2c-53e0db69e3a3","Type":"ContainerStarted","Data":"05dfc9285e85493e8d536f525229391adbb1c099c461271590a2b6ccba3f58e7"} Jul 28 17:37:51.914275 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:51.913824 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" event={"ID":"8701697a-9097-4a29-bc2c-53e0db69e3a3","Type":"ContainerStarted","Data":"41000880a25121eaf0f309b3221e61c81f93d6107966dc80cd11517a3153a60d"} Jul 28 17:37:51.914275 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:51.913848 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" event={"ID":"8701697a-9097-4a29-bc2c-53e0db69e3a3","Type":"ContainerStarted","Data":"6b22f0f4d7af1185fe54b87e32ecdfb8a4cee851b8decb08b836809923e7adc9"} Jul 28 17:37:51.914275 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:51.913867 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" event={"ID":"8701697a-9097-4a29-bc2c-53e0db69e3a3","Type":"ContainerStarted","Data":"d167a06c9155f4caaf250fd9567c16ae580b032183dbadf279efa99267c762a6"} Jul 28 17:37:51.938827 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:51.938559 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" podStartSLOduration=13.938538063 podStartE2EDuration="13.938538063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:37:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:37:51.938374006 +0000 UTC m=+1732.658201718" watchObservedRunningTime="2026-07-28 17:37:51.938538063 +0000 UTC m=+1732.658365774" Jul 28 17:37:54.659169 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:54.659127 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:54.659802 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:54.659753 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:54.660288 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:54.660263 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:54.660387 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:54.660310 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:54.661461 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:54.661431 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.23:6571: connect: connection refused" Jul 28 17:37:54.661681 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:54.661470 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused" Jul 28 17:37:54.661788 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:54.661472 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused" Jul 28 17:37:54.661788 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:54.661506 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.23:8816: connect: connection refused" Jul 28 17:37:57.635169 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:57.635117 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused" Jul 28 17:37:57.635169 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:57.635141 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": dial tcp 10.133.0.22:6567: connect: connection refused" Jul 28 17:37:57.660164 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:57.660128 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:57.660721 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:57.660695 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:57.660863 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:57.660739 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:57.660929 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:57.660875 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:57.660981 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:57.660937 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:57.665496 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:57.665472 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:57.945095 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:57.945001 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:57.945095 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:57.945078 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:57.945316 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:57.945100 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:57.945555 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:57.945531 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:57.945667 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:57.945618 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:37:57.948376 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:37:57.948355 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:38:07.635219 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:07.635166 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused" Jul 28 17:38:07.635675 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:07.635170 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": dial tcp 10.133.0.22:6567: connect: connection refused" Jul 28 17:38:07.635675 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:07.635345 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:38:07.635675 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:07.635414 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:38:08.987151 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:08.987109 2571 generic.go:356] "Generic (PLEG): container finished" podID="81d11314-e263-4602-9b65-fdfc111fc130" containerID="49fc6f002200a60cb16d217cb62e17f51d69ccb0817d562472ec59768b6a7ced" exitCode=137 Jul 28 17:38:08.987151 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:08.987133 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" event={"ID":"81d11314-e263-4602-9b65-fdfc111fc130","Type":"ContainerDied","Data":"49fc6f002200a60cb16d217cb62e17f51d69ccb0817d562472ec59768b6a7ced"} Jul 28 17:38:08.987553 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:08.987179 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" event={"ID":"81d11314-e263-4602-9b65-fdfc111fc130","Type":"ContainerDied","Data":"e3effb7c7dfe046ee4d5b92eaa625d347823927d9ea6c26ade9154da4c171e03"} Jul 28 17:38:08.987553 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:08.987146 2571 generic.go:356] "Generic (PLEG): container finished" podID="81d11314-e263-4602-9b65-fdfc111fc130" containerID="e3effb7c7dfe046ee4d5b92eaa625d347823927d9ea6c26ade9154da4c171e03" exitCode=137 Jul 28 17:38:09.070408 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:09.070380 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:38:09.110423 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:09.110390 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-ui-tls\") pod \"81d11314-e263-4602-9b65-fdfc111fc130\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " Jul 28 17:38:09.110588 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:09.110451 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-registry-tls\") pod \"81d11314-e263-4602-9b65-fdfc111fc130\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " Jul 28 17:38:09.110588 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:09.110472 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55n8z\" (UniqueName: \"kubernetes.io/projected/81d11314-e263-4602-9b65-fdfc111fc130-kube-api-access-55n8z\") pod \"81d11314-e263-4602-9b65-fdfc111fc130\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " Jul 28 17:38:09.110588 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:09.110512 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-online-tls\") pod \"81d11314-e263-4602-9b65-fdfc111fc130\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " Jul 28 17:38:09.110588 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:09.110530 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-offline-tls\") pod \"81d11314-e263-4602-9b65-fdfc111fc130\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " Jul 28 17:38:09.110829 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:09.110657 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/81d11314-e263-4602-9b65-fdfc111fc130-feast-data\") pod \"81d11314-e263-4602-9b65-fdfc111fc130\" (UID: \"81d11314-e263-4602-9b65-fdfc111fc130\") " Jul 28 17:38:09.111130 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:09.111104 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81d11314-e263-4602-9b65-fdfc111fc130-feast-data" (OuterVolumeSpecName: "feast-data") pod "81d11314-e263-4602-9b65-fdfc111fc130" (UID: "81d11314-e263-4602-9b65-fdfc111fc130"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:38:09.112962 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:09.112933 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "81d11314-e263-4602-9b65-fdfc111fc130" (UID: "81d11314-e263-4602-9b65-fdfc111fc130"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:38:09.113072 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:09.112935 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81d11314-e263-4602-9b65-fdfc111fc130-kube-api-access-55n8z" (OuterVolumeSpecName: "kube-api-access-55n8z") pod "81d11314-e263-4602-9b65-fdfc111fc130" (UID: "81d11314-e263-4602-9b65-fdfc111fc130"). InnerVolumeSpecName "kube-api-access-55n8z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:38:09.113119 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:09.113070 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-online-tls" (OuterVolumeSpecName: "online-tls") pod "81d11314-e263-4602-9b65-fdfc111fc130" (UID: "81d11314-e263-4602-9b65-fdfc111fc130"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:38:09.113119 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:09.113093 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "81d11314-e263-4602-9b65-fdfc111fc130" (UID: "81d11314-e263-4602-9b65-fdfc111fc130"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:38:09.113198 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:09.113182 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "81d11314-e263-4602-9b65-fdfc111fc130" (UID: "81d11314-e263-4602-9b65-fdfc111fc130"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:38:09.211829 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:09.211725 2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-registry-tls\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:38:09.211829 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:09.211792 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55n8z\" (UniqueName: \"kubernetes.io/projected/81d11314-e263-4602-9b65-fdfc111fc130-kube-api-access-55n8z\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:38:09.211829 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:09.211805 2571 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-online-tls\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:38:09.211829 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:09.211816 2571 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-offline-tls\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:38:09.211829 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:09.211823 2571 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/81d11314-e263-4602-9b65-fdfc111fc130-feast-data\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:38:09.211829 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:09.211831 2571 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/81d11314-e263-4602-9b65-fdfc111fc130-ui-tls\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:38:09.993090 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:09.993051 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" event={"ID":"81d11314-e263-4602-9b65-fdfc111fc130","Type":"ContainerDied","Data":"d1359dc2227333093410b260b3711e47014f8c348519710947299767efd48049"} Jul 28 17:38:09.993552 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:09.993101 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl" Jul 28 17:38:09.993552 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:09.993112 2571 scope.go:117] "RemoveContainer" containerID="3e7e97db4c4ccbadc066a5d28c694454aa4846fbd59ed2d841e7afa2cb53e741" Jul 28 17:38:10.001794 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:10.001739 2571 scope.go:117] "RemoveContainer" containerID="49fc6f002200a60cb16d217cb62e17f51d69ccb0817d562472ec59768b6a7ced" Jul 28 17:38:10.009227 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:10.009193 2571 scope.go:117] "RemoveContainer" containerID="090b3f9fb2c613961b049facc6569f6a030f8ba3c75d519d5c310d9092d87b58" Jul 28 17:38:10.011515 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:10.011490 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl"] Jul 28 17:38:10.015184 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:10.015165 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-j9fsl"] Jul 28 17:38:10.017232 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:10.017218 2571 scope.go:117] "RemoveContainer" containerID="e3effb7c7dfe046ee4d5b92eaa625d347823927d9ea6c26ade9154da4c171e03" Jul 28 17:38:10.024348 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:10.024330 2571 scope.go:117] "RemoveContainer" containerID="ebb20729731e8d3f2b9b28ff8fdd00cdc26153d4d31cbedb736bdc0d60c2207c" Jul 28 17:38:10.031180 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:10.031161 2571 scope.go:117] "RemoveContainer" containerID="fe0853e0c33bc5c67c4cafdac253dc1da3f1e6511f565f1cf9aef443fa7d7be5" Jul 28 17:38:11.784816 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:38:11.784754 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81d11314-e263-4602-9b65-fdfc111fc130" path="/var/lib/kubelet/pods/81d11314-e263-4602-9b65-fdfc111fc130/volumes" Jul 28 17:39:00.198779 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.198731 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg"] Jul 28 17:39:00.199190 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.199001 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="ui" Jul 28 17:39:00.199190 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.199013 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="ui" Jul 28 17:39:00.199190 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.199028 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="registry" Jul 28 17:39:00.199190 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.199033 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="registry" Jul 28 17:39:00.199190 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.199041 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="feast-init" Jul 28 17:39:00.199190 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.199047 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="feast-init" Jul 28 17:39:00.199190 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.199059 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="feast-apply" Jul 28 17:39:00.199190 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.199065 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="feast-apply" Jul 28 17:39:00.199190 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.199073 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="online" Jul 28 17:39:00.199190 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.199079 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="online" Jul 28 17:39:00.199190 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.199085 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="offline" Jul 28 17:39:00.199190 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.199090 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="offline" Jul 28 17:39:00.199190 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.199133 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="online" Jul 28 17:39:00.199190 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.199139 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="offline" Jul 28 17:39:00.199190 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.199144 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="ui" Jul 28 17:39:00.199190 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.199152 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="81d11314-e263-4602-9b65-fdfc111fc130" containerName="registry" Jul 28 17:39:00.203431 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.203406 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:00.210016 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.206242 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Jul 28 17:39:00.210016 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.206310 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Jul 28 17:39:00.210016 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.206398 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Jul 28 17:39:00.210016 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.206531 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Jul 28 17:39:00.210016 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.207388 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Jul 28 17:39:00.210016 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.207415 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Jul 28 17:39:00.210016 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.208446 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-ksxrl\"" Jul 28 17:39:00.214698 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.214665 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg"] Jul 28 17:39:00.298122 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.298083 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4czh\" (UniqueName: \"kubernetes.io/projected/3e71bdc8-f806-4351-901b-374d437a5bc7-kube-api-access-v4czh\") pod \"feast-simple-feast-remote-setup-db6955c6c-kb7hg\" (UID: \"3e71bdc8-f806-4351-901b-374d437a5bc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:00.298122 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.298124 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3e71bdc8-f806-4351-901b-374d437a5bc7-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-kb7hg\" (UID: \"3e71bdc8-f806-4351-901b-374d437a5bc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:00.298361 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.298179 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3e71bdc8-f806-4351-901b-374d437a5bc7-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-kb7hg\" (UID: \"3e71bdc8-f806-4351-901b-374d437a5bc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:00.298361 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.298218 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3e71bdc8-f806-4351-901b-374d437a5bc7-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-kb7hg\" (UID: \"3e71bdc8-f806-4351-901b-374d437a5bc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:00.298361 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.298240 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3e71bdc8-f806-4351-901b-374d437a5bc7-feast-data\") pod \"feast-simple-feast-remote-setup-db6955c6c-kb7hg\" (UID: \"3e71bdc8-f806-4351-901b-374d437a5bc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:00.298361 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.298277 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/3e71bdc8-f806-4351-901b-374d437a5bc7-registry-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-kb7hg\" (UID: \"3e71bdc8-f806-4351-901b-374d437a5bc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:00.399121 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.399078 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/3e71bdc8-f806-4351-901b-374d437a5bc7-registry-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-kb7hg\" (UID: \"3e71bdc8-f806-4351-901b-374d437a5bc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:00.399326 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.399146 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4czh\" (UniqueName: \"kubernetes.io/projected/3e71bdc8-f806-4351-901b-374d437a5bc7-kube-api-access-v4czh\") pod \"feast-simple-feast-remote-setup-db6955c6c-kb7hg\" (UID: \"3e71bdc8-f806-4351-901b-374d437a5bc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:00.399326 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.399172 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3e71bdc8-f806-4351-901b-374d437a5bc7-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-kb7hg\" (UID: \"3e71bdc8-f806-4351-901b-374d437a5bc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:00.399326 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.399198 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3e71bdc8-f806-4351-901b-374d437a5bc7-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-kb7hg\" (UID: \"3e71bdc8-f806-4351-901b-374d437a5bc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:00.399326 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.399217 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3e71bdc8-f806-4351-901b-374d437a5bc7-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-kb7hg\" (UID: \"3e71bdc8-f806-4351-901b-374d437a5bc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:00.399326 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.399242 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3e71bdc8-f806-4351-901b-374d437a5bc7-feast-data\") pod \"feast-simple-feast-remote-setup-db6955c6c-kb7hg\" (UID: \"3e71bdc8-f806-4351-901b-374d437a5bc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:00.399684 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.399662 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3e71bdc8-f806-4351-901b-374d437a5bc7-feast-data\") pod \"feast-simple-feast-remote-setup-db6955c6c-kb7hg\" (UID: \"3e71bdc8-f806-4351-901b-374d437a5bc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:00.400102 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.400080 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/3e71bdc8-f806-4351-901b-374d437a5bc7-registry-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-kb7hg\" (UID: \"3e71bdc8-f806-4351-901b-374d437a5bc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:00.401896 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.401872 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3e71bdc8-f806-4351-901b-374d437a5bc7-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-kb7hg\" (UID: \"3e71bdc8-f806-4351-901b-374d437a5bc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:00.401896 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.401890 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3e71bdc8-f806-4351-901b-374d437a5bc7-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-kb7hg\" (UID: \"3e71bdc8-f806-4351-901b-374d437a5bc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:00.402020 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.401877 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3e71bdc8-f806-4351-901b-374d437a5bc7-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-kb7hg\" (UID: \"3e71bdc8-f806-4351-901b-374d437a5bc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:00.407307 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.407281 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4czh\" (UniqueName: \"kubernetes.io/projected/3e71bdc8-f806-4351-901b-374d437a5bc7-kube-api-access-v4czh\") pod \"feast-simple-feast-remote-setup-db6955c6c-kb7hg\" (UID: \"3e71bdc8-f806-4351-901b-374d437a5bc7\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:00.519637 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.519520 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:00.657500 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:00.657463 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg"] Jul 28 17:39:00.660754 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:39:00.660723 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e71bdc8_f806_4351_901b_374d437a5bc7.slice/crio-0ad7b517ed34360a620996a3a21ff77f186420226bd94f0bdb5713c9979fabed WatchSource:0}: Error finding container 0ad7b517ed34360a620996a3a21ff77f186420226bd94f0bdb5713c9979fabed: Status 404 returned error can't find the container with id 0ad7b517ed34360a620996a3a21ff77f186420226bd94f0bdb5713c9979fabed Jul 28 17:39:01.169954 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:01.169913 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" event={"ID":"3e71bdc8-f806-4351-901b-374d437a5bc7","Type":"ContainerStarted","Data":"51deb8e464486423fa6c367ded3fbb6dcfe82b2762ace1c446a0c2296fbb9272"} Jul 28 17:39:01.169954 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:01.169958 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" event={"ID":"3e71bdc8-f806-4351-901b-374d437a5bc7","Type":"ContainerStarted","Data":"0ad7b517ed34360a620996a3a21ff77f186420226bd94f0bdb5713c9979fabed"} Jul 28 17:39:06.191247 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:06.191206 2571 generic.go:356] "Generic (PLEG): container finished" podID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerID="51deb8e464486423fa6c367ded3fbb6dcfe82b2762ace1c446a0c2296fbb9272" exitCode=0 Jul 28 17:39:06.191645 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:06.191282 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" event={"ID":"3e71bdc8-f806-4351-901b-374d437a5bc7","Type":"ContainerDied","Data":"51deb8e464486423fa6c367ded3fbb6dcfe82b2762ace1c446a0c2296fbb9272"} Jul 28 17:39:07.197265 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:07.197224 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" event={"ID":"3e71bdc8-f806-4351-901b-374d437a5bc7","Type":"ContainerStarted","Data":"f86d98ac32855100132f3d101d686270e9189c43b68f56f663215c593c5487b7"} Jul 28 17:39:13.220983 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:13.220945 2571 generic.go:356] "Generic (PLEG): container finished" podID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerID="f86d98ac32855100132f3d101d686270e9189c43b68f56f663215c593c5487b7" exitCode=0 Jul 28 17:39:13.221368 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:13.220995 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" event={"ID":"3e71bdc8-f806-4351-901b-374d437a5bc7","Type":"ContainerDied","Data":"f86d98ac32855100132f3d101d686270e9189c43b68f56f663215c593c5487b7"} Jul 28 17:39:14.230075 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:14.230038 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" event={"ID":"3e71bdc8-f806-4351-901b-374d437a5bc7","Type":"ContainerStarted","Data":"6f6567c0e39cda511bb97a362a81473f8d4cc6aecb90beb0005e13991f7d4bae"} Jul 28 17:39:14.230075 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:14.230081 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" event={"ID":"3e71bdc8-f806-4351-901b-374d437a5bc7","Type":"ContainerStarted","Data":"26309d3e4b28753dc4c35fad5cfdf605c512d7f443892e43f9a920c60deec18d"} Jul 28 17:39:14.230626 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:14.230094 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" event={"ID":"3e71bdc8-f806-4351-901b-374d437a5bc7","Type":"ContainerStarted","Data":"e17f95f469985c54e8badb4df5d4d84af557138d6d227a6687a087ec4c96af64"} Jul 28 17:39:14.253203 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:14.253145 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" podStartSLOduration=14.253123992999999 podStartE2EDuration="14.253123993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:39:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:39:14.25129974 +0000 UTC m=+1814.971127453" watchObservedRunningTime="2026-07-28 17:39:14.253123993 +0000 UTC m=+1814.972951704" Jul 28 17:39:15.520128 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:15.520080 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:15.520128 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:15.520141 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:15.520700 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:15.520157 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:15.521836 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:15.521784 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Jul 28 17:39:15.521836 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:15.521817 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.24:8816: connect: connection refused" Jul 28 17:39:15.521999 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:15.521831 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Jul 28 17:39:18.520736 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:18.520667 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.24:8816: connect: connection refused" Jul 28 17:39:18.521262 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:18.520691 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Jul 28 17:39:18.521262 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:18.520699 2571 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Jul 28 17:39:20.520507 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:20.520457 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:20.520507 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:20.520518 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:20.520999 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:20.520541 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:21.521344 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:21.521314 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:21.521830 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:21.521487 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:21.527486 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:21.527461 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:22.266359 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:22.266327 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:22.266554 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:22.266521 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:39:22.270556 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:39:22.270530 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:40:23.279023 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:23.278937 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg"] Jul 28 17:40:23.281467 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:23.279185 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="online" containerID="cri-o://e17f95f469985c54e8badb4df5d4d84af557138d6d227a6687a087ec4c96af64" gracePeriod=30 Jul 28 17:40:23.281467 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:23.279253 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="offline" containerID="cri-o://26309d3e4b28753dc4c35fad5cfdf605c512d7f443892e43f9a920c60deec18d" gracePeriod=30 Jul 28 17:40:23.281467 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:23.279245 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="ui" containerID="cri-o://6f6567c0e39cda511bb97a362a81473f8d4cc6aecb90beb0005e13991f7d4bae" gracePeriod=30 Jul 28 17:40:23.425628 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:23.425482 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln"] Jul 28 17:40:23.425853 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:23.425818 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="registry" containerID="cri-o://d167a06c9155f4caaf250fd9567c16ae580b032183dbadf279efa99267c762a6" gracePeriod=30 Jul 28 17:40:23.426072 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:23.426045 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="ui" containerID="cri-o://05dfc9285e85493e8d536f525229391adbb1c099c461271590a2b6ccba3f58e7" gracePeriod=30 Jul 28 17:40:23.426207 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:23.426108 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="online" containerID="cri-o://6b22f0f4d7af1185fe54b87e32ecdfb8a4cee851b8decb08b836809923e7adc9" gracePeriod=30 Jul 28 17:40:23.426336 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:23.426255 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="offline" containerID="cri-o://41000880a25121eaf0f309b3221e61c81f93d6107966dc80cd11517a3153a60d" gracePeriod=30 Jul 28 17:40:23.924273 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:40:23.924223 2571 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e71bdc8_f806_4351_901b_374d437a5bc7.slice/crio-6f6567c0e39cda511bb97a362a81473f8d4cc6aecb90beb0005e13991f7d4bae.scope\": RecentStats: unable to find data in memory cache]" Jul 28 17:40:24.498836 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:24.498736 2571 generic.go:356] "Generic (PLEG): container finished" podID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerID="6f6567c0e39cda511bb97a362a81473f8d4cc6aecb90beb0005e13991f7d4bae" exitCode=0 Jul 28 17:40:24.498836 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:24.498783 2571 generic.go:356] "Generic (PLEG): container finished" podID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerID="e17f95f469985c54e8badb4df5d4d84af557138d6d227a6687a087ec4c96af64" exitCode=0 Jul 28 17:40:24.498836 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:24.498809 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" event={"ID":"3e71bdc8-f806-4351-901b-374d437a5bc7","Type":"ContainerDied","Data":"6f6567c0e39cda511bb97a362a81473f8d4cc6aecb90beb0005e13991f7d4bae"} Jul 28 17:40:24.499313 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:24.498851 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" event={"ID":"3e71bdc8-f806-4351-901b-374d437a5bc7","Type":"ContainerDied","Data":"e17f95f469985c54e8badb4df5d4d84af557138d6d227a6687a087ec4c96af64"} Jul 28 17:40:24.501636 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:24.501607 2571 generic.go:356] "Generic (PLEG): container finished" podID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerID="05dfc9285e85493e8d536f525229391adbb1c099c461271590a2b6ccba3f58e7" exitCode=0 Jul 28 17:40:24.501636 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:24.501632 2571 generic.go:356] "Generic (PLEG): container finished" podID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerID="6b22f0f4d7af1185fe54b87e32ecdfb8a4cee851b8decb08b836809923e7adc9" exitCode=0 Jul 28 17:40:24.501820 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:24.501670 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" event={"ID":"8701697a-9097-4a29-bc2c-53e0db69e3a3","Type":"ContainerDied","Data":"05dfc9285e85493e8d536f525229391adbb1c099c461271590a2b6ccba3f58e7"} Jul 28 17:40:24.501820 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:24.501707 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" event={"ID":"8701697a-9097-4a29-bc2c-53e0db69e3a3","Type":"ContainerDied","Data":"6b22f0f4d7af1185fe54b87e32ecdfb8a4cee851b8decb08b836809923e7adc9"} Jul 28 17:40:27.945238 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:27.945187 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused" Jul 28 17:40:27.945238 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:27.945223 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused" Jul 28 17:40:32.266516 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:32.266460 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Jul 28 17:40:32.266949 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:32.266479 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Jul 28 17:40:37.945486 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:37.945439 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused" Jul 28 17:40:37.945976 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:37.945439 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused" Jul 28 17:40:42.267043 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:42.266996 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Jul 28 17:40:42.267043 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:42.267016 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Jul 28 17:40:47.945329 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:47.945291 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="online" probeResult="failure" output="Get \"https://10.133.0.23:6567/health\": dial tcp 10.133.0.23:6567: connect: connection refused" Jul 28 17:40:47.945879 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:47.945285 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.23:8443: connect: connection refused" Jul 28 17:40:47.945879 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:47.945395 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:40:47.945879 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:47.945443 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:40:52.266330 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:52.266277 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.24:8443: connect: connection refused" Jul 28 17:40:52.266846 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:52.266296 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="online" probeResult="failure" output="Get \"https://10.133.0.24:6567/health\": dial tcp 10.133.0.24:6567: connect: connection refused" Jul 28 17:40:52.266846 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:52.266426 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:40:52.266846 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:52.266566 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:40:53.538133 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.538063 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:40:53.586243 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.586204 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3e71bdc8-f806-4351-901b-374d437a5bc7-ui-tls\") pod \"3e71bdc8-f806-4351-901b-374d437a5bc7\" (UID: \"3e71bdc8-f806-4351-901b-374d437a5bc7\") " Jul 28 17:40:53.586243 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.586252 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3e71bdc8-f806-4351-901b-374d437a5bc7-offline-tls\") pod \"3e71bdc8-f806-4351-901b-374d437a5bc7\" (UID: \"3e71bdc8-f806-4351-901b-374d437a5bc7\") " Jul 28 17:40:53.586496 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.586272 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3e71bdc8-f806-4351-901b-374d437a5bc7-feast-data\") pod \"3e71bdc8-f806-4351-901b-374d437a5bc7\" (UID: \"3e71bdc8-f806-4351-901b-374d437a5bc7\") " Jul 28 17:40:53.586496 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.586308 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3e71bdc8-f806-4351-901b-374d437a5bc7-online-tls\") pod \"3e71bdc8-f806-4351-901b-374d437a5bc7\" (UID: \"3e71bdc8-f806-4351-901b-374d437a5bc7\") " Jul 28 17:40:53.586496 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.586339 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4czh\" (UniqueName: \"kubernetes.io/projected/3e71bdc8-f806-4351-901b-374d437a5bc7-kube-api-access-v4czh\") pod \"3e71bdc8-f806-4351-901b-374d437a5bc7\" (UID: \"3e71bdc8-f806-4351-901b-374d437a5bc7\") " Jul 28 17:40:53.586496 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.586357 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/3e71bdc8-f806-4351-901b-374d437a5bc7-registry-tls\") pod \"3e71bdc8-f806-4351-901b-374d437a5bc7\" (UID: \"3e71bdc8-f806-4351-901b-374d437a5bc7\") " Jul 28 17:40:53.586882 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.586839 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e71bdc8-f806-4351-901b-374d437a5bc7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "3e71bdc8-f806-4351-901b-374d437a5bc7" (UID: "3e71bdc8-f806-4351-901b-374d437a5bc7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:40:53.587024 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.586950 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e71bdc8-f806-4351-901b-374d437a5bc7-feast-data" (OuterVolumeSpecName: "feast-data") pod "3e71bdc8-f806-4351-901b-374d437a5bc7" (UID: "3e71bdc8-f806-4351-901b-374d437a5bc7"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:40:53.588674 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.588642 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e71bdc8-f806-4351-901b-374d437a5bc7-kube-api-access-v4czh" (OuterVolumeSpecName: "kube-api-access-v4czh") pod "3e71bdc8-f806-4351-901b-374d437a5bc7" (UID: "3e71bdc8-f806-4351-901b-374d437a5bc7"). InnerVolumeSpecName "kube-api-access-v4czh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:40:53.588792 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.588670 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e71bdc8-f806-4351-901b-374d437a5bc7-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "3e71bdc8-f806-4351-901b-374d437a5bc7" (UID: "3e71bdc8-f806-4351-901b-374d437a5bc7"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:40:53.588792 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.588688 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e71bdc8-f806-4351-901b-374d437a5bc7-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "3e71bdc8-f806-4351-901b-374d437a5bc7" (UID: "3e71bdc8-f806-4351-901b-374d437a5bc7"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:40:53.588792 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.588710 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e71bdc8-f806-4351-901b-374d437a5bc7-online-tls" (OuterVolumeSpecName: "online-tls") pod "3e71bdc8-f806-4351-901b-374d437a5bc7" (UID: "3e71bdc8-f806-4351-901b-374d437a5bc7"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:40:53.606368 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.606334 2571 generic.go:356] "Generic (PLEG): container finished" podID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerID="26309d3e4b28753dc4c35fad5cfdf605c512d7f443892e43f9a920c60deec18d" exitCode=137 Jul 28 17:40:53.606561 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.606395 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" Jul 28 17:40:53.606561 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.606413 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" event={"ID":"3e71bdc8-f806-4351-901b-374d437a5bc7","Type":"ContainerDied","Data":"26309d3e4b28753dc4c35fad5cfdf605c512d7f443892e43f9a920c60deec18d"} Jul 28 17:40:53.606561 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.606465 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg" event={"ID":"3e71bdc8-f806-4351-901b-374d437a5bc7","Type":"ContainerDied","Data":"0ad7b517ed34360a620996a3a21ff77f186420226bd94f0bdb5713c9979fabed"} Jul 28 17:40:53.606561 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.606484 2571 scope.go:117] "RemoveContainer" containerID="6f6567c0e39cda511bb97a362a81473f8d4cc6aecb90beb0005e13991f7d4bae" Jul 28 17:40:53.609910 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.609882 2571 generic.go:356] "Generic (PLEG): container finished" podID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerID="41000880a25121eaf0f309b3221e61c81f93d6107966dc80cd11517a3153a60d" exitCode=137 Jul 28 17:40:53.609910 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.609913 2571 generic.go:356] "Generic (PLEG): container finished" podID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerID="d167a06c9155f4caaf250fd9567c16ae580b032183dbadf279efa99267c762a6" exitCode=137 Jul 28 17:40:53.610077 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.609964 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" event={"ID":"8701697a-9097-4a29-bc2c-53e0db69e3a3","Type":"ContainerDied","Data":"41000880a25121eaf0f309b3221e61c81f93d6107966dc80cd11517a3153a60d"} Jul 28 17:40:53.610077 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.610004 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" event={"ID":"8701697a-9097-4a29-bc2c-53e0db69e3a3","Type":"ContainerDied","Data":"d167a06c9155f4caaf250fd9567c16ae580b032183dbadf279efa99267c762a6"} Jul 28 17:40:53.616133 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.616115 2571 scope.go:117] "RemoveContainer" containerID="26309d3e4b28753dc4c35fad5cfdf605c512d7f443892e43f9a920c60deec18d" Jul 28 17:40:53.624238 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.624216 2571 scope.go:117] "RemoveContainer" containerID="e17f95f469985c54e8badb4df5d4d84af557138d6d227a6687a087ec4c96af64" Jul 28 17:40:53.628773 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.628731 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg"] Jul 28 17:40:53.636183 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.633472 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-kb7hg"] Jul 28 17:40:53.636514 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.636489 2571 scope.go:117] "RemoveContainer" containerID="f86d98ac32855100132f3d101d686270e9189c43b68f56f663215c593c5487b7" Jul 28 17:40:53.644647 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.644604 2571 scope.go:117] "RemoveContainer" containerID="51deb8e464486423fa6c367ded3fbb6dcfe82b2762ace1c446a0c2296fbb9272" Jul 28 17:40:53.652114 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.652090 2571 scope.go:117] "RemoveContainer" containerID="6f6567c0e39cda511bb97a362a81473f8d4cc6aecb90beb0005e13991f7d4bae" Jul 28 17:40:53.652415 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:40:53.652393 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f6567c0e39cda511bb97a362a81473f8d4cc6aecb90beb0005e13991f7d4bae\": container with ID starting with 6f6567c0e39cda511bb97a362a81473f8d4cc6aecb90beb0005e13991f7d4bae not found: ID does not exist" containerID="6f6567c0e39cda511bb97a362a81473f8d4cc6aecb90beb0005e13991f7d4bae" Jul 28 17:40:53.652490 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.652426 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f6567c0e39cda511bb97a362a81473f8d4cc6aecb90beb0005e13991f7d4bae"} err="failed to get container status \"6f6567c0e39cda511bb97a362a81473f8d4cc6aecb90beb0005e13991f7d4bae\": rpc error: code = NotFound desc = could not find container \"6f6567c0e39cda511bb97a362a81473f8d4cc6aecb90beb0005e13991f7d4bae\": container with ID starting with 6f6567c0e39cda511bb97a362a81473f8d4cc6aecb90beb0005e13991f7d4bae not found: ID does not exist" Jul 28 17:40:53.652490 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.652446 2571 scope.go:117] "RemoveContainer" containerID="26309d3e4b28753dc4c35fad5cfdf605c512d7f443892e43f9a920c60deec18d" Jul 28 17:40:53.652670 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:40:53.652656 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26309d3e4b28753dc4c35fad5cfdf605c512d7f443892e43f9a920c60deec18d\": container with ID starting with 26309d3e4b28753dc4c35fad5cfdf605c512d7f443892e43f9a920c60deec18d not found: ID does not exist" containerID="26309d3e4b28753dc4c35fad5cfdf605c512d7f443892e43f9a920c60deec18d" Jul 28 17:40:53.652711 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.652673 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26309d3e4b28753dc4c35fad5cfdf605c512d7f443892e43f9a920c60deec18d"} err="failed to get container status \"26309d3e4b28753dc4c35fad5cfdf605c512d7f443892e43f9a920c60deec18d\": rpc error: code = NotFound desc = could not find container \"26309d3e4b28753dc4c35fad5cfdf605c512d7f443892e43f9a920c60deec18d\": container with ID starting with 26309d3e4b28753dc4c35fad5cfdf605c512d7f443892e43f9a920c60deec18d not found: ID does not exist" Jul 28 17:40:53.652711 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.652686 2571 scope.go:117] "RemoveContainer" containerID="e17f95f469985c54e8badb4df5d4d84af557138d6d227a6687a087ec4c96af64" Jul 28 17:40:53.652993 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:40:53.652974 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e17f95f469985c54e8badb4df5d4d84af557138d6d227a6687a087ec4c96af64\": container with ID starting with e17f95f469985c54e8badb4df5d4d84af557138d6d227a6687a087ec4c96af64 not found: ID does not exist" containerID="e17f95f469985c54e8badb4df5d4d84af557138d6d227a6687a087ec4c96af64" Jul 28 17:40:53.653055 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.652999 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e17f95f469985c54e8badb4df5d4d84af557138d6d227a6687a087ec4c96af64"} err="failed to get container status \"e17f95f469985c54e8badb4df5d4d84af557138d6d227a6687a087ec4c96af64\": rpc error: code = NotFound desc = could not find container \"e17f95f469985c54e8badb4df5d4d84af557138d6d227a6687a087ec4c96af64\": container with ID starting with e17f95f469985c54e8badb4df5d4d84af557138d6d227a6687a087ec4c96af64 not found: ID does not exist" Jul 28 17:40:53.653055 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.653014 2571 scope.go:117] "RemoveContainer" containerID="f86d98ac32855100132f3d101d686270e9189c43b68f56f663215c593c5487b7" Jul 28 17:40:53.653260 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:40:53.653242 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f86d98ac32855100132f3d101d686270e9189c43b68f56f663215c593c5487b7\": container with ID starting with f86d98ac32855100132f3d101d686270e9189c43b68f56f663215c593c5487b7 not found: ID does not exist" containerID="f86d98ac32855100132f3d101d686270e9189c43b68f56f663215c593c5487b7" Jul 28 17:40:53.653304 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.653266 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f86d98ac32855100132f3d101d686270e9189c43b68f56f663215c593c5487b7"} err="failed to get container status \"f86d98ac32855100132f3d101d686270e9189c43b68f56f663215c593c5487b7\": rpc error: code = NotFound desc = could not find container \"f86d98ac32855100132f3d101d686270e9189c43b68f56f663215c593c5487b7\": container with ID starting with f86d98ac32855100132f3d101d686270e9189c43b68f56f663215c593c5487b7 not found: ID does not exist" Jul 28 17:40:53.653304 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.653282 2571 scope.go:117] "RemoveContainer" containerID="51deb8e464486423fa6c367ded3fbb6dcfe82b2762ace1c446a0c2296fbb9272" Jul 28 17:40:53.653508 ip-10-0-141-34 kubenswrapper[2571]: E0728 17:40:53.653491 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51deb8e464486423fa6c367ded3fbb6dcfe82b2762ace1c446a0c2296fbb9272\": container with ID starting with 51deb8e464486423fa6c367ded3fbb6dcfe82b2762ace1c446a0c2296fbb9272 not found: ID does not exist" containerID="51deb8e464486423fa6c367ded3fbb6dcfe82b2762ace1c446a0c2296fbb9272" Jul 28 17:40:53.653549 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.653514 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51deb8e464486423fa6c367ded3fbb6dcfe82b2762ace1c446a0c2296fbb9272"} err="failed to get container status \"51deb8e464486423fa6c367ded3fbb6dcfe82b2762ace1c446a0c2296fbb9272\": rpc error: code = NotFound desc = could not find container \"51deb8e464486423fa6c367ded3fbb6dcfe82b2762ace1c446a0c2296fbb9272\": container with ID starting with 51deb8e464486423fa6c367ded3fbb6dcfe82b2762ace1c446a0c2296fbb9272 not found: ID does not exist" Jul 28 17:40:53.687221 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.687182 2571 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3e71bdc8-f806-4351-901b-374d437a5bc7-offline-tls\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:40:53.687221 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.687211 2571 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3e71bdc8-f806-4351-901b-374d437a5bc7-feast-data\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:40:53.687440 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.687242 2571 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3e71bdc8-f806-4351-901b-374d437a5bc7-online-tls\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:40:53.687440 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.687252 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4czh\" (UniqueName: \"kubernetes.io/projected/3e71bdc8-f806-4351-901b-374d437a5bc7-kube-api-access-v4czh\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:40:53.687440 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.687263 2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/3e71bdc8-f806-4351-901b-374d437a5bc7-registry-tls\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:40:53.687440 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.687273 2571 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3e71bdc8-f806-4351-901b-374d437a5bc7-ui-tls\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:40:53.784000 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:53.783968 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" path="/var/lib/kubelet/pods/3e71bdc8-f806-4351-901b-374d437a5bc7/volumes" Jul 28 17:40:54.069549 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.069522 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:40:54.090201 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.090166 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-online-tls\") pod \"8701697a-9097-4a29-bc2c-53e0db69e3a3\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " Jul 28 17:40:54.090306 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.090223 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-offline-tls\") pod \"8701697a-9097-4a29-bc2c-53e0db69e3a3\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " Jul 28 17:40:54.090306 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.090256 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8701697a-9097-4a29-bc2c-53e0db69e3a3-feast-data\") pod \"8701697a-9097-4a29-bc2c-53e0db69e3a3\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " Jul 28 17:40:54.090306 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.090287 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-ui-tls\") pod \"8701697a-9097-4a29-bc2c-53e0db69e3a3\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " Jul 28 17:40:54.090509 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.090329 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-747ld\" (UniqueName: \"kubernetes.io/projected/8701697a-9097-4a29-bc2c-53e0db69e3a3-kube-api-access-747ld\") pod \"8701697a-9097-4a29-bc2c-53e0db69e3a3\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " Jul 28 17:40:54.090509 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.090367 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-registry-tls\") pod \"8701697a-9097-4a29-bc2c-53e0db69e3a3\" (UID: \"8701697a-9097-4a29-bc2c-53e0db69e3a3\") " Jul 28 17:40:54.091134 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.090999 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8701697a-9097-4a29-bc2c-53e0db69e3a3-feast-data" (OuterVolumeSpecName: "feast-data") pod "8701697a-9097-4a29-bc2c-53e0db69e3a3" (UID: "8701697a-9097-4a29-bc2c-53e0db69e3a3"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:40:54.093153 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.093108 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "8701697a-9097-4a29-bc2c-53e0db69e3a3" (UID: "8701697a-9097-4a29-bc2c-53e0db69e3a3"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:40:54.093275 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.093243 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "8701697a-9097-4a29-bc2c-53e0db69e3a3" (UID: "8701697a-9097-4a29-bc2c-53e0db69e3a3"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:40:54.093427 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.093405 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-online-tls" (OuterVolumeSpecName: "online-tls") pod "8701697a-9097-4a29-bc2c-53e0db69e3a3" (UID: "8701697a-9097-4a29-bc2c-53e0db69e3a3"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:40:54.093944 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.093913 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8701697a-9097-4a29-bc2c-53e0db69e3a3-kube-api-access-747ld" (OuterVolumeSpecName: "kube-api-access-747ld") pod "8701697a-9097-4a29-bc2c-53e0db69e3a3" (UID: "8701697a-9097-4a29-bc2c-53e0db69e3a3"). InnerVolumeSpecName "kube-api-access-747ld". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:40:54.093944 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.093924 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8701697a-9097-4a29-bc2c-53e0db69e3a3" (UID: "8701697a-9097-4a29-bc2c-53e0db69e3a3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:40:54.190996 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.190953 2571 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-online-tls\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:40:54.190996 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.190987 2571 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-offline-tls\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:40:54.190996 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.190998 2571 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8701697a-9097-4a29-bc2c-53e0db69e3a3-feast-data\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:40:54.190996 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.191010 2571 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-ui-tls\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:40:54.191265 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.191022 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-747ld\" (UniqueName: \"kubernetes.io/projected/8701697a-9097-4a29-bc2c-53e0db69e3a3-kube-api-access-747ld\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:40:54.191265 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.191033 2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8701697a-9097-4a29-bc2c-53e0db69e3a3-registry-tls\") on node \"ip-10-0-141-34.ec2.internal\" DevicePath \"\"" Jul 28 17:40:54.617352 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.617318 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" event={"ID":"8701697a-9097-4a29-bc2c-53e0db69e3a3","Type":"ContainerDied","Data":"79d3d62655756a4250fb130864aca500c7513823c1ac6c7d163fb790bb0451fa"} Jul 28 17:40:54.617827 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.617367 2571 scope.go:117] "RemoveContainer" containerID="05dfc9285e85493e8d536f525229391adbb1c099c461271590a2b6ccba3f58e7" Jul 28 17:40:54.617827 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.617367 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln" Jul 28 17:40:54.625709 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.625690 2571 scope.go:117] "RemoveContainer" containerID="41000880a25121eaf0f309b3221e61c81f93d6107966dc80cd11517a3153a60d" Jul 28 17:40:54.633423 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.633401 2571 scope.go:117] "RemoveContainer" containerID="6b22f0f4d7af1185fe54b87e32ecdfb8a4cee851b8decb08b836809923e7adc9" Jul 28 17:40:54.639236 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.638513 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln"] Jul 28 17:40:54.641655 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.641634 2571 scope.go:117] "RemoveContainer" containerID="d167a06c9155f4caaf250fd9567c16ae580b032183dbadf279efa99267c762a6" Jul 28 17:40:54.642618 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.642594 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-2jsln"] Jul 28 17:40:54.649506 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.649482 2571 scope.go:117] "RemoveContainer" containerID="d05fdbab25b0f0b97f500cb4ef940f2e8040c4617462a0e2d222104cb1de412f" Jul 28 17:40:54.656661 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:54.656641 2571 scope.go:117] "RemoveContainer" containerID="009352ad777b74e50b34d4f11fa6567bc54a204a05be069175eeeb711605fd3f" Jul 28 17:40:55.784474 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:40:55.784438 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" path="/var/lib/kubelet/pods/8701697a-9097-4a29-bc2c-53e0db69e3a3/volumes" Jul 28 17:41:30.853559 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853518 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gvbm4/must-gather-b77vf"] Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853781 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="feast-apply" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853796 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="feast-apply" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853806 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="online" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853813 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="online" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853820 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="ui" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853826 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="ui" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853834 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="feast-init" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853839 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="feast-init" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853845 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="ui" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853850 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="ui" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853857 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="offline" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853862 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="offline" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853868 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="registry" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853873 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="registry" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853879 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="feast-apply" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853884 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="feast-apply" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853893 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="offline" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853898 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="offline" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853906 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="online" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853911 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="online" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853919 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="feast-init" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853925 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="feast-init" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853968 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="online" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853974 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="offline" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853981 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="ui" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853987 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="online" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853994 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="registry" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.853998 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8701697a-9097-4a29-bc2c-53e0db69e3a3" containerName="ui" Jul 28 17:41:30.854074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.854003 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e71bdc8-f806-4351-901b-374d437a5bc7" containerName="offline" Jul 28 17:41:30.856705 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.856687 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gvbm4/must-gather-b77vf" Jul 28 17:41:30.858988 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.858961 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gvbm4\"/\"default-dockercfg-ptdtn\"" Jul 28 17:41:30.859134 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.858963 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gvbm4\"/\"openshift-service-ca.crt\"" Jul 28 17:41:30.859522 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.859502 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gvbm4\"/\"kube-root-ca.crt\"" Jul 28 17:41:30.866021 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.865991 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gvbm4/must-gather-b77vf"] Jul 28 17:41:30.956568 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.956520 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0ff07cd6-9806-4f35-9089-ea545bb646bd-must-gather-output\") pod \"must-gather-b77vf\" (UID: \"0ff07cd6-9806-4f35-9089-ea545bb646bd\") " pod="openshift-must-gather-gvbm4/must-gather-b77vf" Jul 28 17:41:30.956749 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:30.956641 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcr2m\" (UniqueName: \"kubernetes.io/projected/0ff07cd6-9806-4f35-9089-ea545bb646bd-kube-api-access-hcr2m\") pod \"must-gather-b77vf\" (UID: \"0ff07cd6-9806-4f35-9089-ea545bb646bd\") " pod="openshift-must-gather-gvbm4/must-gather-b77vf" Jul 28 17:41:31.057218 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:31.057176 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0ff07cd6-9806-4f35-9089-ea545bb646bd-must-gather-output\") pod \"must-gather-b77vf\" (UID: \"0ff07cd6-9806-4f35-9089-ea545bb646bd\") " pod="openshift-must-gather-gvbm4/must-gather-b77vf" Jul 28 17:41:31.057350 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:31.057241 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcr2m\" (UniqueName: \"kubernetes.io/projected/0ff07cd6-9806-4f35-9089-ea545bb646bd-kube-api-access-hcr2m\") pod \"must-gather-b77vf\" (UID: \"0ff07cd6-9806-4f35-9089-ea545bb646bd\") " pod="openshift-must-gather-gvbm4/must-gather-b77vf" Jul 28 17:41:31.057556 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:31.057535 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0ff07cd6-9806-4f35-9089-ea545bb646bd-must-gather-output\") pod \"must-gather-b77vf\" (UID: \"0ff07cd6-9806-4f35-9089-ea545bb646bd\") " pod="openshift-must-gather-gvbm4/must-gather-b77vf" Jul 28 17:41:31.065944 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:31.065910 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcr2m\" (UniqueName: \"kubernetes.io/projected/0ff07cd6-9806-4f35-9089-ea545bb646bd-kube-api-access-hcr2m\") pod \"must-gather-b77vf\" (UID: \"0ff07cd6-9806-4f35-9089-ea545bb646bd\") " pod="openshift-must-gather-gvbm4/must-gather-b77vf" Jul 28 17:41:31.166713 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:31.166674 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gvbm4/must-gather-b77vf" Jul 28 17:41:31.301373 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:31.301337 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gvbm4/must-gather-b77vf"] Jul 28 17:41:31.306254 ip-10-0-141-34 kubenswrapper[2571]: W0728 17:41:31.306214 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ff07cd6_9806_4f35_9089_ea545bb646bd.slice/crio-2871a059df93c2dee772f2a4302aac68a9e18760cb2c5697395db453a4ad8ff8 WatchSource:0}: Error finding container 2871a059df93c2dee772f2a4302aac68a9e18760cb2c5697395db453a4ad8ff8: Status 404 returned error can't find the container with id 2871a059df93c2dee772f2a4302aac68a9e18760cb2c5697395db453a4ad8ff8 Jul 28 17:41:31.308101 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:31.308085 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 17:41:31.740607 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:31.740567 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gvbm4/must-gather-b77vf" event={"ID":"0ff07cd6-9806-4f35-9089-ea545bb646bd","Type":"ContainerStarted","Data":"2871a059df93c2dee772f2a4302aac68a9e18760cb2c5697395db453a4ad8ff8"} Jul 28 17:41:32.748104 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:32.748066 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gvbm4/must-gather-b77vf" event={"ID":"0ff07cd6-9806-4f35-9089-ea545bb646bd","Type":"ContainerStarted","Data":"ef1d71e5304988788a51ad6afb7ca62dd32c0a71f24ac82fcdb05a441ae52800"} Jul 28 17:41:32.748556 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:32.748281 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gvbm4/must-gather-b77vf" event={"ID":"0ff07cd6-9806-4f35-9089-ea545bb646bd","Type":"ContainerStarted","Data":"904f443936cb45dc61eee77d270700284850a1687584ed9146c72f729e3def3f"} Jul 28 17:41:32.764561 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:32.764504 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gvbm4/must-gather-b77vf" podStartSLOduration=2.123236507 podStartE2EDuration="2.764487312s" totalImagesPullingTime="641.250805ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:41:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:41:32.763515652 +0000 UTC m=+1953.483343363" watchObservedRunningTime="2026-07-28 17:41:32.764487312 +0000 UTC m=+1953.484315022" Jul 28 17:41:33.474941 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:33.474908 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-w95vd_0af3fecb-e79a-4b70-ba3a-158278da3aaa/global-pull-secret-syncer/0.log" Jul 28 17:41:33.611792 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:33.611701 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nmkgg_0cbdabd3-4530-4336-b84e-4b1bb7f75650/konnectivity-agent/0.log" Jul 28 17:41:33.690790 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:33.690747 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-34.ec2.internal_220364fcdea4bfcb02dba704a736fdd7/haproxy/0.log" Jul 28 17:41:37.544208 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:37.544170 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dksmk_e14a8715-b4ff-48e3-82f6-b725bacaa253/node-exporter/0.log" Jul 28 17:41:37.574331 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:37.574290 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dksmk_e14a8715-b4ff-48e3-82f6-b725bacaa253/kube-rbac-proxy/0.log" Jul 28 17:41:37.610096 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:37.610062 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dksmk_e14a8715-b4ff-48e3-82f6-b725bacaa253/init-textfile/0.log" Jul 28 17:41:37.738041 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:37.738001 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-45v8q_be9b4bb7-3adb-43b4-9eeb-a82ac76481ba/kube-rbac-proxy-main/0.log" Jul 28 17:41:37.771634 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:37.771597 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-45v8q_be9b4bb7-3adb-43b4-9eeb-a82ac76481ba/kube-rbac-proxy-self/0.log" Jul 28 17:41:37.803238 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:37.803086 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-45v8q_be9b4bb7-3adb-43b4-9eeb-a82ac76481ba/openshift-state-metrics/0.log" Jul 28 17:41:40.632039 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:40.632002 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6b75845fc7-h87lt_ce8b2dc0-925e-470f-816a-cea0de9df4ec/console/0.log" Jul 28 17:41:40.666328 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:40.666296 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-4q56h_714fbd64-f864-4c39-8e79-1ab21e7c1121/download-server/0.log" Jul 28 17:41:41.165087 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:41.165046 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv"] Jul 28 17:41:41.171012 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:41.170976 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv" Jul 28 17:41:41.177456 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:41.177419 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv"] Jul 28 17:41:41.243453 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:41.243404 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/406504d5-98ce-4d16-93d2-11031dce9297-proc\") pod \"perf-node-gather-daemonset-9vwbv\" (UID: \"406504d5-98ce-4d16-93d2-11031dce9297\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv" Jul 28 17:41:41.243657 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:41.243462 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr2xx\" (UniqueName: \"kubernetes.io/projected/406504d5-98ce-4d16-93d2-11031dce9297-kube-api-access-xr2xx\") pod \"perf-node-gather-daemonset-9vwbv\" (UID: \"406504d5-98ce-4d16-93d2-11031dce9297\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv" Jul 28 17:41:41.243657 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:41.243527 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/406504d5-98ce-4d16-93d2-11031dce9297-sys\") pod \"perf-node-gather-daemonset-9vwbv\" (UID: \"406504d5-98ce-4d16-93d2-11031dce9297\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv" Jul 28 17:41:41.243657 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:41.243557 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/406504d5-98ce-4d16-93d2-11031dce9297-lib-modules\") pod \"perf-node-gather-daemonset-9vwbv\" (UID: \"406504d5-98ce-4d16-93d2-11031dce9297\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv" Jul 28 17:41:41.243657 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:41.243602 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/406504d5-98ce-4d16-93d2-11031dce9297-podres\") pod \"perf-node-gather-daemonset-9vwbv\" (UID: \"406504d5-98ce-4d16-93d2-11031dce9297\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv" Jul 28 17:41:41.344625 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:41.344591 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/406504d5-98ce-4d16-93d2-11031dce9297-proc\") pod \"perf-node-gather-daemonset-9vwbv\" (UID: \"406504d5-98ce-4d16-93d2-11031dce9297\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv" Jul 28 17:41:41.344847 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:41.344633 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr2xx\" (UniqueName: \"kubernetes.io/projected/406504d5-98ce-4d16-93d2-11031dce9297-kube-api-access-xr2xx\") pod \"perf-node-gather-daemonset-9vwbv\" (UID: \"406504d5-98ce-4d16-93d2-11031dce9297\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv" Jul 28 17:41:41.344847 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:41.344690 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/406504d5-98ce-4d16-93d2-11031dce9297-sys\") pod \"perf-node-gather-daemonset-9vwbv\" (UID: \"406504d5-98ce-4d16-93d2-11031dce9297\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv" Jul 28 17:41:41.344847 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:41.344717 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/406504d5-98ce-4d16-93d2-11031dce9297-lib-modules\") pod \"perf-node-gather-daemonset-9vwbv\" (UID: \"406504d5-98ce-4d16-93d2-11031dce9297\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv" Jul 28 17:41:41.344847 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:41.344719 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/406504d5-98ce-4d16-93d2-11031dce9297-proc\") pod \"perf-node-gather-daemonset-9vwbv\" (UID: \"406504d5-98ce-4d16-93d2-11031dce9297\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv" Jul 28 17:41:41.344847 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:41.344812 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/406504d5-98ce-4d16-93d2-11031dce9297-sys\") pod \"perf-node-gather-daemonset-9vwbv\" (UID: \"406504d5-98ce-4d16-93d2-11031dce9297\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv" Jul 28 17:41:41.344847 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:41.344818 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/406504d5-98ce-4d16-93d2-11031dce9297-podres\") pod \"perf-node-gather-daemonset-9vwbv\" (UID: \"406504d5-98ce-4d16-93d2-11031dce9297\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv" Jul 28 17:41:41.345074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:41.344884 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/406504d5-98ce-4d16-93d2-11031dce9297-podres\") pod \"perf-node-gather-daemonset-9vwbv\" (UID: \"406504d5-98ce-4d16-93d2-11031dce9297\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv" Jul 28 17:41:41.345074 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:41.344887 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/406504d5-98ce-4d16-93d2-11031dce9297-lib-modules\") pod \"perf-node-gather-daemonset-9vwbv\" (UID: \"406504d5-98ce-4d16-93d2-11031dce9297\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv" Jul 28 17:41:41.352353 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:41.352315 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr2xx\" (UniqueName: \"kubernetes.io/projected/406504d5-98ce-4d16-93d2-11031dce9297-kube-api-access-xr2xx\") pod \"perf-node-gather-daemonset-9vwbv\" (UID: \"406504d5-98ce-4d16-93d2-11031dce9297\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv" Jul 28 17:41:41.484237 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:41.484131 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv" Jul 28 17:41:41.650977 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:41.650942 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv"] Jul 28 17:41:41.795852 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:41.795373 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv" event={"ID":"406504d5-98ce-4d16-93d2-11031dce9297","Type":"ContainerStarted","Data":"78d2d78c6758f1fc9f82f9822b4f64c8bbd64ec1ee401fc3ebb2d374c6f617e2"} Jul 28 17:41:41.795852 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:41.795443 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv" event={"ID":"406504d5-98ce-4d16-93d2-11031dce9297","Type":"ContainerStarted","Data":"f88970124b2860664e9dd5a8e853102ec7aecbe94afa59c0e69c87750471b5bf"} Jul 28 17:41:41.796072 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:41.795903 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv" Jul 28 17:41:41.818311 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:41.814232 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv" podStartSLOduration=0.814211088 podStartE2EDuration="814.211088ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:41:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:41:41.811628547 +0000 UTC m=+1962.531456260" watchObservedRunningTime="2026-07-28 17:41:41.814211088 +0000 UTC m=+1962.534038800" Jul 28 17:41:42.214944 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:42.214908 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qjmfr_51754089-3d17-471f-85f1-5b493bed2336/dns/0.log" Jul 28 17:41:42.240602 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:42.240574 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qjmfr_51754089-3d17-471f-85f1-5b493bed2336/kube-rbac-proxy/0.log" Jul 28 17:41:42.268998 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:42.268958 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6vlbb_7fc05f85-4048-4422-8800-59ca1e11e254/dns-node-resolver/0.log" Jul 28 17:41:42.773969 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:42.773935 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dthjv_507a907d-af5f-4815-93c4-0efe8cc04000/node-ca/0.log" Jul 28 17:41:43.905894 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:43.905867 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4nnr6_91c276cf-bc64-466b-a82d-3cdbf4ccc495/serve-healthcheck-canary/0.log" Jul 28 17:41:44.574476 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:44.574445 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ps8hz_b9cd99dc-8b97-4a3e-8eba-9d55efda9909/kube-rbac-proxy/0.log" Jul 28 17:41:44.599637 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:44.599602 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ps8hz_b9cd99dc-8b97-4a3e-8eba-9d55efda9909/exporter/0.log" Jul 28 17:41:44.626684 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:44.626656 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ps8hz_b9cd99dc-8b97-4a3e-8eba-9d55efda9909/extractor/0.log" Jul 28 17:41:47.810777 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:47.810724 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-9vwbv" Jul 28 17:41:51.093140 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:51.093042 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v9cpp_52ae3ebe-36b6-44ca-94aa-ee34533ca203/kube-multus-additional-cni-plugins/0.log" Jul 28 17:41:51.117478 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:51.117447 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v9cpp_52ae3ebe-36b6-44ca-94aa-ee34533ca203/egress-router-binary-copy/0.log" Jul 28 17:41:51.142384 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:51.142351 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v9cpp_52ae3ebe-36b6-44ca-94aa-ee34533ca203/cni-plugins/0.log" Jul 28 17:41:51.167632 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:51.167605 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v9cpp_52ae3ebe-36b6-44ca-94aa-ee34533ca203/bond-cni-plugin/0.log" Jul 28 17:41:51.192024 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:51.191994 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v9cpp_52ae3ebe-36b6-44ca-94aa-ee34533ca203/routeoverride-cni/0.log" Jul 28 17:41:51.217983 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:51.217950 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v9cpp_52ae3ebe-36b6-44ca-94aa-ee34533ca203/whereabouts-cni-bincopy/0.log" Jul 28 17:41:51.243361 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:51.243329 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v9cpp_52ae3ebe-36b6-44ca-94aa-ee34533ca203/whereabouts-cni/0.log" Jul 28 17:41:51.305365 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:51.305338 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-xwqnd_50a88180-0620-4676-ac16-95806564a133/kube-multus/0.log" Jul 28 17:41:51.458391 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:51.458359 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-w2qls_c0d05255-0573-4806-84b3-c03ff36cea82/network-metrics-daemon/0.log" Jul 28 17:41:51.481872 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:51.481843 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-w2qls_c0d05255-0573-4806-84b3-c03ff36cea82/kube-rbac-proxy/0.log" Jul 28 17:41:52.302440 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:52.302292 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlxqx_bfc273ea-75a3-41b1-9158-6a8e00aeaa2e/ovn-controller/0.log" Jul 28 17:41:52.330160 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:52.330127 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlxqx_bfc273ea-75a3-41b1-9158-6a8e00aeaa2e/ovn-acl-logging/0.log" Jul 28 17:41:52.365648 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:52.365614 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlxqx_bfc273ea-75a3-41b1-9158-6a8e00aeaa2e/kube-rbac-proxy-node/0.log" Jul 28 17:41:52.394730 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:52.394702 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlxqx_bfc273ea-75a3-41b1-9158-6a8e00aeaa2e/kube-rbac-proxy-ovn-metrics/0.log" Jul 28 17:41:52.426148 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:52.426116 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlxqx_bfc273ea-75a3-41b1-9158-6a8e00aeaa2e/northd/0.log" Jul 28 17:41:52.451464 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:52.451435 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlxqx_bfc273ea-75a3-41b1-9158-6a8e00aeaa2e/nbdb/0.log" Jul 28 17:41:52.476472 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:52.476447 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlxqx_bfc273ea-75a3-41b1-9158-6a8e00aeaa2e/sbdb/0.log" Jul 28 17:41:52.522098 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:52.522059 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dlxqx_bfc273ea-75a3-41b1-9158-6a8e00aeaa2e/ovnkube-controller/0.log" Jul 28 17:41:54.013726 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:54.013698 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-6kkbm_ab2a66ee-852a-4d34-88fc-1000fef6b92c/network-check-target-container/0.log" Jul 28 17:41:55.002406 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:55.002376 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-knlrx_8f0df316-8fe9-4dcf-ab0c-c502dd565001/iptables-alerter/0.log" Jul 28 17:41:55.747915 ip-10-0-141-34 kubenswrapper[2571]: I0728 17:41:55.747888 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-qwhwb_a47e2ee1-1e91-4768-a10d-6b5a8fb46dd3/tuned/0.log"