Jun 11 21:15:41.461896 ip-10-0-143-6 systemd[1]: Starting Kubernetes Kubelet... Jun 11 21:15:41.894683 ip-10-0-143-6 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 21:15:41.894683 ip-10-0-143-6 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 11 21:15:41.894683 ip-10-0-143-6 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 21:15:41.894683 ip-10-0-143-6 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 11 21:15:41.894683 ip-10-0-143-6 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 21:15:41.895881 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.895527 2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 11 21:15:41.901403 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901385 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 21:15:41.901403 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901402 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 21:15:41.901469 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901406 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 21:15:41.901469 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901410 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 21:15:41.901469 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901413 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 21:15:41.901469 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901416 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 21:15:41.901469 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901419 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 21:15:41.901469 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901423 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 21:15:41.901469 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901425 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 21:15:41.901469 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901428 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 21:15:41.901469 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901431 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 21:15:41.901469 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901434 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 21:15:41.901469 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901437 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 21:15:41.901469 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901439 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 21:15:41.901469 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901442 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 21:15:41.901469 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901446 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 21:15:41.901469 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901449 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 21:15:41.901469 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901453 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 21:15:41.901469 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901458 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 21:15:41.901469 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901461 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 21:15:41.901469 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901470 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 21:15:41.901469 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901473 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 21:15:41.901963 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901476 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 21:15:41.901963 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901478 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 21:15:41.901963 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901481 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 21:15:41.901963 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901484 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 21:15:41.901963 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901489 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 21:15:41.901963 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901492 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 21:15:41.901963 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901495 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 21:15:41.901963 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901498 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 21:15:41.901963 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901500 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 21:15:41.901963 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901503 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 21:15:41.901963 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901506 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 21:15:41.901963 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901508 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 21:15:41.901963 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901511 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 21:15:41.901963 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901514 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 21:15:41.901963 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901516 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 21:15:41.901963 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901519 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 21:15:41.901963 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901521 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 21:15:41.901963 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901524 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 21:15:41.901963 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901527 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 21:15:41.902425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901529 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 21:15:41.902425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901532 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 21:15:41.902425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901534 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 21:15:41.902425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901537 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 21:15:41.902425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901541 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 21:15:41.902425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901544 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 21:15:41.902425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901547 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 21:15:41.902425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901551 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 21:15:41.902425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901555 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 21:15:41.902425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901558 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 21:15:41.902425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901572 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 21:15:41.902425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901575 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 21:15:41.902425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901579 2570 feature_gate.go:328] unrecognized feature gate: Example Jun 11 21:15:41.902425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901583 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 21:15:41.902425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901587 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 21:15:41.902425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901590 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 21:15:41.902425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901594 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 21:15:41.902425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901596 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 21:15:41.902425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901599 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 21:15:41.902907 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901602 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 21:15:41.902907 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901604 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 21:15:41.902907 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901607 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 21:15:41.902907 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901609 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 21:15:41.902907 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901612 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 21:15:41.902907 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901614 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 21:15:41.902907 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901617 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 21:15:41.902907 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901621 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 21:15:41.902907 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901624 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 21:15:41.902907 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901626 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 21:15:41.902907 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901629 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 21:15:41.902907 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901632 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 21:15:41.902907 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901634 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 21:15:41.902907 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901637 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 21:15:41.902907 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901639 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 21:15:41.902907 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901642 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 21:15:41.902907 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901644 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 21:15:41.902907 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901647 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 21:15:41.902907 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901650 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 21:15:41.902907 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901652 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 21:15:41.903400 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901655 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 21:15:41.903400 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901657 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 21:15:41.903400 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901660 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 21:15:41.903400 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901663 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 21:15:41.903400 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901665 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 21:15:41.903400 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901668 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 21:15:41.903400 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901674 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 21:15:41.903400 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901678 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 21:15:41.903400 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901681 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 21:15:41.903400 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901684 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 21:15:41.903400 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901687 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 21:15:41.903400 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901689 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 21:15:41.903400 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901692 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 21:15:41.903400 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901695 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 21:15:41.903400 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901698 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 21:15:41.903400 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901700 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 21:15:41.903400 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901703 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 21:15:41.903400 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901705 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 21:15:41.903400 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.901708 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 21:15:41.903882 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902445 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 21:15:41.903882 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902452 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 21:15:41.903882 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902455 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 21:15:41.903882 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902457 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 21:15:41.903882 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902460 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 21:15:41.903882 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902463 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 21:15:41.903882 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902465 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 21:15:41.903882 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902468 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 21:15:41.903882 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902471 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 21:15:41.903882 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902474 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 21:15:41.903882 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902476 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 21:15:41.903882 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902480 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 21:15:41.903882 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902482 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 21:15:41.903882 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902485 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 21:15:41.903882 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902487 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 21:15:41.903882 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902490 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 21:15:41.903882 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902493 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 21:15:41.903882 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902495 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 21:15:41.903882 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902498 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 21:15:41.903882 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902500 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 21:15:41.904373 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902505 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 21:15:41.904373 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902508 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 21:15:41.904373 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902511 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 21:15:41.904373 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902514 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 21:15:41.904373 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902517 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 21:15:41.904373 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902519 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 21:15:41.904373 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902522 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 21:15:41.904373 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902524 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 21:15:41.904373 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902527 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 21:15:41.904373 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902529 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 21:15:41.904373 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902532 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 21:15:41.904373 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902535 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 21:15:41.904373 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902537 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 21:15:41.904373 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902539 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 21:15:41.904373 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902542 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 21:15:41.904373 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902544 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 21:15:41.904373 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902547 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 21:15:41.904373 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902551 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 21:15:41.904373 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902554 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 21:15:41.905386 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902557 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 21:15:41.905386 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902574 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 21:15:41.905386 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902578 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 21:15:41.905386 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902581 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 21:15:41.905386 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902584 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 21:15:41.905386 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902586 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 21:15:41.905386 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902590 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 21:15:41.905386 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902593 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 21:15:41.905386 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902595 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 21:15:41.905386 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902598 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 21:15:41.905386 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902601 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 21:15:41.905386 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902604 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 21:15:41.905386 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902608 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 21:15:41.905386 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902612 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 21:15:41.905386 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902616 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 21:15:41.905386 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902619 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 21:15:41.905386 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902622 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 21:15:41.905386 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902625 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 21:15:41.905386 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902628 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 21:15:41.905386 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902632 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 21:15:41.905971 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902634 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 21:15:41.905971 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902637 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 21:15:41.905971 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902640 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 21:15:41.905971 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902642 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 21:15:41.905971 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902645 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 21:15:41.905971 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902648 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 21:15:41.905971 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902650 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 21:15:41.905971 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902653 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 21:15:41.905971 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902655 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 21:15:41.905971 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902658 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 21:15:41.905971 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902660 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 21:15:41.905971 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902663 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 21:15:41.905971 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902665 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 21:15:41.905971 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902668 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 21:15:41.905971 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902670 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 21:15:41.905971 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902673 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 21:15:41.905971 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902675 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 21:15:41.905971 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902678 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 21:15:41.905971 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902681 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 21:15:41.905971 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902683 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 21:15:41.906458 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902686 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 21:15:41.906458 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902689 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 21:15:41.906458 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902691 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 21:15:41.906458 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902694 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 21:15:41.906458 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902698 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 21:15:41.906458 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902701 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 21:15:41.906458 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902704 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 21:15:41.906458 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902707 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 21:15:41.906458 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902710 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 21:15:41.906458 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902713 2570 feature_gate.go:328] unrecognized feature gate: Example Jun 11 21:15:41.906458 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902717 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 21:15:41.906458 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902719 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 21:15:41.906458 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902722 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 21:15:41.906458 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902725 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 21:15:41.906458 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902728 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 21:15:41.906458 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902730 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 21:15:41.906458 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902732 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 21:15:41.906458 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902735 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 21:15:41.906458 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902737 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 21:15:41.906945 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.902740 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 21:15:41.906945 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903126 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 21:15:41.906945 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903131 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 21:15:41.906945 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903134 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 21:15:41.906945 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903137 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 21:15:41.906945 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903140 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 21:15:41.906945 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903143 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 21:15:41.906945 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903146 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 21:15:41.906945 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903149 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 21:15:41.906945 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903152 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 21:15:41.906945 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903155 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 21:15:41.906945 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903158 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 21:15:41.906945 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903161 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 21:15:41.906945 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903164 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 21:15:41.906945 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903166 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 21:15:41.906945 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903171 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 21:15:41.906945 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903174 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 21:15:41.906945 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903177 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 21:15:41.906945 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903180 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 21:15:41.907425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903184 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 21:15:41.907425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903187 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 21:15:41.907425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903191 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 21:15:41.907425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903194 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 21:15:41.907425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903196 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 21:15:41.907425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903199 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 21:15:41.907425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903202 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 21:15:41.907425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903204 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 21:15:41.907425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903207 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 21:15:41.907425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903209 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 21:15:41.907425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903212 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 21:15:41.907425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903214 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 21:15:41.907425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903217 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 21:15:41.907425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903219 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 21:15:41.907425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903222 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 21:15:41.907425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903225 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 21:15:41.907425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903227 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 21:15:41.907425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903230 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 21:15:41.907425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903232 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 21:15:41.907425 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903235 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 21:15:41.907922 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903237 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 21:15:41.907922 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903240 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 21:15:41.907922 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903242 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 21:15:41.907922 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903245 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 21:15:41.907922 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903248 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 21:15:41.907922 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903251 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 21:15:41.907922 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903254 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 21:15:41.907922 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903257 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 21:15:41.907922 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903259 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 21:15:41.907922 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903262 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 21:15:41.907922 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903266 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 21:15:41.907922 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903269 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 21:15:41.907922 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903272 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 21:15:41.907922 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903275 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 21:15:41.907922 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903279 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 21:15:41.907922 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903282 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 21:15:41.907922 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903285 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 21:15:41.907922 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903288 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 21:15:41.907922 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903290 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 21:15:41.908374 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903293 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 21:15:41.908374 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903295 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 21:15:41.908374 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903298 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 21:15:41.908374 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903300 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 21:15:41.908374 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903303 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 21:15:41.908374 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903305 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 21:15:41.908374 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903308 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 21:15:41.908374 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903313 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 21:15:41.908374 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903315 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 21:15:41.908374 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903318 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 21:15:41.908374 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903320 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 21:15:41.908374 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903323 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 21:15:41.908374 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903326 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 21:15:41.908374 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903328 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 21:15:41.908374 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903332 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 21:15:41.908374 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903334 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 21:15:41.908374 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903337 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 21:15:41.908374 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903339 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 21:15:41.908374 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903342 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 21:15:41.908374 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903345 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 21:15:41.908883 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903347 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 21:15:41.908883 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903350 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 21:15:41.908883 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903352 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 21:15:41.908883 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903355 2570 feature_gate.go:328] unrecognized feature gate: Example Jun 11 21:15:41.908883 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903357 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 21:15:41.908883 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903360 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 21:15:41.908883 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903363 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 21:15:41.908883 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903365 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 21:15:41.908883 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903368 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 21:15:41.908883 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903371 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 21:15:41.908883 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903373 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 21:15:41.908883 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903376 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 21:15:41.908883 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903380 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 21:15:41.908883 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903383 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 21:15:41.908883 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903385 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 21:15:41.908883 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903388 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 21:15:41.908883 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903390 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 21:15:41.908883 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903393 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 21:15:41.908883 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903396 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 21:15:41.908883 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903399 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 21:15:41.909374 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903402 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 21:15:41.909374 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.903405 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 21:15:41.909374 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903486 2570 flags.go:64] FLAG: --address="0.0.0.0" Jun 11 21:15:41.909374 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903497 2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 11 21:15:41.909374 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903506 2570 flags.go:64] FLAG: --anonymous-auth="true" Jun 11 21:15:41.909374 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903513 2570 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 11 21:15:41.909374 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903517 2570 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 11 21:15:41.909374 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903521 2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 11 21:15:41.909374 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903525 2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 11 21:15:41.909374 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903529 2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 11 21:15:41.909374 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903532 2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 11 21:15:41.909374 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903536 2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 11 21:15:41.909374 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903540 2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 11 21:15:41.909374 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903543 2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 11 21:15:41.909374 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903546 2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 11 21:15:41.909374 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903549 2570 flags.go:64] FLAG: --cgroup-root="" Jun 11 21:15:41.909374 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903552 2570 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 11 21:15:41.909374 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903555 2570 flags.go:64] FLAG: --client-ca-file="" Jun 11 21:15:41.909374 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903558 2570 flags.go:64] FLAG: --cloud-provider="external" Jun 11 21:15:41.909374 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903575 2570 flags.go:64] FLAG: --cluster-dns="[]" Jun 11 21:15:41.909374 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903583 2570 flags.go:64] FLAG: --cluster-domain="" Jun 11 21:15:41.909374 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903587 2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 11 21:15:41.909374 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903591 2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 11 21:15:41.909374 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903594 2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903598 2570 flags.go:64] FLAG: --container-log-max-files="5" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903602 2570 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903605 2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903608 2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903611 2570 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903615 2570 flags.go:64] FLAG: --contention-profiling="false" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903618 2570 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903625 2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903629 2570 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903632 2570 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903636 2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903640 2570 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903643 2570 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903646 2570 flags.go:64] FLAG: --enable-load-reader="false" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903649 2570 flags.go:64] FLAG: --enable-server="true" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903652 2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903656 2570 flags.go:64] FLAG: --event-burst="100" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903659 2570 flags.go:64] FLAG: --event-qps="50" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903663 2570 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903666 2570 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903669 2570 flags.go:64] FLAG: --eviction-hard="" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903673 2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903676 2570 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903679 2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 11 21:15:41.909966 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903682 2570 flags.go:64] FLAG: --eviction-soft="" Jun 11 21:15:41.910668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903685 2570 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 11 21:15:41.910668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903688 2570 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 11 21:15:41.910668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903691 2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 11 21:15:41.910668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903694 2570 flags.go:64] FLAG: --experimental-mounter-path="" Jun 11 21:15:41.910668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903697 2570 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 11 21:15:41.910668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903701 2570 flags.go:64] FLAG: --fail-swap-on="true" Jun 11 21:15:41.910668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903704 2570 flags.go:64] FLAG: --feature-gates="" Jun 11 21:15:41.910668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903708 2570 flags.go:64] FLAG: --file-check-frequency="20s" Jun 11 21:15:41.910668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903711 2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 11 21:15:41.910668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903714 2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 11 21:15:41.910668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903717 2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 11 21:15:41.910668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903721 2570 flags.go:64] FLAG: --healthz-port="10248" Jun 11 21:15:41.910668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903724 2570 flags.go:64] FLAG: --help="false" Jun 11 21:15:41.910668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903727 2570 flags.go:64] FLAG: --hostname-override="ip-10-0-143-6.ec2.internal" Jun 11 21:15:41.910668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903731 2570 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 11 21:15:41.910668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903734 2570 flags.go:64] FLAG: --http-check-frequency="20s" Jun 11 21:15:41.910668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903737 2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 11 21:15:41.910668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903741 2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 11 21:15:41.910668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903744 2570 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 11 21:15:41.910668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903747 2570 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 11 21:15:41.910668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903750 2570 flags.go:64] FLAG: --image-service-endpoint="" Jun 11 21:15:41.910668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903753 2570 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 11 21:15:41.910668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903755 2570 flags.go:64] FLAG: --kube-api-burst="100" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903758 2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903765 2570 flags.go:64] FLAG: --kube-api-qps="50" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903768 2570 flags.go:64] FLAG: --kube-reserved="" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903771 2570 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903774 2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903777 2570 flags.go:64] FLAG: --kubelet-cgroups="" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903780 2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903783 2570 flags.go:64] FLAG: --lock-file="" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903786 2570 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903789 2570 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903792 2570 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903798 2570 flags.go:64] FLAG: --log-json-split-stream="false" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903801 2570 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903804 2570 flags.go:64] FLAG: --log-text-split-stream="false" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903807 2570 flags.go:64] FLAG: --logging-format="text" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903810 2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903814 2570 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903816 2570 flags.go:64] FLAG: --manifest-url="" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903819 2570 flags.go:64] FLAG: --manifest-url-header="" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903824 2570 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903827 2570 flags.go:64] FLAG: --max-open-files="1000000" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903831 2570 flags.go:64] FLAG: --max-pods="110" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903834 2570 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903839 2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 11 21:15:41.911430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903842 2570 flags.go:64] FLAG: --memory-manager-policy="None" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903845 2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903848 2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903851 2570 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903855 2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903862 2570 flags.go:64] FLAG: --node-status-max-images="50" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903865 2570 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903868 2570 flags.go:64] FLAG: --oom-score-adj="-999" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903871 2570 flags.go:64] FLAG: --pod-cidr="" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903876 2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4a2e90c8ef7d783e4d635e24fb09435f5e1f6e954583512c0944f9e57bc77b52" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903882 2570 flags.go:64] FLAG: --pod-manifest-path="" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903885 2570 flags.go:64] FLAG: --pod-max-pids="-1" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903888 2570 flags.go:64] FLAG: --pods-per-core="0" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903891 2570 flags.go:64] FLAG: --port="10250" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903894 2570 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903897 2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0cb4433e63ac0dcbb" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903900 2570 flags.go:64] FLAG: --qos-reserved="" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903904 2570 flags.go:64] FLAG: --read-only-port="10255" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903907 2570 flags.go:64] FLAG: --register-node="true" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903910 2570 flags.go:64] FLAG: --register-with-taints="" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903914 2570 flags.go:64] FLAG: --registry-burst="10" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903917 2570 flags.go:64] FLAG: --registry-qps="5" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903920 2570 flags.go:64] FLAG: --reserved-cpus="" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903922 2570 flags.go:64] FLAG: --reserved-memory="" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903927 2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 11 21:15:41.912419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903930 2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903934 2570 flags.go:64] FLAG: --rotate-certificates="false" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903937 2570 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903940 2570 flags.go:64] FLAG: --runonce="false" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903942 2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903946 2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903950 2570 flags.go:64] FLAG: --seccomp-default="false" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903953 2570 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903955 2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903959 2570 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903962 2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903965 2570 flags.go:64] FLAG: --storage-driver-password="root" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903968 2570 flags.go:64] FLAG: --storage-driver-secure="false" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903971 2570 flags.go:64] FLAG: --storage-driver-table="stats" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903974 2570 flags.go:64] FLAG: --storage-driver-user="root" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903977 2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903980 2570 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903984 2570 flags.go:64] FLAG: --system-cgroups="" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903987 2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903992 2570 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903995 2570 flags.go:64] FLAG: --tls-cert-file="" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.903998 2570 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.904002 2570 flags.go:64] FLAG: --tls-min-version="" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.904005 2570 flags.go:64] FLAG: --tls-private-key-file="" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.904008 2570 flags.go:64] FLAG: --topology-manager-policy="none" Jun 11 21:15:41.913153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.904011 2570 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 11 21:15:41.913887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.904015 2570 flags.go:64] FLAG: --topology-manager-scope="container" Jun 11 21:15:41.913887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.904018 2570 flags.go:64] FLAG: --v="2" Jun 11 21:15:41.913887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.904022 2570 flags.go:64] FLAG: --version="false" Jun 11 21:15:41.913887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.904026 2570 flags.go:64] FLAG: --vmodule="" Jun 11 21:15:41.913887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.904029 2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 11 21:15:41.913887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.904033 2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 11 21:15:41.913887 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904150 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 21:15:41.913887 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904154 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 21:15:41.913887 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904157 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 21:15:41.913887 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904160 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 21:15:41.913887 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904163 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 21:15:41.913887 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904166 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 21:15:41.913887 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904170 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 21:15:41.913887 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904173 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 21:15:41.913887 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904175 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 21:15:41.913887 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904178 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 21:15:41.913887 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904181 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 21:15:41.913887 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904183 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 21:15:41.913887 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904186 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 21:15:41.913887 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904188 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 21:15:41.913887 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904191 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 21:15:41.913887 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904193 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 21:15:41.914420 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904196 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 21:15:41.914420 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904200 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 21:15:41.914420 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904202 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 21:15:41.914420 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904205 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 21:15:41.914420 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904208 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 21:15:41.914420 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904210 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 21:15:41.914420 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904213 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 21:15:41.914420 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904217 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 21:15:41.914420 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904220 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 21:15:41.914420 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904223 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 21:15:41.914420 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904226 2570 feature_gate.go:328] unrecognized feature gate: Example Jun 11 21:15:41.914420 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904229 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 21:15:41.914420 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904232 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 21:15:41.914420 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904234 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 21:15:41.914420 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904237 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 21:15:41.914420 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904240 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 21:15:41.914420 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904243 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 21:15:41.914420 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904250 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 21:15:41.914420 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904253 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 21:15:41.914897 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904256 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 21:15:41.914897 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904258 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 21:15:41.914897 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904261 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 21:15:41.914897 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904265 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 21:15:41.914897 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904269 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 21:15:41.914897 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904273 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 21:15:41.914897 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904276 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 21:15:41.914897 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904279 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 21:15:41.914897 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904282 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 21:15:41.914897 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904284 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 21:15:41.914897 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904287 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 21:15:41.914897 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904290 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 21:15:41.914897 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904293 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 21:15:41.914897 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904296 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 21:15:41.914897 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904300 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 21:15:41.914897 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904303 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 21:15:41.914897 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904306 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 21:15:41.914897 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904309 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 21:15:41.914897 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904312 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 21:15:41.915390 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904315 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 21:15:41.915390 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904318 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 21:15:41.915390 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904321 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 21:15:41.915390 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904324 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 21:15:41.915390 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904327 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 21:15:41.915390 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904330 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 21:15:41.915390 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904332 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 21:15:41.915390 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904335 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 21:15:41.915390 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904338 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 21:15:41.915390 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904340 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 21:15:41.915390 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904343 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 21:15:41.915390 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904345 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 21:15:41.915390 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904350 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 21:15:41.915390 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904353 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 21:15:41.915390 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904356 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 21:15:41.915390 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904359 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 21:15:41.915390 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904363 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 21:15:41.915390 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904365 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 21:15:41.915390 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904368 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 21:15:41.915916 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904370 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 21:15:41.915916 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904373 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 21:15:41.915916 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904376 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 21:15:41.915916 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904379 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 21:15:41.915916 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904381 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 21:15:41.915916 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904384 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 21:15:41.915916 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904387 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 21:15:41.915916 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904390 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 21:15:41.915916 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904394 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 21:15:41.915916 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904396 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 21:15:41.915916 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904399 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 21:15:41.915916 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904402 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 21:15:41.915916 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904404 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 21:15:41.915916 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904407 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 21:15:41.915916 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904409 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 21:15:41.915916 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904413 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 21:15:41.915916 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904416 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 21:15:41.915916 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904419 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 21:15:41.915916 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904422 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 21:15:41.915916 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904424 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 21:15:41.916409 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904427 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 21:15:41.916409 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904429 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 21:15:41.916409 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904432 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 21:15:41.916409 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904435 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 21:15:41.916409 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904437 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 21:15:41.916409 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.904440 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 21:15:41.916409 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.905079 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 21:15:41.916409 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.913788 2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 11 21:15:41.916409 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.913808 2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 11 21:15:41.916409 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.913829 2570 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 11 21:15:41.916409 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.913836 2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 11 21:15:41.916409 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913897 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 21:15:41.916409 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913903 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 21:15:41.916409 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913906 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 21:15:41.916409 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913909 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 21:15:41.916804 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913912 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 21:15:41.916804 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913915 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 21:15:41.916804 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913918 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 21:15:41.916804 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913921 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 21:15:41.916804 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913924 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 21:15:41.916804 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913926 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 21:15:41.916804 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913929 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 21:15:41.916804 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913932 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 21:15:41.916804 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913934 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 21:15:41.916804 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913937 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 21:15:41.916804 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913940 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 21:15:41.916804 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913942 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 21:15:41.916804 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913945 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 21:15:41.916804 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913948 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 21:15:41.916804 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913951 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 21:15:41.916804 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913954 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 21:15:41.916804 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913957 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 21:15:41.916804 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913959 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 21:15:41.916804 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913962 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 21:15:41.916804 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913965 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 21:15:41.917298 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913968 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 21:15:41.917298 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913970 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 21:15:41.917298 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913973 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 21:15:41.917298 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913975 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 21:15:41.917298 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913978 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 21:15:41.917298 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913981 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 21:15:41.917298 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913984 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 21:15:41.917298 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913986 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 21:15:41.917298 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913989 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 21:15:41.917298 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913992 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 21:15:41.917298 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913994 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 21:15:41.917298 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.913997 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 21:15:41.917298 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914000 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 21:15:41.917298 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914002 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 21:15:41.917298 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914005 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 21:15:41.917298 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914009 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 21:15:41.917298 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914013 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 21:15:41.917298 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914016 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 21:15:41.917298 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914019 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 21:15:41.917298 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914021 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 21:15:41.917889 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914024 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 21:15:41.917889 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914026 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 21:15:41.917889 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914029 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 21:15:41.917889 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914032 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 21:15:41.917889 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914034 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 21:15:41.917889 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914037 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 21:15:41.917889 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914041 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 21:15:41.917889 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914044 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 21:15:41.917889 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914046 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 21:15:41.917889 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914049 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 21:15:41.917889 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914052 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 21:15:41.917889 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914054 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 21:15:41.917889 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914057 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 21:15:41.917889 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914059 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 21:15:41.917889 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914063 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 21:15:41.917889 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914067 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 21:15:41.917889 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914070 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 21:15:41.917889 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914073 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 21:15:41.917889 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914076 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 21:15:41.918352 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914079 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 21:15:41.918352 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914082 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 21:15:41.918352 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914084 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 21:15:41.918352 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914087 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 21:15:41.918352 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914090 2570 feature_gate.go:328] unrecognized feature gate: Example Jun 11 21:15:41.918352 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914092 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 21:15:41.918352 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914095 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 21:15:41.918352 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914098 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 21:15:41.918352 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914102 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 21:15:41.918352 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914106 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 21:15:41.918352 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914109 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 21:15:41.918352 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914112 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 21:15:41.918352 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914115 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 21:15:41.918352 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914117 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 21:15:41.918352 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914120 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 21:15:41.918352 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914122 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 21:15:41.918352 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914126 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 21:15:41.918352 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914128 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 21:15:41.918352 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914131 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 21:15:41.918834 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914134 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 21:15:41.918834 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914136 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 21:15:41.918834 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914140 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 21:15:41.918834 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914142 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 21:15:41.918834 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914145 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 21:15:41.918834 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914148 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 21:15:41.918834 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914150 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 21:15:41.918834 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914153 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 21:15:41.918834 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914156 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 21:15:41.918834 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914159 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 21:15:41.918834 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914161 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 21:15:41.918834 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914164 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 21:15:41.918834 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914167 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 21:15:41.918834 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914170 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 21:15:41.918834 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914172 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 21:15:41.918834 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914175 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 21:15:41.918834 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914177 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 21:15:41.919286 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.914183 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 21:15:41.919286 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914296 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 21:15:41.919286 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914301 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 21:15:41.919286 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914305 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 21:15:41.919286 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914308 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 21:15:41.919286 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914311 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 21:15:41.919286 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914314 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 21:15:41.919286 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914318 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 21:15:41.919286 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914323 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 21:15:41.919286 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914326 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 21:15:41.919286 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914328 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 21:15:41.919286 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914331 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 21:15:41.919286 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914334 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 21:15:41.919286 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914337 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 21:15:41.919286 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914340 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 21:15:41.919703 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914343 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 21:15:41.919703 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914346 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 21:15:41.919703 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914348 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 21:15:41.919703 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914351 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 21:15:41.919703 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914354 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 21:15:41.919703 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914356 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 21:15:41.919703 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914359 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 21:15:41.919703 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914361 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 21:15:41.919703 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914364 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 21:15:41.919703 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914367 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 21:15:41.919703 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914369 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 21:15:41.919703 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914372 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 21:15:41.919703 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914375 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 21:15:41.919703 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914378 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 21:15:41.919703 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914381 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 21:15:41.919703 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914383 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 21:15:41.919703 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914386 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 21:15:41.919703 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914389 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 21:15:41.919703 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914392 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 21:15:41.919703 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914394 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 21:15:41.920224 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914397 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 21:15:41.920224 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914399 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 21:15:41.920224 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914402 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 21:15:41.920224 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914405 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 21:15:41.920224 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914408 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 21:15:41.920224 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914410 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 21:15:41.920224 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914413 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 21:15:41.920224 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914415 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 21:15:41.920224 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914418 2570 feature_gate.go:328] unrecognized feature gate: Example Jun 11 21:15:41.920224 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914420 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 21:15:41.920224 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914423 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 21:15:41.920224 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914426 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 21:15:41.920224 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914429 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 21:15:41.920224 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914432 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 21:15:41.920224 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914435 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 21:15:41.920224 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914437 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 21:15:41.920224 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914440 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 21:15:41.920224 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914442 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 21:15:41.920224 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914445 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 21:15:41.920224 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914447 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 21:15:41.920736 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914450 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 21:15:41.920736 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914453 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 21:15:41.920736 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914455 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 21:15:41.920736 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914458 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 21:15:41.920736 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914461 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 21:15:41.920736 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914463 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 21:15:41.920736 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914466 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 21:15:41.920736 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914468 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 21:15:41.920736 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914471 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 21:15:41.920736 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914474 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 21:15:41.920736 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914476 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 21:15:41.920736 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914479 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 21:15:41.920736 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914481 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 21:15:41.920736 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914484 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 21:15:41.920736 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914487 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 21:15:41.920736 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914489 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 21:15:41.920736 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914492 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 21:15:41.920736 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914495 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 21:15:41.920736 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914498 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 21:15:41.920736 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914500 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 21:15:41.921332 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914503 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 21:15:41.921332 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914507 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 21:15:41.921332 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914510 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 21:15:41.921332 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914513 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 21:15:41.921332 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914518 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 21:15:41.921332 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914521 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 21:15:41.921332 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914524 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 21:15:41.921332 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914526 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 21:15:41.921332 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914529 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 21:15:41.921332 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914532 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 21:15:41.921332 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914534 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 21:15:41.921332 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914537 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 21:15:41.921332 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914540 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 21:15:41.921332 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914543 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 21:15:41.921332 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914545 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 21:15:41.921332 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914548 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 21:15:41.921332 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914551 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 21:15:41.921332 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914553 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 21:15:41.921332 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914556 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 21:15:41.921825 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914558 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 21:15:41.921825 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914578 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 21:15:41.921825 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914583 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 21:15:41.921825 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914586 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 21:15:41.921825 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914589 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 21:15:41.921825 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:41.914592 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 21:15:41.921825 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.914598 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 21:15:41.921825 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.914725 2570 server.go:962] "Client rotation is on, will bootstrap in background" Jun 11 21:15:41.921825 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.919346 2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 11 21:15:41.921825 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.920851 2570 server.go:1019] "Starting client certificate rotation" Jun 11 21:15:41.921825 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.920971 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 11 21:15:41.921825 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.921021 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 11 21:15:41.923595 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.923560 2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 11 21:15:41.924773 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.924753 2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 11 21:15:41.935586 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.935556 2570 log.go:25] "Validated CRI v1 runtime API" Jun 11 21:15:41.939264 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.939249 2570 log.go:25] "Validated CRI v1 image API" Jun 11 21:15:41.940957 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.940941 2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 11 21:15:41.945619 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.945598 2570 fs.go:135] Filesystem UUIDs: map[33f8194b-ea24-428e-bf07-eba8cc717b48:/dev/nvme0n1p3 5307c4ed-4d6d-44cd-9f62-2a26d991226d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jun 11 21:15:41.945691 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.945616 2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 11 21:15:41.952636 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.952616 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 11 21:15:41.953279 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.953167 2570 manager.go:216] Machine: {Timestamp:2026-06-11 21:15:41.951189466 +0000 UTC m=+0.362513023 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099945 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec237b1b665dc1d10eb6eb1237107369 SystemUUID:ec237b1b-665d-c1d1-0eb6-eb1237107369 BootID:4e4923db-b49d-4af4-81c7-8d65d4877277 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f9:38:bc:b8:85 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f9:38:bc:b8:85 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:76:3f:f1:35:85:d0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 11 21:15:41.953279 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.953276 2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 11 21:15:41.953400 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.953388 2570 manager.go:232] Version: {KernelVersion:5.14.0-570.120.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260608-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 11 21:15:41.957374 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.957349 2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 11 21:15:41.957514 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.957378 2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-6.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 11 21:15:41.958657 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.958647 2570 topology_manager.go:138] "Creating topology manager with none policy" Jun 11 21:15:41.958699 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.958660 2570 container_manager_linux.go:309] "Creating device plugin manager" Jun 11 21:15:41.958699 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.958667 2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 11 21:15:41.959558 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.959548 2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 11 21:15:41.960949 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.960938 2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 11 21:15:41.964180 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.964167 2570 state_mem.go:36] "Initialized new in-memory state store" Jun 11 21:15:41.964299 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.964290 2570 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 11 21:15:41.964334 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.964320 2570 kubelet.go:486] "Attempting to sync node with API server" Jun 11 21:15:41.964334 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.964330 2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 11 21:15:41.964389 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.964345 2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 11 21:15:41.964389 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.964354 2570 kubelet.go:398] "Adding apiserver pod source" Jun 11 21:15:41.964389 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.964363 2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 11 21:15:41.964755 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.964744 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 11 21:15:41.964810 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.964787 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 11 21:15:41.967143 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.967120 2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 11 21:15:41.968475 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.968461 2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 11 21:15:41.968525 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.968493 2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 11 21:15:41.968667 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.968654 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 11 21:15:41.968708 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.968676 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 11 21:15:41.968708 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.968683 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 11 21:15:41.968708 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.968690 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 11 21:15:41.968708 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.968698 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 11 21:15:41.968708 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.968703 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 11 21:15:41.969327 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.969318 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 11 21:15:41.969327 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.969327 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 11 21:15:41.969387 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.969334 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 11 21:15:41.969387 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.969339 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 11 21:15:41.969387 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.969347 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 11 21:15:41.969387 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.969356 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 11 21:15:41.970176 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.970166 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 11 21:15:41.970212 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.970180 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 11 21:15:41.972407 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.972383 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bw42w" Jun 11 21:15:41.973558 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.973544 2570 server.go:1268] "Started kubelet" Jun 11 21:15:41.973740 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.973691 2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 11 21:15:41.973801 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.973761 2570 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 11 21:15:41.973922 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.973867 2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 11 21:15:41.974442 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:41.974419 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 11 21:15:41.974517 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:41.974428 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-6.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 11 21:15:41.974517 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.974446 2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-6.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 11 21:15:41.974511 ip-10-0-143-6 systemd[1]: Started Kubernetes Kubelet. Jun 11 21:15:41.975873 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.975854 2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 11 21:15:41.976151 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.976136 2570 server.go:310] "Adding debug handlers to kubelet server" Jun 11 21:15:41.978649 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.978627 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bw42w" Jun 11 21:15:41.979683 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:41.978659 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-6.ec2.internal.18b8235ea1cd466c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-6.ec2.internal,UID:ip-10-0-143-6.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-6.ec2.internal,},FirstTimestamp:2026-06-11 21:15:41.973513836 +0000 UTC m=+0.384837367,LastTimestamp:2026-06-11 21:15:41.973513836 +0000 UTC m=+0.384837367,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-6.ec2.internal,}" Jun 11 21:15:41.981487 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.981467 2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 11 21:15:41.981487 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.981468 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 11 21:15:41.981683 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:41.981666 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Jun 11 21:15:41.982166 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.982045 2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 11 21:15:41.982272 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.982253 2570 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 11 21:15:41.982272 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.982271 2570 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 11 21:15:41.982397 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.982342 2570 reconstruct.go:97] "Volume reconstruction finished" Jun 11 21:15:41.982397 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.982353 2570 reconciler.go:29] "Reconciler: start to sync state" Jun 11 21:15:41.982809 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:41.982765 2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 11 21:15:41.982990 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.982966 2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 11 21:15:41.982990 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.982987 2570 factory.go:55] Registering systemd factory Jun 11 21:15:41.983105 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.982997 2570 factory.go:223] Registration of the systemd container factory successfully Jun 11 21:15:41.983242 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.983224 2570 factory.go:153] Registering CRI-O factory Jun 11 21:15:41.983242 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.983243 2570 factory.go:223] Registration of the crio container factory successfully Jun 11 21:15:41.983349 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.983264 2570 factory.go:103] Registering Raw factory Jun 11 21:15:41.983349 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.983277 2570 manager.go:1196] Started watching for new ooms in manager Jun 11 21:15:41.983433 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:41.983412 2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-6.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 11 21:15:41.983472 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:41.983440 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 11 21:15:41.983733 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.983713 2570 manager.go:349] Starting recovery of all containers Jun 11 21:15:41.991207 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.991168 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 11 21:15:41.995034 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.995013 2570 manager.go:354] Recovery completed Jun 11 21:15:41.996506 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:41.996488 2570 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jun 11 21:15:41.999520 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:41.999509 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 21:15:42.001238 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.001223 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientMemory" Jun 11 21:15:42.001304 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.001255 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 21:15:42.001304 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.001268 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientPID" Jun 11 21:15:42.001862 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.001847 2570 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 11 21:15:42.001862 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.001860 2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 11 21:15:42.001862 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.001851 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 21:15:42.001988 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.001876 2570 state_mem.go:36] "Initialized new in-memory state store" Jun 11 21:15:42.002465 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.002451 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientMemory" Jun 11 21:15:42.002525 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.002476 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 21:15:42.002525 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.002486 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientPID" Jun 11 21:15:42.002977 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.002965 2570 policy_none.go:49] "None policy: Start" Jun 11 21:15:42.003015 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.002984 2570 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 11 21:15:42.003015 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.002995 2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 11 21:15:42.003981 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.003971 2570 policy_none.go:47] "Start" Jun 11 21:15:42.046060 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.045896 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 11 21:15:42.050222 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.046085 2570 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 11 21:15:42.050222 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.046114 2570 kubelet.go:2448] "Starting kubelet main sync loop" Jun 11 21:15:42.050222 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:42.046154 2570 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jun 11 21:15:42.050222 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.047261 2570 manager.go:337] "Starting Device Plugin manager" Jun 11 21:15:42.050222 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:42.047290 2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 11 21:15:42.050222 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.047303 2570 server.go:87] "Starting device plugin registration server" Jun 11 21:15:42.050222 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.047648 2570 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 11 21:15:42.050222 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.047660 2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 11 21:15:42.050222 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.047749 2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 11 21:15:42.050222 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.047828 2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 11 21:15:42.050222 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.047852 2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 11 21:15:42.050222 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:42.048371 2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 11 21:15:42.050222 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:42.048430 2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-6.ec2.internal\" not found" Jun 11 21:15:42.050222 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.048505 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 21:15:42.146507 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.146424 2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal"] Jun 11 21:15:42.146628 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.146526 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 21:15:42.147846 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.147825 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 21:15:42.148449 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.148418 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientMemory" Jun 11 21:15:42.148532 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.148456 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 21:15:42.148532 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.148469 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientPID" Jun 11 21:15:42.148532 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.148491 2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-6.ec2.internal" Jun 11 21:15:42.149006 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.148987 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientMemory" Jun 11 21:15:42.149057 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.149025 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 21:15:42.149057 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.149038 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientPID" Jun 11 21:15:42.150262 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.150250 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 21:15:42.150359 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.150346 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" Jun 11 21:15:42.150392 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.150375 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 21:15:42.152178 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.152156 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientMemory" Jun 11 21:15:42.152254 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.152181 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 21:15:42.152254 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.152196 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientPID" Jun 11 21:15:42.152254 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.152199 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientMemory" Jun 11 21:15:42.152254 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.152216 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 21:15:42.152254 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.152228 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientPID" Jun 11 21:15:42.153417 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.153401 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal" Jun 11 21:15:42.153482 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.153426 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 21:15:42.154598 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.154582 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientMemory" Jun 11 21:15:42.154691 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.154613 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 21:15:42.154691 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.154629 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeHasSufficientPID" Jun 11 21:15:42.159988 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.159973 2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-6.ec2.internal" Jun 11 21:15:42.160065 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:42.159992 2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-6.ec2.internal\": node \"ip-10-0-143-6.ec2.internal\" not found" Jun 11 21:15:42.169151 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:42.169127 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-6.ec2.internal\" not found" node="ip-10-0-143-6.ec2.internal" Jun 11 21:15:42.173011 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:42.172997 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-6.ec2.internal\" not found" node="ip-10-0-143-6.ec2.internal" Jun 11 21:15:42.177995 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:42.177975 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Jun 11 21:15:42.184244 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.184225 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de484ba5593857b981e91fc7cd0040af-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal\" (UID: \"de484ba5593857b981e91fc7cd0040af\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" Jun 11 21:15:42.184303 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.184256 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f9fdf41d985ef036011199dc3759ac96-config\") pod \"kube-apiserver-proxy-ip-10-0-143-6.ec2.internal\" (UID: \"f9fdf41d985ef036011199dc3759ac96\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal" Jun 11 21:15:42.184303 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.184280 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/de484ba5593857b981e91fc7cd0040af-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal\" (UID: \"de484ba5593857b981e91fc7cd0040af\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" Jun 11 21:15:42.279004 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:42.278978 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Jun 11 21:15:42.285337 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.285314 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/de484ba5593857b981e91fc7cd0040af-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal\" (UID: \"de484ba5593857b981e91fc7cd0040af\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" Jun 11 21:15:42.285398 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.285344 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de484ba5593857b981e91fc7cd0040af-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal\" (UID: \"de484ba5593857b981e91fc7cd0040af\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" Jun 11 21:15:42.285398 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.285362 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f9fdf41d985ef036011199dc3759ac96-config\") pod \"kube-apiserver-proxy-ip-10-0-143-6.ec2.internal\" (UID: \"f9fdf41d985ef036011199dc3759ac96\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal" Jun 11 21:15:42.285464 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.285402 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/de484ba5593857b981e91fc7cd0040af-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal\" (UID: \"de484ba5593857b981e91fc7cd0040af\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" Jun 11 21:15:42.285464 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.285418 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de484ba5593857b981e91fc7cd0040af-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal\" (UID: \"de484ba5593857b981e91fc7cd0040af\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" Jun 11 21:15:42.285464 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.285402 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f9fdf41d985ef036011199dc3759ac96-config\") pod \"kube-apiserver-proxy-ip-10-0-143-6.ec2.internal\" (UID: \"f9fdf41d985ef036011199dc3759ac96\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal" Jun 11 21:15:42.379514 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:42.379480 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Jun 11 21:15:42.471303 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.471224 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" Jun 11 21:15:42.476525 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.476509 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal" Jun 11 21:15:42.479599 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:42.479578 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Jun 11 21:15:42.580673 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:42.580622 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Jun 11 21:15:42.681140 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:42.681116 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Jun 11 21:15:42.781693 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:42.781587 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Jun 11 21:15:42.850189 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.850159 2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 21:15:42.881997 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:42.881970 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Jun 11 21:15:42.920350 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.920322 2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 11 21:15:42.920884 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.920508 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 11 21:15:42.920884 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.920524 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 11 21:15:42.974345 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:42.974307 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9fdf41d985ef036011199dc3759ac96.slice/crio-4d61486087b39e15551af90e1cc2a3a1b0520926b672b2b39d5abd505ce1f41d WatchSource:0}: Error finding container 4d61486087b39e15551af90e1cc2a3a1b0520926b672b2b39d5abd505ce1f41d: Status 404 returned error can't find the container with id 4d61486087b39e15551af90e1cc2a3a1b0520926b672b2b39d5abd505ce1f41d Jun 11 21:15:42.974752 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:42.974725 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde484ba5593857b981e91fc7cd0040af.slice/crio-a1483eae81de0b093964abe3faaed852b9a3b80040abdd2b498be24466fc565f WatchSource:0}: Error finding container a1483eae81de0b093964abe3faaed852b9a3b80040abdd2b498be24466fc565f: Status 404 returned error can't find the container with id a1483eae81de0b093964abe3faaed852b9a3b80040abdd2b498be24466fc565f Jun 11 21:15:42.980981 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.980967 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 21:15:42.983048 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.983009 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-10 21:10:41 +0000 UTC" deadline="2028-01-29 15:18:29.142543831 +0000 UTC" Jun 11 21:15:42.983048 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:42.983045 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14322h2m46.15950153s" Jun 11 21:15:42.983048 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:42.983049 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Jun 11 21:15:43.051354 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.050186 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" event={"ID":"de484ba5593857b981e91fc7cd0040af","Type":"ContainerStarted","Data":"a1483eae81de0b093964abe3faaed852b9a3b80040abdd2b498be24466fc565f"} Jun 11 21:15:43.052438 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.052408 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal" event={"ID":"f9fdf41d985ef036011199dc3759ac96","Type":"ContainerStarted","Data":"4d61486087b39e15551af90e1cc2a3a1b0520926b672b2b39d5abd505ce1f41d"} Jun 11 21:15:43.083658 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:43.083639 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Jun 11 21:15:43.184196 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:43.184157 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Jun 11 21:15:43.284706 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:43.284669 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Jun 11 21:15:43.344753 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.344676 2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 21:15:43.385789 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:43.385758 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-6.ec2.internal\" not found" Jun 11 21:15:43.424631 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.424601 2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 21:15:43.482744 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.482710 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" Jun 11 21:15:43.494017 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.493973 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 11 21:15:43.495194 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.495162 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal" Jun 11 21:15:43.501744 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.501711 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 11 21:15:43.867390 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.867366 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 21:15:43.965641 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.965607 2570 apiserver.go:52] "Watching apiserver" Jun 11 21:15:43.972055 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.972031 2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 11 21:15:43.972415 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.972389 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-4c247","kube-system/konnectivity-agent-x65dp","kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x","openshift-image-registry/node-ca-thgw8","openshift-multus/multus-additional-cni-plugins-w26g9","openshift-network-diagnostics/network-check-target-shxrd","openshift-cluster-node-tuning-operator/tuned-fhhkw","openshift-dns/node-resolver-s8b5p","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal","openshift-multus/multus-bl9g7","openshift-multus/network-metrics-daemon-2bmhl","openshift-network-operator/iptables-alerter-cqksf","openshift-ovn-kubernetes/ovnkube-node-b4gmk"] Jun 11 21:15:43.973940 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.973917 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:15:43.974044 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:43.974010 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shxrd" podUID="54688996-c5d4-445d-a6ff-4997a19fe1d6" Jun 11 21:15:43.975119 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.975095 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cqksf" Jun 11 21:15:43.976412 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.976392 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:43.977886 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.977603 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 11 21:15:43.977886 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.977610 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 11 21:15:43.977886 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.977738 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-67d56\"" Jun 11 21:15:43.977886 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.977611 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 11 21:15:43.978843 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.978322 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" Jun 11 21:15:43.978843 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.978750 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 11 21:15:43.978994 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.978865 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 11 21:15:43.978994 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.978970 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 11 21:15:43.979097 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.978997 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 11 21:15:43.979457 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.979361 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4v5gd\"" Jun 11 21:15:43.979576 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.979541 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 11 21:15:43.979722 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.979703 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 11 21:15:43.980027 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.980000 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-thgw8" Jun 11 21:15:43.980653 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.980450 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 11 21:15:43.980653 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.980490 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-w89qw\"" Jun 11 21:15:43.980999 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.980971 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 11 21:15:43.981088 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.980978 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 11 21:15:43.981384 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.981367 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:43.981658 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.981637 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 11 21:15:43.982334 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.982305 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 11 21:15:43.982424 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.982336 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 11 21:15:43.982424 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.982403 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 11 21:15:43.982544 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.982514 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cjc52\"" Jun 11 21:15:43.983210 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.983190 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s8b5p" Jun 11 21:15:43.987025 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.984711 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 11 21:15:43.987025 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.985109 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xnwzj\"" Jun 11 21:15:43.987025 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.985288 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 11 21:15:43.987025 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.985406 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 11 21:15:43.987025 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.985519 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 11 21:15:43.987025 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.986923 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 11 21:15:43.987323 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.987149 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 11 21:15:43.987323 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.987305 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 11 21:15:43.988055 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.987611 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pr9dx\"" Jun 11 21:15:43.989333 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.988338 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:15:43.989333 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:43.988431 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4c247" podUID="6f4a1fd8-505b-4061-a432-620cb3c68386" Jun 11 21:15:43.990827 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.990546 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x65dp" Jun 11 21:15:43.992366 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.992346 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:43.992904 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.992887 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 11 21:15:43.992984 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.992941 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 11 21:15:43.992984 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.992965 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jrt7d\"" Jun 11 21:15:43.993708 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.993689 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:43.993792 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.993720 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b88bd69-b4df-4345-a24c-e51adeba9023-host\") pod \"node-ca-thgw8\" (UID: \"7b88bd69-b4df-4345-a24c-e51adeba9023\") " pod="openshift-image-registry/node-ca-thgw8" Jun 11 21:15:43.993792 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.993745 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5678560c-39b0-4892-a879-3613ac253097-system-cni-dir\") pod \"multus-additional-cni-plugins-w26g9\" (UID: \"5678560c-39b0-4892-a879-3613ac253097\") " pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:43.993792 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.993769 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c7849da4-f326-4b8c-a436-5b0d68958170-ovnkube-config\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:43.993943 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.993792 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7krj\" (UniqueName: \"kubernetes.io/projected/5678560c-39b0-4892-a879-3613ac253097-kube-api-access-c7krj\") pod \"multus-additional-cni-plugins-w26g9\" (UID: \"5678560c-39b0-4892-a879-3613ac253097\") " pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:43.993943 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.993802 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bl9g7" Jun 11 21:15:43.993943 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.993814 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f88e5323-c309-4bdb-ae95-359b2a92061b-registration-dir\") pod \"aws-ebs-csi-driver-node-kc22x\" (UID: \"f88e5323-c309-4bdb-ae95-359b2a92061b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" Jun 11 21:15:43.993943 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.993838 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c7849da4-f326-4b8c-a436-5b0d68958170-env-overrides\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:43.993943 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.993866 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/71fed145-1fbc-4433-bfb0-394c5631bec6-iptables-alerter-script\") pod \"iptables-alerter-cqksf\" (UID: \"71fed145-1fbc-4433-bfb0-394c5631bec6\") " pod="openshift-network-operator/iptables-alerter-cqksf" Jun 11 21:15:43.993943 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.993888 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/71fed145-1fbc-4433-bfb0-394c5631bec6-host-slash\") pod \"iptables-alerter-cqksf\" (UID: \"71fed145-1fbc-4433-bfb0-394c5631bec6\") " pod="openshift-network-operator/iptables-alerter-cqksf" Jun 11 21:15:43.994214 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.993946 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-run-openvswitch\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:43.994214 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.993970 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7b88bd69-b4df-4345-a24c-e51adeba9023-serviceca\") pod \"node-ca-thgw8\" (UID: \"7b88bd69-b4df-4345-a24c-e51adeba9023\") " pod="openshift-image-registry/node-ca-thgw8" Jun 11 21:15:43.994214 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.993992 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f88e5323-c309-4bdb-ae95-359b2a92061b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kc22x\" (UID: \"f88e5323-c309-4bdb-ae95-359b2a92061b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" Jun 11 21:15:43.994214 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994015 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdfaf7d3-ae13-48e9-b9ea-040ae02d886a-tmp-dir\") pod \"node-resolver-s8b5p\" (UID: \"bdfaf7d3-ae13-48e9-b9ea-040ae02d886a\") " pod="openshift-dns/node-resolver-s8b5p" Jun 11 21:15:43.994214 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994038 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zplqm\" (UniqueName: \"kubernetes.io/projected/71fed145-1fbc-4433-bfb0-394c5631bec6-kube-api-access-zplqm\") pod \"iptables-alerter-cqksf\" (UID: \"71fed145-1fbc-4433-bfb0-394c5631bec6\") " pod="openshift-network-operator/iptables-alerter-cqksf" Jun 11 21:15:43.994214 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994065 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5678560c-39b0-4892-a879-3613ac253097-cni-binary-copy\") pod \"multus-additional-cni-plugins-w26g9\" (UID: \"5678560c-39b0-4892-a879-3613ac253097\") " pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:43.994214 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994088 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f88e5323-c309-4bdb-ae95-359b2a92061b-socket-dir\") pod \"aws-ebs-csi-driver-node-kc22x\" (UID: \"f88e5323-c309-4bdb-ae95-359b2a92061b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" Jun 11 21:15:43.994214 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994110 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f88e5323-c309-4bdb-ae95-359b2a92061b-device-dir\") pod \"aws-ebs-csi-driver-node-kc22x\" (UID: \"f88e5323-c309-4bdb-ae95-359b2a92061b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" Jun 11 21:15:43.994214 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994137 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bdfaf7d3-ae13-48e9-b9ea-040ae02d886a-hosts-file\") pod \"node-resolver-s8b5p\" (UID: \"bdfaf7d3-ae13-48e9-b9ea-040ae02d886a\") " pod="openshift-dns/node-resolver-s8b5p" Jun 11 21:15:43.994214 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994157 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-host-cni-bin\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:43.994214 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994181 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c7849da4-f326-4b8c-a436-5b0d68958170-ovn-node-metrics-cert\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:43.994214 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994204 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5678560c-39b0-4892-a879-3613ac253097-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w26g9\" (UID: \"5678560c-39b0-4892-a879-3613ac253097\") " pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:43.994752 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994227 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-host-run-netns\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:43.994752 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994250 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-host-cni-netd\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:43.994752 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994352 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c7849da4-f326-4b8c-a436-5b0d68958170-ovnkube-script-lib\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:43.994752 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994380 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5678560c-39b0-4892-a879-3613ac253097-os-release\") pod \"multus-additional-cni-plugins-w26g9\" (UID: \"5678560c-39b0-4892-a879-3613ac253097\") " pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:43.994752 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994415 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f88e5323-c309-4bdb-ae95-359b2a92061b-etc-selinux\") pod \"aws-ebs-csi-driver-node-kc22x\" (UID: \"f88e5323-c309-4bdb-ae95-359b2a92061b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" Jun 11 21:15:43.994752 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994452 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f88e5323-c309-4bdb-ae95-359b2a92061b-sys-fs\") pod \"aws-ebs-csi-driver-node-kc22x\" (UID: \"f88e5323-c309-4bdb-ae95-359b2a92061b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" Jun 11 21:15:43.994752 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994353 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 11 21:15:43.994752 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994525 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-host-run-ovn-kubernetes\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:43.994752 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994581 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-var-lib-openvswitch\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:43.994752 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994610 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-node-log\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:43.994752 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994639 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-log-socket\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:43.994752 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994673 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5678560c-39b0-4892-a879-3613ac253097-cnibin\") pod \"multus-additional-cni-plugins-w26g9\" (UID: \"5678560c-39b0-4892-a879-3613ac253097\") " pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:43.994752 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994702 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5678560c-39b0-4892-a879-3613ac253097-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w26g9\" (UID: \"5678560c-39b0-4892-a879-3613ac253097\") " pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:43.994752 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994745 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5678560c-39b0-4892-a879-3613ac253097-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w26g9\" (UID: \"5678560c-39b0-4892-a879-3613ac253097\") " pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:43.995393 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994791 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdvsw\" (UniqueName: \"kubernetes.io/projected/bdfaf7d3-ae13-48e9-b9ea-040ae02d886a-kube-api-access-fdvsw\") pod \"node-resolver-s8b5p\" (UID: \"bdfaf7d3-ae13-48e9-b9ea-040ae02d886a\") " pod="openshift-dns/node-resolver-s8b5p" Jun 11 21:15:43.995393 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994867 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2xdc\" (UniqueName: \"kubernetes.io/projected/54688996-c5d4-445d-a6ff-4997a19fe1d6-kube-api-access-g2xdc\") pod \"network-check-target-shxrd\" (UID: \"54688996-c5d4-445d-a6ff-4997a19fe1d6\") " pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:15:43.995393 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994909 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-host-kubelet\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:43.995393 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.994977 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-systemd-units\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:43.995393 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.995123 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66nnp\" (UniqueName: \"kubernetes.io/projected/c7849da4-f326-4b8c-a436-5b0d68958170-kube-api-access-66nnp\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:43.995393 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.995151 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hs5z\" (UniqueName: \"kubernetes.io/projected/7b88bd69-b4df-4345-a24c-e51adeba9023-kube-api-access-5hs5z\") pod \"node-ca-thgw8\" (UID: \"7b88bd69-b4df-4345-a24c-e51adeba9023\") " pod="openshift-image-registry/node-ca-thgw8" Jun 11 21:15:43.995393 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.995176 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7c5p\" (UniqueName: \"kubernetes.io/projected/f88e5323-c309-4bdb-ae95-359b2a92061b-kube-api-access-f7c5p\") pod \"aws-ebs-csi-driver-node-kc22x\" (UID: \"f88e5323-c309-4bdb-ae95-359b2a92061b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" Jun 11 21:15:43.995393 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.995202 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-host-slash\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:43.995393 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.995224 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-run-systemd\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:43.995393 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.995263 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-etc-openvswitch\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:43.995393 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.995300 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-run-ovn\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:43.995909 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.995648 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wmhjx\"" Jun 11 21:15:43.995909 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.995753 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 11 21:15:43.995909 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.995881 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 11 21:15:43.996175 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.996157 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:15:43.996270 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:43.996218 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bmhl" podUID="329e0949-4197-4a29-afd8-1300e76b75cf" Jun 11 21:15:43.996344 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.996276 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 11 21:15:43.996415 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:43.996358 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lmmct\"" Jun 11 21:15:44.015776 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.015750 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-v982x" Jun 11 21:15:44.023308 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.023062 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-v982x" Jun 11 21:15:44.083636 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.083609 2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 11 21:15:44.096361 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096324 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5678560c-39b0-4892-a879-3613ac253097-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w26g9\" (UID: \"5678560c-39b0-4892-a879-3613ac253097\") " pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:44.096529 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096373 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-etc-kubernetes\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.096529 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096400 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6f4a1fd8-505b-4061-a432-620cb3c68386-kubelet-config\") pod \"global-pull-secret-syncer-4c247\" (UID: \"6f4a1fd8-505b-4061-a432-620cb3c68386\") " pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:15:44.096529 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096421 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b641e893-d298-4942-8ccb-6145ef84bf81-agent-certs\") pod \"konnectivity-agent-x65dp\" (UID: \"b641e893-d298-4942-8ccb-6145ef84bf81\") " pod="kube-system/konnectivity-agent-x65dp" Jun 11 21:15:44.096529 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096443 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f88e5323-c309-4bdb-ae95-359b2a92061b-etc-selinux\") pod \"aws-ebs-csi-driver-node-kc22x\" (UID: \"f88e5323-c309-4bdb-ae95-359b2a92061b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" Jun 11 21:15:44.096529 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096488 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f88e5323-c309-4bdb-ae95-359b2a92061b-sys-fs\") pod \"aws-ebs-csi-driver-node-kc22x\" (UID: \"f88e5323-c309-4bdb-ae95-359b2a92061b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" Jun 11 21:15:44.096529 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096521 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-hostroot\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.096842 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096543 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/275b0bd4-429e-4171-b5f8-70a3bb42f08f-etc-tuned\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.096842 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096585 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-host-run-ovn-kubernetes\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.096842 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096592 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f88e5323-c309-4bdb-ae95-359b2a92061b-sys-fs\") pod \"aws-ebs-csi-driver-node-kc22x\" (UID: \"f88e5323-c309-4bdb-ae95-359b2a92061b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" Jun 11 21:15:44.096842 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096610 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-var-lib-openvswitch\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.096842 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096619 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f88e5323-c309-4bdb-ae95-359b2a92061b-etc-selinux\") pod \"aws-ebs-csi-driver-node-kc22x\" (UID: \"f88e5323-c309-4bdb-ae95-359b2a92061b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" Jun 11 21:15:44.096842 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096637 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-host-run-ovn-kubernetes\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.096842 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096632 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-node-log\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.096842 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096680 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-node-log\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.096842 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096692 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-var-lib-openvswitch\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.096842 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096740 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-log-socket\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.096842 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096766 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5678560c-39b0-4892-a879-3613ac253097-cnibin\") pod \"multus-additional-cni-plugins-w26g9\" (UID: \"5678560c-39b0-4892-a879-3613ac253097\") " pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:44.096842 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096775 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-log-socket\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.096842 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096826 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5678560c-39b0-4892-a879-3613ac253097-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w26g9\" (UID: \"5678560c-39b0-4892-a879-3613ac253097\") " pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:44.097500 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096854 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5678560c-39b0-4892-a879-3613ac253097-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w26g9\" (UID: \"5678560c-39b0-4892-a879-3613ac253097\") " pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:44.097500 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096882 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-etc-systemd\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.097500 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096891 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5678560c-39b0-4892-a879-3613ac253097-cnibin\") pod \"multus-additional-cni-plugins-w26g9\" (UID: \"5678560c-39b0-4892-a879-3613ac253097\") " pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:44.097500 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096907 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-host-kubelet\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.097500 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096933 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-multus-daemon-config\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.097500 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096960 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-host-slash\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.097500 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.096984 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-run-systemd\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.097500 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.097010 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f88e5323-c309-4bdb-ae95-359b2a92061b-registration-dir\") pod \"aws-ebs-csi-driver-node-kc22x\" (UID: \"f88e5323-c309-4bdb-ae95-359b2a92061b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" Jun 11 21:15:44.097500 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.097019 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5678560c-39b0-4892-a879-3613ac253097-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w26g9\" (UID: \"5678560c-39b0-4892-a879-3613ac253097\") " pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:44.097500 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.097034 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-multus-cni-dir\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.097500 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.097071 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whglr\" (UniqueName: \"kubernetes.io/projected/275b0bd4-429e-4171-b5f8-70a3bb42f08f-kube-api-access-whglr\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.097500 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.097075 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5678560c-39b0-4892-a879-3613ac253097-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w26g9\" (UID: \"5678560c-39b0-4892-a879-3613ac253097\") " pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:44.097500 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.097090 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-host-slash\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.097500 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.097119 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c7849da4-f326-4b8c-a436-5b0d68958170-ovnkube-config\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.097500 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.097131 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-run-systemd\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.097500 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.097178 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-os-release\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.097500 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.097232 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-multus-socket-dir-parent\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.098294 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.097237 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-host-kubelet\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.098294 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.097181 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f88e5323-c309-4bdb-ae95-359b2a92061b-registration-dir\") pod \"aws-ebs-csi-driver-node-kc22x\" (UID: \"f88e5323-c309-4bdb-ae95-359b2a92061b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" Jun 11 21:15:44.098294 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.097272 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5sgz\" (UniqueName: \"kubernetes.io/projected/329e0949-4197-4a29-afd8-1300e76b75cf-kube-api-access-q5sgz\") pod \"network-metrics-daemon-2bmhl\" (UID: \"329e0949-4197-4a29-afd8-1300e76b75cf\") " pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:15:44.098294 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.097302 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c7849da4-f326-4b8c-a436-5b0d68958170-env-overrides\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.098294 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.097364 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-run-openvswitch\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.098294 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.097419 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7b88bd69-b4df-4345-a24c-e51adeba9023-serviceca\") pod \"node-ca-thgw8\" (UID: \"7b88bd69-b4df-4345-a24c-e51adeba9023\") " pod="openshift-image-registry/node-ca-thgw8" Jun 11 21:15:44.098294 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.097445 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f88e5323-c309-4bdb-ae95-359b2a92061b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kc22x\" (UID: \"f88e5323-c309-4bdb-ae95-359b2a92061b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" Jun 11 21:15:44.098294 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.097459 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5678560c-39b0-4892-a879-3613ac253097-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w26g9\" (UID: \"5678560c-39b0-4892-a879-3613ac253097\") " pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:44.098294 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.097465 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-run-openvswitch\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.098294 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.097477 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f88e5323-c309-4bdb-ae95-359b2a92061b-device-dir\") pod \"aws-ebs-csi-driver-node-kc22x\" (UID: \"f88e5323-c309-4bdb-ae95-359b2a92061b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" Jun 11 21:15:44.098294 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.097510 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-etc-sysconfig\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.098294 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.098125 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c7849da4-f326-4b8c-a436-5b0d68958170-ovnkube-config\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.098294 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.098236 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f88e5323-c309-4bdb-ae95-359b2a92061b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kc22x\" (UID: \"f88e5323-c309-4bdb-ae95-359b2a92061b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" Jun 11 21:15:44.098911 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.098336 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f88e5323-c309-4bdb-ae95-359b2a92061b-device-dir\") pod \"aws-ebs-csi-driver-node-kc22x\" (UID: \"f88e5323-c309-4bdb-ae95-359b2a92061b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" Jun 11 21:15:44.098911 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.098497 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-run\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.098911 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.098579 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdfaf7d3-ae13-48e9-b9ea-040ae02d886a-tmp-dir\") pod \"node-resolver-s8b5p\" (UID: \"bdfaf7d3-ae13-48e9-b9ea-040ae02d886a\") " pod="openshift-dns/node-resolver-s8b5p" Jun 11 21:15:44.098911 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.098616 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs\") pod \"network-metrics-daemon-2bmhl\" (UID: \"329e0949-4197-4a29-afd8-1300e76b75cf\") " pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:15:44.098911 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.098669 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-etc-sysctl-d\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.098911 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.098738 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-var-lib-kubelet\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.098911 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.098783 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7b88bd69-b4df-4345-a24c-e51adeba9023-serviceca\") pod \"node-ca-thgw8\" (UID: \"7b88bd69-b4df-4345-a24c-e51adeba9023\") " pod="openshift-image-registry/node-ca-thgw8" Jun 11 21:15:44.098911 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.098795 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-host-run-netns\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.099324 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.099126 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c7849da4-f326-4b8c-a436-5b0d68958170-env-overrides\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.099324 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.099144 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdfaf7d3-ae13-48e9-b9ea-040ae02d886a-tmp-dir\") pod \"node-resolver-s8b5p\" (UID: \"bdfaf7d3-ae13-48e9-b9ea-040ae02d886a\") " pod="openshift-dns/node-resolver-s8b5p" Jun 11 21:15:44.099687 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.099587 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-host-cni-netd\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.099687 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.099632 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c7849da4-f326-4b8c-a436-5b0d68958170-ovnkube-script-lib\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.099687 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.099669 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5678560c-39b0-4892-a879-3613ac253097-os-release\") pod \"multus-additional-cni-plugins-w26g9\" (UID: \"5678560c-39b0-4892-a879-3613ac253097\") " pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:44.099875 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.099701 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-host-var-lib-cni-multus\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.099875 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.099736 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-host-run-multus-certs\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.099875 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.099820 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-etc-kubernetes\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.099875 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.099857 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-etc-sysctl-conf\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.100065 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.099885 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-host-run-netns\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.100065 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.099922 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-cni-binary-copy\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.100065 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.099959 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f4a1fd8-505b-4061-a432-620cb3c68386-original-pull-secret\") pod \"global-pull-secret-syncer-4c247\" (UID: \"6f4a1fd8-505b-4061-a432-620cb3c68386\") " pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:15:44.100065 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.099992 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-lib-modules\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.100065 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.100021 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/275b0bd4-429e-4171-b5f8-70a3bb42f08f-tmp\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.100065 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.100053 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdvsw\" (UniqueName: \"kubernetes.io/projected/bdfaf7d3-ae13-48e9-b9ea-040ae02d886a-kube-api-access-fdvsw\") pod \"node-resolver-s8b5p\" (UID: \"bdfaf7d3-ae13-48e9-b9ea-040ae02d886a\") " pod="openshift-dns/node-resolver-s8b5p" Jun 11 21:15:44.100334 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.100091 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2xdc\" (UniqueName: \"kubernetes.io/projected/54688996-c5d4-445d-a6ff-4997a19fe1d6-kube-api-access-g2xdc\") pod \"network-check-target-shxrd\" (UID: \"54688996-c5d4-445d-a6ff-4997a19fe1d6\") " pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:15:44.100334 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.100132 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-systemd-units\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.100334 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.100241 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66nnp\" (UniqueName: \"kubernetes.io/projected/c7849da4-f326-4b8c-a436-5b0d68958170-kube-api-access-66nnp\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.100334 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.100281 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hs5z\" (UniqueName: \"kubernetes.io/projected/7b88bd69-b4df-4345-a24c-e51adeba9023-kube-api-access-5hs5z\") pod \"node-ca-thgw8\" (UID: \"7b88bd69-b4df-4345-a24c-e51adeba9023\") " pod="openshift-image-registry/node-ca-thgw8" Jun 11 21:15:44.100334 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.100329 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7c5p\" (UniqueName: \"kubernetes.io/projected/f88e5323-c309-4bdb-ae95-359b2a92061b-kube-api-access-f7c5p\") pod \"aws-ebs-csi-driver-node-kc22x\" (UID: \"f88e5323-c309-4bdb-ae95-359b2a92061b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" Jun 11 21:15:44.100559 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.100362 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-system-cni-dir\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.100559 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.100426 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-host-run-k8s-cni-cncf-io\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.100559 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.100473 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-etc-openvswitch\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.100559 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.100502 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-run-ovn\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.100559 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.100547 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.100816 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.100591 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-run-ovn\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.100816 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.100606 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b88bd69-b4df-4345-a24c-e51adeba9023-host\") pod \"node-ca-thgw8\" (UID: \"7b88bd69-b4df-4345-a24c-e51adeba9023\") " pod="openshift-image-registry/node-ca-thgw8" Jun 11 21:15:44.100816 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.100668 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b88bd69-b4df-4345-a24c-e51adeba9023-host\") pod \"node-ca-thgw8\" (UID: \"7b88bd69-b4df-4345-a24c-e51adeba9023\") " pod="openshift-image-registry/node-ca-thgw8" Jun 11 21:15:44.100816 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.100723 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-systemd-units\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.100816 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.100777 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-host-run-netns\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.101929 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.101356 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c7849da4-f326-4b8c-a436-5b0d68958170-ovnkube-script-lib\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.101929 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.101440 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-host-cni-netd\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.101929 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.101455 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5678560c-39b0-4892-a879-3613ac253097-system-cni-dir\") pod \"multus-additional-cni-plugins-w26g9\" (UID: \"5678560c-39b0-4892-a879-3613ac253097\") " pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:44.101929 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.101494 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.101929 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.101505 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7krj\" (UniqueName: \"kubernetes.io/projected/5678560c-39b0-4892-a879-3613ac253097-kube-api-access-c7krj\") pod \"multus-additional-cni-plugins-w26g9\" (UID: \"5678560c-39b0-4892-a879-3613ac253097\") " pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:44.101929 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.101513 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5678560c-39b0-4892-a879-3613ac253097-os-release\") pod \"multus-additional-cni-plugins-w26g9\" (UID: \"5678560c-39b0-4892-a879-3613ac253097\") " pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:44.101929 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.101638 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-etc-openvswitch\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.101929 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.101713 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5678560c-39b0-4892-a879-3613ac253097-system-cni-dir\") pod \"multus-additional-cni-plugins-w26g9\" (UID: \"5678560c-39b0-4892-a879-3613ac253097\") " pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:44.102428 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.101958 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-cnibin\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.102428 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.102021 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-host-var-lib-kubelet\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.102428 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.102140 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-multus-conf-dir\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.102428 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.102217 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg7ww\" (UniqueName: \"kubernetes.io/projected/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-kube-api-access-lg7ww\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.102428 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.102282 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-etc-modprobe-d\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.102428 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.102319 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-host\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.102428 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.102376 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/71fed145-1fbc-4433-bfb0-394c5631bec6-iptables-alerter-script\") pod \"iptables-alerter-cqksf\" (UID: \"71fed145-1fbc-4433-bfb0-394c5631bec6\") " pod="openshift-network-operator/iptables-alerter-cqksf" Jun 11 21:15:44.102779 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.102448 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/71fed145-1fbc-4433-bfb0-394c5631bec6-host-slash\") pod \"iptables-alerter-cqksf\" (UID: \"71fed145-1fbc-4433-bfb0-394c5631bec6\") " pod="openshift-network-operator/iptables-alerter-cqksf" Jun 11 21:15:44.102779 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.102488 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-host-var-lib-cni-bin\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.102779 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.102515 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6f4a1fd8-505b-4061-a432-620cb3c68386-dbus\") pod \"global-pull-secret-syncer-4c247\" (UID: \"6f4a1fd8-505b-4061-a432-620cb3c68386\") " pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:15:44.102779 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.102578 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/71fed145-1fbc-4433-bfb0-394c5631bec6-host-slash\") pod \"iptables-alerter-cqksf\" (UID: \"71fed145-1fbc-4433-bfb0-394c5631bec6\") " pod="openshift-network-operator/iptables-alerter-cqksf" Jun 11 21:15:44.102779 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.102655 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b641e893-d298-4942-8ccb-6145ef84bf81-konnectivity-ca\") pod \"konnectivity-agent-x65dp\" (UID: \"b641e893-d298-4942-8ccb-6145ef84bf81\") " pod="kube-system/konnectivity-agent-x65dp" Jun 11 21:15:44.102779 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.102678 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zplqm\" (UniqueName: \"kubernetes.io/projected/71fed145-1fbc-4433-bfb0-394c5631bec6-kube-api-access-zplqm\") pod \"iptables-alerter-cqksf\" (UID: \"71fed145-1fbc-4433-bfb0-394c5631bec6\") " pod="openshift-network-operator/iptables-alerter-cqksf" Jun 11 21:15:44.103207 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.103159 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5678560c-39b0-4892-a879-3613ac253097-cni-binary-copy\") pod \"multus-additional-cni-plugins-w26g9\" (UID: \"5678560c-39b0-4892-a879-3613ac253097\") " pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:44.103340 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.103235 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f88e5323-c309-4bdb-ae95-359b2a92061b-socket-dir\") pod \"aws-ebs-csi-driver-node-kc22x\" (UID: \"f88e5323-c309-4bdb-ae95-359b2a92061b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" Jun 11 21:15:44.103340 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.103264 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-sys\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.103340 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.103289 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bdfaf7d3-ae13-48e9-b9ea-040ae02d886a-hosts-file\") pod \"node-resolver-s8b5p\" (UID: \"bdfaf7d3-ae13-48e9-b9ea-040ae02d886a\") " pod="openshift-dns/node-resolver-s8b5p" Jun 11 21:15:44.103340 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.103312 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-host-cni-bin\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.103496 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.103343 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c7849da4-f326-4b8c-a436-5b0d68958170-ovn-node-metrics-cert\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.103532 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.103495 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f88e5323-c309-4bdb-ae95-359b2a92061b-socket-dir\") pod \"aws-ebs-csi-driver-node-kc22x\" (UID: \"f88e5323-c309-4bdb-ae95-359b2a92061b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" Jun 11 21:15:44.103595 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.103555 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bdfaf7d3-ae13-48e9-b9ea-040ae02d886a-hosts-file\") pod \"node-resolver-s8b5p\" (UID: \"bdfaf7d3-ae13-48e9-b9ea-040ae02d886a\") " pod="openshift-dns/node-resolver-s8b5p" Jun 11 21:15:44.103595 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.103556 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/71fed145-1fbc-4433-bfb0-394c5631bec6-iptables-alerter-script\") pod \"iptables-alerter-cqksf\" (UID: \"71fed145-1fbc-4433-bfb0-394c5631bec6\") " pod="openshift-network-operator/iptables-alerter-cqksf" Jun 11 21:15:44.103677 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.103620 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c7849da4-f326-4b8c-a436-5b0d68958170-host-cni-bin\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.103677 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.103652 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5678560c-39b0-4892-a879-3613ac253097-cni-binary-copy\") pod \"multus-additional-cni-plugins-w26g9\" (UID: \"5678560c-39b0-4892-a879-3613ac253097\") " pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:44.103753 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.103720 2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 11 21:15:44.106469 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:44.106436 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 21:15:44.106469 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:44.106465 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 21:15:44.106646 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:44.106478 2570 projected.go:196] Error preparing data for projected volume kube-api-access-g2xdc for pod openshift-network-diagnostics/network-check-target-shxrd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:15:44.106646 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:44.106553 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54688996-c5d4-445d-a6ff-4997a19fe1d6-kube-api-access-g2xdc podName:54688996-c5d4-445d-a6ff-4997a19fe1d6 nodeName:}" failed. No retries permitted until 2026-06-11 21:15:44.606533777 +0000 UTC m=+3.017857308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-g2xdc" (UniqueName: "kubernetes.io/projected/54688996-c5d4-445d-a6ff-4997a19fe1d6-kube-api-access-g2xdc") pod "network-check-target-shxrd" (UID: "54688996-c5d4-445d-a6ff-4997a19fe1d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:15:44.107551 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.107527 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c7849da4-f326-4b8c-a436-5b0d68958170-ovn-node-metrics-cert\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.108804 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.108772 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hs5z\" (UniqueName: \"kubernetes.io/projected/7b88bd69-b4df-4345-a24c-e51adeba9023-kube-api-access-5hs5z\") pod \"node-ca-thgw8\" (UID: \"7b88bd69-b4df-4345-a24c-e51adeba9023\") " pod="openshift-image-registry/node-ca-thgw8" Jun 11 21:15:44.109219 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.109165 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdvsw\" (UniqueName: \"kubernetes.io/projected/bdfaf7d3-ae13-48e9-b9ea-040ae02d886a-kube-api-access-fdvsw\") pod \"node-resolver-s8b5p\" (UID: \"bdfaf7d3-ae13-48e9-b9ea-040ae02d886a\") " pod="openshift-dns/node-resolver-s8b5p" Jun 11 21:15:44.109506 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.109485 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66nnp\" (UniqueName: \"kubernetes.io/projected/c7849da4-f326-4b8c-a436-5b0d68958170-kube-api-access-66nnp\") pod \"ovnkube-node-b4gmk\" (UID: \"c7849da4-f326-4b8c-a436-5b0d68958170\") " pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.109590 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.109501 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7c5p\" (UniqueName: \"kubernetes.io/projected/f88e5323-c309-4bdb-ae95-359b2a92061b-kube-api-access-f7c5p\") pod \"aws-ebs-csi-driver-node-kc22x\" (UID: \"f88e5323-c309-4bdb-ae95-359b2a92061b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" Jun 11 21:15:44.110002 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.109973 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7krj\" (UniqueName: \"kubernetes.io/projected/5678560c-39b0-4892-a879-3613ac253097-kube-api-access-c7krj\") pod \"multus-additional-cni-plugins-w26g9\" (UID: \"5678560c-39b0-4892-a879-3613ac253097\") " pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:44.110704 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.110687 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zplqm\" (UniqueName: \"kubernetes.io/projected/71fed145-1fbc-4433-bfb0-394c5631bec6-kube-api-access-zplqm\") pod \"iptables-alerter-cqksf\" (UID: \"71fed145-1fbc-4433-bfb0-394c5631bec6\") " pod="openshift-network-operator/iptables-alerter-cqksf" Jun 11 21:15:44.204229 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204140 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-etc-modprobe-d\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.204229 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204188 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-host\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.204229 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204224 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-host-var-lib-cni-bin\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.204499 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204244 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6f4a1fd8-505b-4061-a432-620cb3c68386-dbus\") pod \"global-pull-secret-syncer-4c247\" (UID: \"6f4a1fd8-505b-4061-a432-620cb3c68386\") " pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:15:44.204499 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204267 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b641e893-d298-4942-8ccb-6145ef84bf81-konnectivity-ca\") pod \"konnectivity-agent-x65dp\" (UID: \"b641e893-d298-4942-8ccb-6145ef84bf81\") " pod="kube-system/konnectivity-agent-x65dp" Jun 11 21:15:44.204499 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204293 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-sys\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.204499 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204318 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-etc-kubernetes\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.204499 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204331 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-host-var-lib-cni-bin\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.204499 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204343 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6f4a1fd8-505b-4061-a432-620cb3c68386-kubelet-config\") pod \"global-pull-secret-syncer-4c247\" (UID: \"6f4a1fd8-505b-4061-a432-620cb3c68386\") " pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:15:44.204499 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204349 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-etc-modprobe-d\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.204499 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204368 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6f4a1fd8-505b-4061-a432-620cb3c68386-dbus\") pod \"global-pull-secret-syncer-4c247\" (UID: \"6f4a1fd8-505b-4061-a432-620cb3c68386\") " pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:15:44.204499 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204393 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6f4a1fd8-505b-4061-a432-620cb3c68386-kubelet-config\") pod \"global-pull-secret-syncer-4c247\" (UID: \"6f4a1fd8-505b-4061-a432-620cb3c68386\") " pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:15:44.204499 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204407 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-host\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.204499 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204375 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b641e893-d298-4942-8ccb-6145ef84bf81-agent-certs\") pod \"konnectivity-agent-x65dp\" (UID: \"b641e893-d298-4942-8ccb-6145ef84bf81\") " pod="kube-system/konnectivity-agent-x65dp" Jun 11 21:15:44.204499 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204438 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-sys\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.204499 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204451 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-hostroot\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.204499 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204488 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-etc-kubernetes\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.204499 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204492 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-hostroot\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.204499 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204510 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/275b0bd4-429e-4171-b5f8-70a3bb42f08f-etc-tuned\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.205239 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204541 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-etc-systemd\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.205239 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204585 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-multus-daemon-config\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.205239 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204613 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-multus-cni-dir\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.205239 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204637 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whglr\" (UniqueName: \"kubernetes.io/projected/275b0bd4-429e-4171-b5f8-70a3bb42f08f-kube-api-access-whglr\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.205239 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204661 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-os-release\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.205239 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204686 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-multus-socket-dir-parent\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.205239 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204729 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5sgz\" (UniqueName: \"kubernetes.io/projected/329e0949-4197-4a29-afd8-1300e76b75cf-kube-api-access-q5sgz\") pod \"network-metrics-daemon-2bmhl\" (UID: \"329e0949-4197-4a29-afd8-1300e76b75cf\") " pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:15:44.205239 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204761 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-etc-sysconfig\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.205239 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204786 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-run\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.205239 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204791 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-etc-systemd\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.205239 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204785 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-multus-cni-dir\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.205239 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204813 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs\") pod \"network-metrics-daemon-2bmhl\" (UID: \"329e0949-4197-4a29-afd8-1300e76b75cf\") " pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:15:44.205239 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204851 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-etc-sysctl-d\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.205239 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204878 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-var-lib-kubelet\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.205239 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204903 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-multus-socket-dir-parent\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.205239 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:44.204907 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:15:44.205239 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204959 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-host-var-lib-cni-multus\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.205239 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204971 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-etc-sysconfig\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.206066 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204980 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b641e893-d298-4942-8ccb-6145ef84bf81-konnectivity-ca\") pod \"konnectivity-agent-x65dp\" (UID: \"b641e893-d298-4942-8ccb-6145ef84bf81\") " pod="kube-system/konnectivity-agent-x65dp" Jun 11 21:15:44.206066 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:44.205001 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs podName:329e0949-4197-4a29-afd8-1300e76b75cf nodeName:}" failed. No retries permitted until 2026-06-11 21:15:44.704980853 +0000 UTC m=+3.116304396 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs") pod "network-metrics-daemon-2bmhl" (UID: "329e0949-4197-4a29-afd8-1300e76b75cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:15:44.206066 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.204921 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-host-var-lib-cni-multus\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.206066 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205042 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-host-run-multus-certs\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.206066 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205078 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-etc-kubernetes\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.206066 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205139 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-etc-sysctl-conf\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.206066 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205166 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-host-run-netns\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.206066 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205176 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-run\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.206066 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205190 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-cni-binary-copy\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.206066 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205217 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f4a1fd8-505b-4061-a432-620cb3c68386-original-pull-secret\") pod \"global-pull-secret-syncer-4c247\" (UID: \"6f4a1fd8-505b-4061-a432-620cb3c68386\") " pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:15:44.206066 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205241 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-lib-modules\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.206066 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205251 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-os-release\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.206066 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205261 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/275b0bd4-429e-4171-b5f8-70a3bb42f08f-tmp\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.206066 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205299 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-var-lib-kubelet\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.206066 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205138 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-etc-sysctl-d\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.206066 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205330 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-system-cni-dir\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.206066 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205354 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-host-run-multus-certs\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.206887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205358 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-host-run-k8s-cni-cncf-io\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.206887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205394 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-cnibin\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.206887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205401 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-etc-kubernetes\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.206887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205418 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-host-var-lib-kubelet\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.206887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205449 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-multus-conf-dir\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.206887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205475 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg7ww\" (UniqueName: \"kubernetes.io/projected/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-kube-api-access-lg7ww\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.206887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205488 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-host-run-netns\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.206887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205537 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-etc-sysctl-conf\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.206887 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:44.205608 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 21:15:44.206887 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:44.205656 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f4a1fd8-505b-4061-a432-620cb3c68386-original-pull-secret podName:6f4a1fd8-505b-4061-a432-620cb3c68386 nodeName:}" failed. No retries permitted until 2026-06-11 21:15:44.705639995 +0000 UTC m=+3.116963515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f4a1fd8-505b-4061-a432-620cb3c68386-original-pull-secret") pod "global-pull-secret-syncer-4c247" (UID: "6f4a1fd8-505b-4061-a432-620cb3c68386") : object "kube-system"/"original-pull-secret" not registered Jun 11 21:15:44.206887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205664 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-multus-daemon-config\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.206887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205741 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-cnibin\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.206887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205774 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/275b0bd4-429e-4171-b5f8-70a3bb42f08f-lib-modules\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.206887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205789 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-system-cni-dir\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.206887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205828 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-host-run-k8s-cni-cncf-io\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.206887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205819 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-multus-conf-dir\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.206887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205870 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-host-var-lib-kubelet\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.206887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.205938 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-cni-binary-copy\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.207807 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.207001 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/275b0bd4-429e-4171-b5f8-70a3bb42f08f-etc-tuned\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.207807 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.207124 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b641e893-d298-4942-8ccb-6145ef84bf81-agent-certs\") pod \"konnectivity-agent-x65dp\" (UID: \"b641e893-d298-4942-8ccb-6145ef84bf81\") " pod="kube-system/konnectivity-agent-x65dp" Jun 11 21:15:44.207807 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.207681 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/275b0bd4-429e-4171-b5f8-70a3bb42f08f-tmp\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.218981 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.218953 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg7ww\" (UniqueName: \"kubernetes.io/projected/9094ab8c-8937-44bb-9f4e-ed8445c3c55f-kube-api-access-lg7ww\") pod \"multus-bl9g7\" (UID: \"9094ab8c-8937-44bb-9f4e-ed8445c3c55f\") " pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.219204 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.219182 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whglr\" (UniqueName: \"kubernetes.io/projected/275b0bd4-429e-4171-b5f8-70a3bb42f08f-kube-api-access-whglr\") pod \"tuned-fhhkw\" (UID: \"275b0bd4-429e-4171-b5f8-70a3bb42f08f\") " pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.219392 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.219371 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5sgz\" (UniqueName: \"kubernetes.io/projected/329e0949-4197-4a29-afd8-1300e76b75cf-kube-api-access-q5sgz\") pod \"network-metrics-daemon-2bmhl\" (UID: \"329e0949-4197-4a29-afd8-1300e76b75cf\") " pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:15:44.292426 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.292390 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cqksf" Jun 11 21:15:44.300361 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.300334 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:15:44.308111 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.308090 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" Jun 11 21:15:44.315718 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.315695 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-thgw8" Jun 11 21:15:44.322295 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.322275 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-w26g9" Jun 11 21:15:44.330718 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.330688 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s8b5p" Jun 11 21:15:44.338353 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.338333 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x65dp" Jun 11 21:15:44.344054 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.344035 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" Jun 11 21:15:44.349609 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.349590 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bl9g7" Jun 11 21:15:44.607739 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.607645 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2xdc\" (UniqueName: \"kubernetes.io/projected/54688996-c5d4-445d-a6ff-4997a19fe1d6-kube-api-access-g2xdc\") pod \"network-check-target-shxrd\" (UID: \"54688996-c5d4-445d-a6ff-4997a19fe1d6\") " pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:15:44.607899 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:44.607844 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 21:15:44.607899 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:44.607872 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 21:15:44.607899 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:44.607887 2570 projected.go:196] Error preparing data for projected volume kube-api-access-g2xdc for pod openshift-network-diagnostics/network-check-target-shxrd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:15:44.608046 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:44.607986 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54688996-c5d4-445d-a6ff-4997a19fe1d6-kube-api-access-g2xdc podName:54688996-c5d4-445d-a6ff-4997a19fe1d6 nodeName:}" failed. No retries permitted until 2026-06-11 21:15:45.607964643 +0000 UTC m=+4.019288197 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-g2xdc" (UniqueName: "kubernetes.io/projected/54688996-c5d4-445d-a6ff-4997a19fe1d6-kube-api-access-g2xdc") pod "network-check-target-shxrd" (UID: "54688996-c5d4-445d-a6ff-4997a19fe1d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:15:44.708608 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.708557 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs\") pod \"network-metrics-daemon-2bmhl\" (UID: \"329e0949-4197-4a29-afd8-1300e76b75cf\") " pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:15:44.708764 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:44.708628 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f4a1fd8-505b-4061-a432-620cb3c68386-original-pull-secret\") pod \"global-pull-secret-syncer-4c247\" (UID: \"6f4a1fd8-505b-4061-a432-620cb3c68386\") " pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:15:44.708764 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:44.708698 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:15:44.708764 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:44.708739 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 21:15:44.708863 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:44.708788 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs podName:329e0949-4197-4a29-afd8-1300e76b75cf nodeName:}" failed. No retries permitted until 2026-06-11 21:15:45.708764307 +0000 UTC m=+4.120087824 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs") pod "network-metrics-daemon-2bmhl" (UID: "329e0949-4197-4a29-afd8-1300e76b75cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:15:44.708863 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:44.708815 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f4a1fd8-505b-4061-a432-620cb3c68386-original-pull-secret podName:6f4a1fd8-505b-4061-a432-620cb3c68386 nodeName:}" failed. No retries permitted until 2026-06-11 21:15:45.708800915 +0000 UTC m=+4.120124432 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f4a1fd8-505b-4061-a432-620cb3c68386-original-pull-secret") pod "global-pull-secret-syncer-4c247" (UID: "6f4a1fd8-505b-4061-a432-620cb3c68386") : object "kube-system"/"original-pull-secret" not registered Jun 11 21:15:45.017068 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:45.017036 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5678560c_39b0_4892_a879_3613ac253097.slice/crio-ee7ff222021f90d2d2d73011975ff45f93a54c909f916304830d04e7015a3589 WatchSource:0}: Error finding container ee7ff222021f90d2d2d73011975ff45f93a54c909f916304830d04e7015a3589: Status 404 returned error can't find the container with id ee7ff222021f90d2d2d73011975ff45f93a54c909f916304830d04e7015a3589 Jun 11 21:15:45.018020 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:45.017996 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9094ab8c_8937_44bb_9f4e_ed8445c3c55f.slice/crio-2e35646cdabc4149f573dd9e59d334a3ff09a408888bd9c28b5e62d52c71fb55 WatchSource:0}: Error finding container 2e35646cdabc4149f573dd9e59d334a3ff09a408888bd9c28b5e62d52c71fb55: Status 404 returned error can't find the container with id 2e35646cdabc4149f573dd9e59d334a3ff09a408888bd9c28b5e62d52c71fb55 Jun 11 21:15:45.019060 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:45.019025 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b88bd69_b4df_4345_a24c_e51adeba9023.slice/crio-ee32d9b85fda173e5d4528b95d4c2234c1d5277bb2194c8ac11f0df0908af891 WatchSource:0}: Error finding container ee32d9b85fda173e5d4528b95d4c2234c1d5277bb2194c8ac11f0df0908af891: Status 404 returned error can't find the container with id ee32d9b85fda173e5d4528b95d4c2234c1d5277bb2194c8ac11f0df0908af891 Jun 11 21:15:45.021230 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:45.021203 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7849da4_f326_4b8c_a436_5b0d68958170.slice/crio-f796989e1748cc5b2442182b3fef612e825446e57ffe268eb879b47fb8f55413 WatchSource:0}: Error finding container f796989e1748cc5b2442182b3fef612e825446e57ffe268eb879b47fb8f55413: Status 404 returned error can't find the container with id f796989e1748cc5b2442182b3fef612e825446e57ffe268eb879b47fb8f55413 Jun 11 21:15:45.025127 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:45.025012 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-10 21:10:44 +0000 UTC" deadline="2027-11-21 00:27:20.170491469 +0000 UTC" Jun 11 21:15:45.025127 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:45.025040 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12651h11m35.145456169s" Jun 11 21:15:45.025127 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:45.025106 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71fed145_1fbc_4433_bfb0_394c5631bec6.slice/crio-3c7c8c80b8517b61b0acbf5bee9a13187596f9c333a4fa5edc5d3119ac78c06d WatchSource:0}: Error finding container 3c7c8c80b8517b61b0acbf5bee9a13187596f9c333a4fa5edc5d3119ac78c06d: Status 404 returned error can't find the container with id 3c7c8c80b8517b61b0acbf5bee9a13187596f9c333a4fa5edc5d3119ac78c06d Jun 11 21:15:45.026774 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:45.026745 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod275b0bd4_429e_4171_b5f8_70a3bb42f08f.slice/crio-47f9d8b8ed3ff5df7385a3412c80392030c473c9354ae4e03361ab72fb9af423 WatchSource:0}: Error finding container 47f9d8b8ed3ff5df7385a3412c80392030c473c9354ae4e03361ab72fb9af423: Status 404 returned error can't find the container with id 47f9d8b8ed3ff5df7385a3412c80392030c473c9354ae4e03361ab72fb9af423 Jun 11 21:15:45.028193 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:45.028174 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf88e5323_c309_4bdb_ae95_359b2a92061b.slice/crio-fd6019f941db0d58890c8d998c918d81ccc15298cdb14533943ebe24e8469e0d WatchSource:0}: Error finding container fd6019f941db0d58890c8d998c918d81ccc15298cdb14533943ebe24e8469e0d: Status 404 returned error can't find the container with id fd6019f941db0d58890c8d998c918d81ccc15298cdb14533943ebe24e8469e0d Jun 11 21:15:45.029197 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:45.029146 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb641e893_d298_4942_8ccb_6145ef84bf81.slice/crio-c29f44bca17a297ddaab9c2b5bf69f88e4cf24d12982da238c9117e2428c41e2 WatchSource:0}: Error finding container c29f44bca17a297ddaab9c2b5bf69f88e4cf24d12982da238c9117e2428c41e2: Status 404 returned error can't find the container with id c29f44bca17a297ddaab9c2b5bf69f88e4cf24d12982da238c9117e2428c41e2 Jun 11 21:15:45.031825 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:15:45.031690 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdfaf7d3_ae13_48e9_b9ea_040ae02d886a.slice/crio-fc54eb1634fd8624007adb69224e80e256c68fe6a555fe38e046d9257cfa10e8 WatchSource:0}: Error finding container fc54eb1634fd8624007adb69224e80e256c68fe6a555fe38e046d9257cfa10e8: Status 404 returned error can't find the container with id fc54eb1634fd8624007adb69224e80e256c68fe6a555fe38e046d9257cfa10e8 Jun 11 21:15:45.059309 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:45.059277 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" event={"ID":"f88e5323-c309-4bdb-ae95-359b2a92061b","Type":"ContainerStarted","Data":"fd6019f941db0d58890c8d998c918d81ccc15298cdb14533943ebe24e8469e0d"} Jun 11 21:15:45.060511 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:45.060485 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" event={"ID":"c7849da4-f326-4b8c-a436-5b0d68958170","Type":"ContainerStarted","Data":"f796989e1748cc5b2442182b3fef612e825446e57ffe268eb879b47fb8f55413"} Jun 11 21:15:45.061615 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:45.061590 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w26g9" event={"ID":"5678560c-39b0-4892-a879-3613ac253097","Type":"ContainerStarted","Data":"ee7ff222021f90d2d2d73011975ff45f93a54c909f916304830d04e7015a3589"} Jun 11 21:15:45.062741 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:45.062703 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s8b5p" event={"ID":"bdfaf7d3-ae13-48e9-b9ea-040ae02d886a","Type":"ContainerStarted","Data":"fc54eb1634fd8624007adb69224e80e256c68fe6a555fe38e046d9257cfa10e8"} Jun 11 21:15:45.063711 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:45.063689 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cqksf" event={"ID":"71fed145-1fbc-4433-bfb0-394c5631bec6","Type":"ContainerStarted","Data":"3c7c8c80b8517b61b0acbf5bee9a13187596f9c333a4fa5edc5d3119ac78c06d"} Jun 11 21:15:45.067696 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:45.067667 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-thgw8" event={"ID":"7b88bd69-b4df-4345-a24c-e51adeba9023","Type":"ContainerStarted","Data":"ee32d9b85fda173e5d4528b95d4c2234c1d5277bb2194c8ac11f0df0908af891"} Jun 11 21:15:45.070194 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:45.070174 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bl9g7" event={"ID":"9094ab8c-8937-44bb-9f4e-ed8445c3c55f","Type":"ContainerStarted","Data":"2e35646cdabc4149f573dd9e59d334a3ff09a408888bd9c28b5e62d52c71fb55"} Jun 11 21:15:45.075235 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:45.075216 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" event={"ID":"275b0bd4-429e-4171-b5f8-70a3bb42f08f","Type":"ContainerStarted","Data":"47f9d8b8ed3ff5df7385a3412c80392030c473c9354ae4e03361ab72fb9af423"} Jun 11 21:15:45.076528 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:45.076508 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x65dp" event={"ID":"b641e893-d298-4942-8ccb-6145ef84bf81","Type":"ContainerStarted","Data":"c29f44bca17a297ddaab9c2b5bf69f88e4cf24d12982da238c9117e2428c41e2"} Jun 11 21:15:45.530800 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:45.530492 2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 21:15:45.615676 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:45.615638 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2xdc\" (UniqueName: \"kubernetes.io/projected/54688996-c5d4-445d-a6ff-4997a19fe1d6-kube-api-access-g2xdc\") pod \"network-check-target-shxrd\" (UID: \"54688996-c5d4-445d-a6ff-4997a19fe1d6\") " pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:15:45.615870 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:45.615852 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 21:15:45.615946 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:45.615877 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 21:15:45.615946 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:45.615890 2570 projected.go:196] Error preparing data for projected volume kube-api-access-g2xdc for pod openshift-network-diagnostics/network-check-target-shxrd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:15:45.616050 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:45.615949 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54688996-c5d4-445d-a6ff-4997a19fe1d6-kube-api-access-g2xdc podName:54688996-c5d4-445d-a6ff-4997a19fe1d6 nodeName:}" failed. No retries permitted until 2026-06-11 21:15:47.615930885 +0000 UTC m=+6.027254402 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-g2xdc" (UniqueName: "kubernetes.io/projected/54688996-c5d4-445d-a6ff-4997a19fe1d6-kube-api-access-g2xdc") pod "network-check-target-shxrd" (UID: "54688996-c5d4-445d-a6ff-4997a19fe1d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:15:45.716678 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:45.716632 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f4a1fd8-505b-4061-a432-620cb3c68386-original-pull-secret\") pod \"global-pull-secret-syncer-4c247\" (UID: \"6f4a1fd8-505b-4061-a432-620cb3c68386\") " pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:15:45.716843 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:45.716736 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs\") pod \"network-metrics-daemon-2bmhl\" (UID: \"329e0949-4197-4a29-afd8-1300e76b75cf\") " pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:15:45.716899 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:45.716863 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:15:45.716964 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:45.716922 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs podName:329e0949-4197-4a29-afd8-1300e76b75cf nodeName:}" failed. No retries permitted until 2026-06-11 21:15:47.71690266 +0000 UTC m=+6.128226182 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs") pod "network-metrics-daemon-2bmhl" (UID: "329e0949-4197-4a29-afd8-1300e76b75cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:15:45.717326 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:45.717308 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 21:15:45.717405 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:45.717361 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f4a1fd8-505b-4061-a432-620cb3c68386-original-pull-secret podName:6f4a1fd8-505b-4061-a432-620cb3c68386 nodeName:}" failed. No retries permitted until 2026-06-11 21:15:47.717346687 +0000 UTC m=+6.128670211 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f4a1fd8-505b-4061-a432-620cb3c68386-original-pull-secret") pod "global-pull-secret-syncer-4c247" (UID: "6f4a1fd8-505b-4061-a432-620cb3c68386") : object "kube-system"/"original-pull-secret" not registered Jun 11 21:15:46.029582 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:46.029414 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-10 21:10:44 +0000 UTC" deadline="2027-12-05 15:22:13.008658272 +0000 UTC" Jun 11 21:15:46.029582 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:46.029513 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13002h6m26.979149462s" Jun 11 21:15:46.050040 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:46.049120 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:15:46.050040 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:46.049247 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4c247" podUID="6f4a1fd8-505b-4061-a432-620cb3c68386" Jun 11 21:15:46.050040 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:46.049714 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:15:46.050040 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:46.049807 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shxrd" podUID="54688996-c5d4-445d-a6ff-4997a19fe1d6" Jun 11 21:15:46.050040 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:46.049874 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:15:46.050040 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:46.049955 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bmhl" podUID="329e0949-4197-4a29-afd8-1300e76b75cf" Jun 11 21:15:46.094299 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:46.094262 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal" event={"ID":"f9fdf41d985ef036011199dc3759ac96","Type":"ContainerStarted","Data":"ee0f9a6d0b8da11c56d860dadc72e5250cc0d6861c701d925e8b9f4803202582"} Jun 11 21:15:46.109491 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:46.109333 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-6.ec2.internal" podStartSLOduration=3.109313491 podStartE2EDuration="3.109313491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:15:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:15:46.108211496 +0000 UTC m=+4.519535054" watchObservedRunningTime="2026-06-11 21:15:46.109313491 +0000 UTC m=+4.520637031" Jun 11 21:15:46.111509 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:46.110912 2570 generic.go:356] "Generic (PLEG): container finished" podID="de484ba5593857b981e91fc7cd0040af" containerID="9c4e7b7cb3615a2e1c444380c9685ce7a9176ac71bed341751cdbeabf12e2e9b" exitCode=0 Jun 11 21:15:46.111509 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:46.110955 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" event={"ID":"de484ba5593857b981e91fc7cd0040af","Type":"ContainerDied","Data":"9c4e7b7cb3615a2e1c444380c9685ce7a9176ac71bed341751cdbeabf12e2e9b"} Jun 11 21:15:47.124576 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:47.124526 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" event={"ID":"de484ba5593857b981e91fc7cd0040af","Type":"ContainerStarted","Data":"beb88295d2dc4b754362b0b18eec738be399e721c69be90715f0516a1cd027ee"} Jun 11 21:15:47.633448 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:47.632747 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2xdc\" (UniqueName: \"kubernetes.io/projected/54688996-c5d4-445d-a6ff-4997a19fe1d6-kube-api-access-g2xdc\") pod \"network-check-target-shxrd\" (UID: \"54688996-c5d4-445d-a6ff-4997a19fe1d6\") " pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:15:47.633448 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:47.632997 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 21:15:47.633448 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:47.633024 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 21:15:47.633448 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:47.633037 2570 projected.go:196] Error preparing data for projected volume kube-api-access-g2xdc for pod openshift-network-diagnostics/network-check-target-shxrd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:15:47.633448 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:47.633099 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54688996-c5d4-445d-a6ff-4997a19fe1d6-kube-api-access-g2xdc podName:54688996-c5d4-445d-a6ff-4997a19fe1d6 nodeName:}" failed. No retries permitted until 2026-06-11 21:15:51.633077053 +0000 UTC m=+10.044400572 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-g2xdc" (UniqueName: "kubernetes.io/projected/54688996-c5d4-445d-a6ff-4997a19fe1d6-kube-api-access-g2xdc") pod "network-check-target-shxrd" (UID: "54688996-c5d4-445d-a6ff-4997a19fe1d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:15:47.734652 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:47.733835 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f4a1fd8-505b-4061-a432-620cb3c68386-original-pull-secret\") pod \"global-pull-secret-syncer-4c247\" (UID: \"6f4a1fd8-505b-4061-a432-620cb3c68386\") " pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:15:47.734652 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:47.733954 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs\") pod \"network-metrics-daemon-2bmhl\" (UID: \"329e0949-4197-4a29-afd8-1300e76b75cf\") " pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:15:47.734652 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:47.734080 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:15:47.734652 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:47.734149 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs podName:329e0949-4197-4a29-afd8-1300e76b75cf nodeName:}" failed. No retries permitted until 2026-06-11 21:15:51.734125837 +0000 UTC m=+10.145449374 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs") pod "network-metrics-daemon-2bmhl" (UID: "329e0949-4197-4a29-afd8-1300e76b75cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:15:47.734652 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:47.734544 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 21:15:47.734652 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:47.734614 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f4a1fd8-505b-4061-a432-620cb3c68386-original-pull-secret podName:6f4a1fd8-505b-4061-a432-620cb3c68386 nodeName:}" failed. No retries permitted until 2026-06-11 21:15:51.734597805 +0000 UTC m=+10.145921324 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f4a1fd8-505b-4061-a432-620cb3c68386-original-pull-secret") pod "global-pull-secret-syncer-4c247" (UID: "6f4a1fd8-505b-4061-a432-620cb3c68386") : object "kube-system"/"original-pull-secret" not registered Jun 11 21:15:48.051583 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:48.051486 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:15:48.051583 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:48.051532 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:15:48.051741 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:48.051493 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:15:48.051741 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:48.051693 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4c247" podUID="6f4a1fd8-505b-4061-a432-620cb3c68386" Jun 11 21:15:48.055432 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:48.051843 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bmhl" podUID="329e0949-4197-4a29-afd8-1300e76b75cf" Jun 11 21:15:48.055432 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:48.051955 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shxrd" podUID="54688996-c5d4-445d-a6ff-4997a19fe1d6" Jun 11 21:15:50.047409 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:50.047376 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:15:50.047409 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:50.047414 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:15:50.047957 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:50.047376 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:15:50.047957 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:50.047523 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bmhl" podUID="329e0949-4197-4a29-afd8-1300e76b75cf" Jun 11 21:15:50.047957 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:50.047630 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shxrd" podUID="54688996-c5d4-445d-a6ff-4997a19fe1d6" Jun 11 21:15:50.047957 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:50.047695 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4c247" podUID="6f4a1fd8-505b-4061-a432-620cb3c68386" Jun 11 21:15:51.666541 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:51.666501 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2xdc\" (UniqueName: \"kubernetes.io/projected/54688996-c5d4-445d-a6ff-4997a19fe1d6-kube-api-access-g2xdc\") pod \"network-check-target-shxrd\" (UID: \"54688996-c5d4-445d-a6ff-4997a19fe1d6\") " pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:15:51.667010 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:51.666690 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 21:15:51.667010 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:51.666714 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 21:15:51.667010 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:51.666727 2570 projected.go:196] Error preparing data for projected volume kube-api-access-g2xdc for pod openshift-network-diagnostics/network-check-target-shxrd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:15:51.667010 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:51.666791 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54688996-c5d4-445d-a6ff-4997a19fe1d6-kube-api-access-g2xdc podName:54688996-c5d4-445d-a6ff-4997a19fe1d6 nodeName:}" failed. No retries permitted until 2026-06-11 21:15:59.666771123 +0000 UTC m=+18.078094660 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-g2xdc" (UniqueName: "kubernetes.io/projected/54688996-c5d4-445d-a6ff-4997a19fe1d6-kube-api-access-g2xdc") pod "network-check-target-shxrd" (UID: "54688996-c5d4-445d-a6ff-4997a19fe1d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:15:51.767483 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:51.767422 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs\") pod \"network-metrics-daemon-2bmhl\" (UID: \"329e0949-4197-4a29-afd8-1300e76b75cf\") " pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:15:51.767687 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:51.767480 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f4a1fd8-505b-4061-a432-620cb3c68386-original-pull-secret\") pod \"global-pull-secret-syncer-4c247\" (UID: \"6f4a1fd8-505b-4061-a432-620cb3c68386\") " pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:15:51.767687 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:51.767607 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:15:51.767687 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:51.767626 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 21:15:51.767687 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:51.767682 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f4a1fd8-505b-4061-a432-620cb3c68386-original-pull-secret podName:6f4a1fd8-505b-4061-a432-620cb3c68386 nodeName:}" failed. No retries permitted until 2026-06-11 21:15:59.76766359 +0000 UTC m=+18.178987109 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f4a1fd8-505b-4061-a432-620cb3c68386-original-pull-secret") pod "global-pull-secret-syncer-4c247" (UID: "6f4a1fd8-505b-4061-a432-620cb3c68386") : object "kube-system"/"original-pull-secret" not registered Jun 11 21:15:51.767924 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:51.767702 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs podName:329e0949-4197-4a29-afd8-1300e76b75cf nodeName:}" failed. No retries permitted until 2026-06-11 21:15:59.767692777 +0000 UTC m=+18.179016292 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs") pod "network-metrics-daemon-2bmhl" (UID: "329e0949-4197-4a29-afd8-1300e76b75cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:15:52.047328 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:52.047240 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:15:52.047485 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:52.047347 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:15:52.047485 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:52.047368 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bmhl" podUID="329e0949-4197-4a29-afd8-1300e76b75cf" Jun 11 21:15:52.047485 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:52.047422 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4c247" podUID="6f4a1fd8-505b-4061-a432-620cb3c68386" Jun 11 21:15:52.047485 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:52.047465 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:15:52.047698 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:52.047528 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shxrd" podUID="54688996-c5d4-445d-a6ff-4997a19fe1d6" Jun 11 21:15:54.047557 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:54.047485 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:15:54.047557 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:54.047485 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:15:54.047557 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:54.047544 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:15:54.048145 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:54.047705 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bmhl" podUID="329e0949-4197-4a29-afd8-1300e76b75cf" Jun 11 21:15:54.048204 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:54.048169 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shxrd" podUID="54688996-c5d4-445d-a6ff-4997a19fe1d6" Jun 11 21:15:54.048298 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:54.048263 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4c247" podUID="6f4a1fd8-505b-4061-a432-620cb3c68386" Jun 11 21:15:56.047284 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:56.047239 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:15:56.047798 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:56.047288 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:15:56.047798 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:56.047239 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:15:56.047798 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:56.047372 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shxrd" podUID="54688996-c5d4-445d-a6ff-4997a19fe1d6" Jun 11 21:15:56.047798 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:56.047478 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4c247" podUID="6f4a1fd8-505b-4061-a432-620cb3c68386" Jun 11 21:15:56.047798 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:56.047605 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bmhl" podUID="329e0949-4197-4a29-afd8-1300e76b75cf" Jun 11 21:15:58.046658 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:58.046620 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:15:58.047131 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:58.046749 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shxrd" podUID="54688996-c5d4-445d-a6ff-4997a19fe1d6" Jun 11 21:15:58.047131 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:58.046621 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:15:58.047131 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:58.046838 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bmhl" podUID="329e0949-4197-4a29-afd8-1300e76b75cf" Jun 11 21:15:58.047131 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:58.046621 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:15:58.047131 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:58.046938 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4c247" podUID="6f4a1fd8-505b-4061-a432-620cb3c68386" Jun 11 21:15:59.729036 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:59.729001 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2xdc\" (UniqueName: \"kubernetes.io/projected/54688996-c5d4-445d-a6ff-4997a19fe1d6-kube-api-access-g2xdc\") pod \"network-check-target-shxrd\" (UID: \"54688996-c5d4-445d-a6ff-4997a19fe1d6\") " pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:15:59.729508 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:59.729149 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 21:15:59.729508 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:59.729169 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 21:15:59.729508 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:59.729179 2570 projected.go:196] Error preparing data for projected volume kube-api-access-g2xdc for pod openshift-network-diagnostics/network-check-target-shxrd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:15:59.729508 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:59.729234 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54688996-c5d4-445d-a6ff-4997a19fe1d6-kube-api-access-g2xdc podName:54688996-c5d4-445d-a6ff-4997a19fe1d6 nodeName:}" failed. No retries permitted until 2026-06-11 21:16:15.729216746 +0000 UTC m=+34.140540262 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-g2xdc" (UniqueName: "kubernetes.io/projected/54688996-c5d4-445d-a6ff-4997a19fe1d6-kube-api-access-g2xdc") pod "network-check-target-shxrd" (UID: "54688996-c5d4-445d-a6ff-4997a19fe1d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:15:59.829720 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:59.829677 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs\") pod \"network-metrics-daemon-2bmhl\" (UID: \"329e0949-4197-4a29-afd8-1300e76b75cf\") " pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:15:59.829908 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:15:59.829734 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f4a1fd8-505b-4061-a432-620cb3c68386-original-pull-secret\") pod \"global-pull-secret-syncer-4c247\" (UID: \"6f4a1fd8-505b-4061-a432-620cb3c68386\") " pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:15:59.829908 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:59.829869 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:15:59.830019 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:59.829945 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs podName:329e0949-4197-4a29-afd8-1300e76b75cf nodeName:}" failed. No retries permitted until 2026-06-11 21:16:15.829926714 +0000 UTC m=+34.241250233 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs") pod "network-metrics-daemon-2bmhl" (UID: "329e0949-4197-4a29-afd8-1300e76b75cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:15:59.830019 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:59.829951 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 21:15:59.830019 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:15:59.830005 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f4a1fd8-505b-4061-a432-620cb3c68386-original-pull-secret podName:6f4a1fd8-505b-4061-a432-620cb3c68386 nodeName:}" failed. No retries permitted until 2026-06-11 21:16:15.829988391 +0000 UTC m=+34.241311924 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f4a1fd8-505b-4061-a432-620cb3c68386-original-pull-secret") pod "global-pull-secret-syncer-4c247" (UID: "6f4a1fd8-505b-4061-a432-620cb3c68386") : object "kube-system"/"original-pull-secret" not registered Jun 11 21:16:00.047382 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:00.047291 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:16:00.047544 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:00.047291 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:16:00.047544 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:00.047428 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:16:00.047544 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:00.047430 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bmhl" podUID="329e0949-4197-4a29-afd8-1300e76b75cf" Jun 11 21:16:00.047544 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:00.047492 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shxrd" podUID="54688996-c5d4-445d-a6ff-4997a19fe1d6" Jun 11 21:16:00.047779 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:00.047578 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4c247" podUID="6f4a1fd8-505b-4061-a432-620cb3c68386" Jun 11 21:16:02.047747 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:02.047696 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:16:02.048293 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:02.047809 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:16:02.048293 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:02.047824 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bmhl" podUID="329e0949-4197-4a29-afd8-1300e76b75cf" Jun 11 21:16:02.048293 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:02.047912 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shxrd" podUID="54688996-c5d4-445d-a6ff-4997a19fe1d6" Jun 11 21:16:02.048293 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:02.047973 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:16:02.048293 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:02.048054 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4c247" podUID="6f4a1fd8-505b-4061-a432-620cb3c68386" Jun 11 21:16:04.047208 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.047095 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:16:04.047849 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.047097 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:16:04.047849 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:04.047292 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bmhl" podUID="329e0949-4197-4a29-afd8-1300e76b75cf" Jun 11 21:16:04.047849 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:04.047381 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4c247" podUID="6f4a1fd8-505b-4061-a432-620cb3c68386" Jun 11 21:16:04.047849 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.047099 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:16:04.047849 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:04.047474 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shxrd" podUID="54688996-c5d4-445d-a6ff-4997a19fe1d6" Jun 11 21:16:04.161037 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.160769 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" event={"ID":"275b0bd4-429e-4171-b5f8-70a3bb42f08f","Type":"ContainerStarted","Data":"de54a8c0673cf101d94c5cf5b08d7ea87264cbe7e50cde3eccbc51a296893390"} Jun 11 21:16:04.162327 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.162301 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x65dp" event={"ID":"b641e893-d298-4942-8ccb-6145ef84bf81","Type":"ContainerStarted","Data":"0cdd6914cc6ceb489bc422b994025007a2761ccd5ae784667d4783f0590df608"} Jun 11 21:16:04.164019 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.163994 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" event={"ID":"f88e5323-c309-4bdb-ae95-359b2a92061b","Type":"ContainerStarted","Data":"8ffdb14752aad8a0a7512027673c9f9fda8ee6e661eada2bd76d7bfb1fd7d506"} Jun 11 21:16:04.166926 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.166906 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/ovn-acl-logging/0.log" Jun 11 21:16:04.167272 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.167247 2570 generic.go:356] "Generic (PLEG): container finished" podID="c7849da4-f326-4b8c-a436-5b0d68958170" containerID="e2656b172fbd88d8f376e1709d7b079949a0e8724e4b0e80cd847514b55a65a3" exitCode=1 Jun 11 21:16:04.167373 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.167275 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" event={"ID":"c7849da4-f326-4b8c-a436-5b0d68958170","Type":"ContainerStarted","Data":"231ccedd927e842991aaf365df88d716ed9a08e256867e417c50f002d3520cdf"} Jun 11 21:16:04.167373 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.167328 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" event={"ID":"c7849da4-f326-4b8c-a436-5b0d68958170","Type":"ContainerStarted","Data":"f1f113865d6b1b961ed1637f2084ad6a787ca836df4f6aa25d3f13b6300363e9"} Jun 11 21:16:04.167489 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.167374 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" event={"ID":"c7849da4-f326-4b8c-a436-5b0d68958170","Type":"ContainerStarted","Data":"7d8c38034c901134ca18f003c68416eee621ec7c788a1140fbb1b90290853541"} Jun 11 21:16:04.167489 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.167421 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" event={"ID":"c7849da4-f326-4b8c-a436-5b0d68958170","Type":"ContainerStarted","Data":"b6c4dd8f475fa37700c863412b38e62ff0f5d6d5f1c19108e67d45fb7fcb5ba0"} Jun 11 21:16:04.167489 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.167469 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" event={"ID":"c7849da4-f326-4b8c-a436-5b0d68958170","Type":"ContainerDied","Data":"e2656b172fbd88d8f376e1709d7b079949a0e8724e4b0e80cd847514b55a65a3"} Jun 11 21:16:04.167701 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.167517 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" event={"ID":"c7849da4-f326-4b8c-a436-5b0d68958170","Type":"ContainerStarted","Data":"90fc488fdf8c1a36d1013290de05fea109f98f3c04e226f0f6b184c04878155a"} Jun 11 21:16:04.168930 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.168906 2570 generic.go:356] "Generic (PLEG): container finished" podID="5678560c-39b0-4892-a879-3613ac253097" containerID="8791d0e2db3035279bf384751828e552b030565c5a0b62cc45b193d65e76cf79" exitCode=0 Jun 11 21:16:04.169032 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.168994 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w26g9" event={"ID":"5678560c-39b0-4892-a879-3613ac253097","Type":"ContainerDied","Data":"8791d0e2db3035279bf384751828e552b030565c5a0b62cc45b193d65e76cf79"} Jun 11 21:16:04.170668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.170558 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s8b5p" event={"ID":"bdfaf7d3-ae13-48e9-b9ea-040ae02d886a","Type":"ContainerStarted","Data":"b75e6627c4c74d017cfc3e3873b485b4942ffd59bf9ec3431733cc732016a94e"} Jun 11 21:16:04.172060 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.172028 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-thgw8" event={"ID":"7b88bd69-b4df-4345-a24c-e51adeba9023","Type":"ContainerStarted","Data":"11c7d2cee354bcd9851029785c388622114117ec92ce96944e1e665b1a24969f"} Jun 11 21:16:04.173475 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.173454 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bl9g7" event={"ID":"9094ab8c-8937-44bb-9f4e-ed8445c3c55f","Type":"ContainerStarted","Data":"6ffede8636631c76fd6a86c436a506a489446fd1988016e9f731577e2981c304"} Jun 11 21:16:04.176888 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.176849 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fhhkw" podStartSLOduration=3.723765057 podStartE2EDuration="22.176838105s" totalImagesPullingTime="18.453073048s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:16:04.176478224 +0000 UTC m=+22.587801762" watchObservedRunningTime="2026-06-11 21:16:04.176838105 +0000 UTC m=+22.588161642" Jun 11 21:16:04.176985 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.176967 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-6.ec2.internal" podStartSLOduration=21.176962524 podStartE2EDuration="21.176962524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:15:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:15:47.141866636 +0000 UTC m=+5.553190188" watchObservedRunningTime="2026-06-11 21:16:04.176962524 +0000 UTC m=+22.588286070" Jun 11 21:16:04.204815 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.204774 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-s8b5p" podStartSLOduration=3.774629403 podStartE2EDuration="22.204760426s" totalImagesPullingTime="18.430131023s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:16:04.204698762 +0000 UTC m=+22.616022300" watchObservedRunningTime="2026-06-11 21:16:04.204760426 +0000 UTC m=+22.616083963" Jun 11 21:16:04.204928 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.204865 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bl9g7" podStartSLOduration=3.69150566 podStartE2EDuration="22.204861756s" totalImagesPullingTime="18.513356096s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:16:04.191748434 +0000 UTC m=+22.603071983" watchObservedRunningTime="2026-06-11 21:16:04.204861756 +0000 UTC m=+22.616185293" Jun 11 21:16:04.227998 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.227943 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-x65dp" podStartSLOduration=8.011551779 podStartE2EDuration="22.227927678s" totalImagesPullingTime="14.216375899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:16:04.227454407 +0000 UTC m=+22.638777974" watchObservedRunningTime="2026-06-11 21:16:04.227927678 +0000 UTC m=+22.639251215" Jun 11 21:16:04.241094 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:04.241044 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-thgw8" podStartSLOduration=3.798739789 podStartE2EDuration="22.241033606s" totalImagesPullingTime="18.442293817s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:16:04.240760736 +0000 UTC m=+22.652084275" watchObservedRunningTime="2026-06-11 21:16:04.241033606 +0000 UTC m=+22.652357141" Jun 11 21:16:05.118439 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:05.118408 2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 11 21:16:05.177603 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:05.177553 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cqksf" event={"ID":"71fed145-1fbc-4433-bfb0-394c5631bec6","Type":"ContainerStarted","Data":"df64e2b45d57daa28f46d751aedaa1fa7fe52deaa5ff38536ddbf27d29ebe1f6"} Jun 11 21:16:05.179799 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:05.179769 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" event={"ID":"f88e5323-c309-4bdb-ae95-359b2a92061b","Type":"ContainerStarted","Data":"321b2b128f0993b5d3d4b28a590c12129ada7bb9efaba927e8a27c512e5f20e0"} Jun 11 21:16:05.194165 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:05.194107 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cqksf" podStartSLOduration=4.757924427 podStartE2EDuration="23.194089975s" totalImagesPullingTime="18.436165548s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:16:05.193559678 +0000 UTC m=+23.604883219" watchObservedRunningTime="2026-06-11 21:16:05.194089975 +0000 UTC m=+23.605413517" Jun 11 21:16:06.046834 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:06.046801 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:16:06.046997 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:06.046801 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:16:06.046997 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:06.046924 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4c247" podUID="6f4a1fd8-505b-4061-a432-620cb3c68386" Jun 11 21:16:06.046997 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:06.046801 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:16:06.047160 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:06.046992 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bmhl" podUID="329e0949-4197-4a29-afd8-1300e76b75cf" Jun 11 21:16:06.047160 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:06.047106 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shxrd" podUID="54688996-c5d4-445d-a6ff-4997a19fe1d6" Jun 11 21:16:06.060886 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:06.060800 2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-11T21:16:05.118441116Z","UUID":"7071890b-57b8-46b8-943f-2089542e504e","Handler":null,"Name":"","Endpoint":""} Jun 11 21:16:06.062778 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:06.062759 2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 11 21:16:06.062874 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:06.062795 2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 11 21:16:07.186521 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:07.186478 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" event={"ID":"f88e5323-c309-4bdb-ae95-359b2a92061b","Type":"ContainerStarted","Data":"64bdbeb68277c339bfd752c7290c9296e0a3310aafafd129b1207f2cc480bb04"} Jun 11 21:16:07.190220 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:07.190197 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/ovn-acl-logging/0.log" Jun 11 21:16:07.190618 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:07.190594 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" event={"ID":"c7849da4-f326-4b8c-a436-5b0d68958170","Type":"ContainerStarted","Data":"87f1ba00da575ba7e51d378ba94b7024ee6cbfc9daf84d8242a90f0d95e0234b"} Jun 11 21:16:07.205862 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:07.205818 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kc22x" podStartSLOduration=4.489545041 podStartE2EDuration="25.205797775s" totalImagesPullingTime="20.716252734s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:15:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:16:07.205764494 +0000 UTC m=+25.617088059" watchObservedRunningTime="2026-06-11 21:16:07.205797775 +0000 UTC m=+25.617121293" Jun 11 21:16:07.575049 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:07.574794 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-x65dp" Jun 11 21:16:07.575466 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:07.575445 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-x65dp" Jun 11 21:16:08.046584 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:08.046528 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:16:08.046736 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:08.046528 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:16:08.046736 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:08.046665 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:16:08.046736 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:08.046688 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4c247" podUID="6f4a1fd8-505b-4061-a432-620cb3c68386" Jun 11 21:16:08.046900 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:08.046739 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shxrd" podUID="54688996-c5d4-445d-a6ff-4997a19fe1d6" Jun 11 21:16:08.046900 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:08.046800 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bmhl" podUID="329e0949-4197-4a29-afd8-1300e76b75cf" Jun 11 21:16:08.588260 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:08.588188 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-x65dp" Jun 11 21:16:08.588755 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:08.588733 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-x65dp" Jun 11 21:16:09.198325 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:09.198249 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/ovn-acl-logging/0.log" Jun 11 21:16:09.198658 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:09.198628 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" event={"ID":"c7849da4-f326-4b8c-a436-5b0d68958170","Type":"ContainerStarted","Data":"0e3e064c2749176a223f0633d5d893f81f6af50596ed3288a8aae9ad04a2b5f5"} Jun 11 21:16:09.198924 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:09.198885 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:16:09.199158 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:09.199131 2570 scope.go:117] "RemoveContainer" containerID="e2656b172fbd88d8f376e1709d7b079949a0e8724e4b0e80cd847514b55a65a3" Jun 11 21:16:09.200374 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:09.200354 2570 generic.go:356] "Generic (PLEG): container finished" podID="5678560c-39b0-4892-a879-3613ac253097" containerID="f9c8d04571c79afe25d890d545338fb4f824a325881af227ee3696f2f2bd8f0f" exitCode=0 Jun 11 21:16:09.200471 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:09.200417 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w26g9" event={"ID":"5678560c-39b0-4892-a879-3613ac253097","Type":"ContainerDied","Data":"f9c8d04571c79afe25d890d545338fb4f824a325881af227ee3696f2f2bd8f0f"} Jun 11 21:16:09.215834 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:09.215815 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:16:10.047581 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:10.047374 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:16:10.048196 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:10.047372 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:16:10.048196 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:10.047711 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bmhl" podUID="329e0949-4197-4a29-afd8-1300e76b75cf" Jun 11 21:16:10.048196 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:10.047726 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4c247" podUID="6f4a1fd8-505b-4061-a432-620cb3c68386" Jun 11 21:16:10.048196 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:10.047423 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:16:10.048196 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:10.047796 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shxrd" podUID="54688996-c5d4-445d-a6ff-4997a19fe1d6" Jun 11 21:16:10.205204 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:10.205110 2570 generic.go:356] "Generic (PLEG): container finished" podID="5678560c-39b0-4892-a879-3613ac253097" containerID="b6f20ef52b58280941198e12bc649e626bae1c998e6a748db7c9b4f29c043ebe" exitCode=0 Jun 11 21:16:10.205346 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:10.205199 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w26g9" event={"ID":"5678560c-39b0-4892-a879-3613ac253097","Type":"ContainerDied","Data":"b6f20ef52b58280941198e12bc649e626bae1c998e6a748db7c9b4f29c043ebe"} Jun 11 21:16:10.209276 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:10.209257 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/ovn-acl-logging/0.log" Jun 11 21:16:10.209666 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:10.209638 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" event={"ID":"c7849da4-f326-4b8c-a436-5b0d68958170","Type":"ContainerStarted","Data":"02f29485587494586008b0a7ee817b5a0e05cf688272f413fb28072f68e74efb"} Jun 11 21:16:10.209934 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:10.209913 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:16:10.210035 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:10.209977 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:16:10.224678 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:10.224649 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:16:10.249902 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:10.249854 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" podStartSLOduration=9.79021708 podStartE2EDuration="28.249836639s" totalImagesPullingTime="18.459619559s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:16:10.248469348 +0000 UTC m=+28.659792887" watchObservedRunningTime="2026-06-11 21:16:10.249836639 +0000 UTC m=+28.661160173" Jun 11 21:16:10.893235 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:10.893200 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4c247"] Jun 11 21:16:10.893395 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:10.893381 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:16:10.893520 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:10.893496 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4c247" podUID="6f4a1fd8-505b-4061-a432-620cb3c68386" Jun 11 21:16:10.897017 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:10.896954 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2bmhl"] Jun 11 21:16:10.897100 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:10.897071 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:16:10.897285 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:10.897179 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bmhl" podUID="329e0949-4197-4a29-afd8-1300e76b75cf" Jun 11 21:16:10.897696 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:10.897670 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-shxrd"] Jun 11 21:16:10.897798 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:10.897764 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:16:10.897867 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:10.897849 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shxrd" podUID="54688996-c5d4-445d-a6ff-4997a19fe1d6" Jun 11 21:16:11.213682 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:11.213594 2570 generic.go:356] "Generic (PLEG): container finished" podID="5678560c-39b0-4892-a879-3613ac253097" containerID="9dbb0054cf1f9935df01f920cc70ef94840bae3ea60b754707df6637ba394308" exitCode=0 Jun 11 21:16:11.214081 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:11.213677 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w26g9" event={"ID":"5678560c-39b0-4892-a879-3613ac253097","Type":"ContainerDied","Data":"9dbb0054cf1f9935df01f920cc70ef94840bae3ea60b754707df6637ba394308"} Jun 11 21:16:12.047502 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:12.047470 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:16:12.047717 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:12.047594 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shxrd" podUID="54688996-c5d4-445d-a6ff-4997a19fe1d6" Jun 11 21:16:13.046415 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:13.046338 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:16:13.046828 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:13.046343 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:16:13.046828 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:13.046451 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4c247" podUID="6f4a1fd8-505b-4061-a432-620cb3c68386" Jun 11 21:16:13.046828 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:13.046545 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bmhl" podUID="329e0949-4197-4a29-afd8-1300e76b75cf" Jun 11 21:16:14.046777 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:14.046740 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:16:14.047151 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:14.046869 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shxrd" podUID="54688996-c5d4-445d-a6ff-4997a19fe1d6" Jun 11 21:16:15.047276 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:15.046974 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:16:15.047276 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:15.047007 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:16:15.047778 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:15.047383 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4c247" podUID="6f4a1fd8-505b-4061-a432-620cb3c68386" Jun 11 21:16:15.047778 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:15.047598 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bmhl" podUID="329e0949-4197-4a29-afd8-1300e76b75cf" Jun 11 21:16:15.739101 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:15.739051 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2xdc\" (UniqueName: \"kubernetes.io/projected/54688996-c5d4-445d-a6ff-4997a19fe1d6-kube-api-access-g2xdc\") pod \"network-check-target-shxrd\" (UID: \"54688996-c5d4-445d-a6ff-4997a19fe1d6\") " pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:16:15.739294 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:15.739237 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 21:16:15.739294 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:15.739274 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 21:16:15.739294 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:15.739295 2570 projected.go:196] Error preparing data for projected volume kube-api-access-g2xdc for pod openshift-network-diagnostics/network-check-target-shxrd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:16:15.739432 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:15.739357 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54688996-c5d4-445d-a6ff-4997a19fe1d6-kube-api-access-g2xdc podName:54688996-c5d4-445d-a6ff-4997a19fe1d6 nodeName:}" failed. No retries permitted until 2026-06-11 21:16:47.739342916 +0000 UTC m=+66.150666436 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-g2xdc" (UniqueName: "kubernetes.io/projected/54688996-c5d4-445d-a6ff-4997a19fe1d6-kube-api-access-g2xdc") pod "network-check-target-shxrd" (UID: "54688996-c5d4-445d-a6ff-4997a19fe1d6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 21:16:15.839709 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:15.839670 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs\") pod \"network-metrics-daemon-2bmhl\" (UID: \"329e0949-4197-4a29-afd8-1300e76b75cf\") " pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:16:15.839912 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:15.839725 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f4a1fd8-505b-4061-a432-620cb3c68386-original-pull-secret\") pod \"global-pull-secret-syncer-4c247\" (UID: \"6f4a1fd8-505b-4061-a432-620cb3c68386\") " pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:16:15.839912 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:15.839854 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:16:15.840018 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:15.839938 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs podName:329e0949-4197-4a29-afd8-1300e76b75cf nodeName:}" failed. No retries permitted until 2026-06-11 21:16:47.839916829 +0000 UTC m=+66.251240362 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs") pod "network-metrics-daemon-2bmhl" (UID: "329e0949-4197-4a29-afd8-1300e76b75cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 21:16:15.840018 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:15.839938 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 21:16:15.840018 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:15.839975 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f4a1fd8-505b-4061-a432-620cb3c68386-original-pull-secret podName:6f4a1fd8-505b-4061-a432-620cb3c68386 nodeName:}" failed. No retries permitted until 2026-06-11 21:16:47.839966008 +0000 UTC m=+66.251289524 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f4a1fd8-505b-4061-a432-620cb3c68386-original-pull-secret") pod "global-pull-secret-syncer-4c247" (UID: "6f4a1fd8-505b-4061-a432-620cb3c68386") : object "kube-system"/"original-pull-secret" not registered Jun 11 21:16:16.048254 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.047942 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:16:16.048791 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:16.048310 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-shxrd" podUID="54688996-c5d4-445d-a6ff-4997a19fe1d6" Jun 11 21:16:16.350187 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.350159 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-6.ec2.internal" event="NodeReady" Jun 11 21:16:16.350393 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.350283 2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 11 21:16:16.401956 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.401924 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zpqjq"] Jun 11 21:16:16.414900 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.414871 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-f5djm"] Jun 11 21:16:16.415035 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.414992 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zpqjq" Jun 11 21:16:16.419027 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.419003 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-td2lf\"" Jun 11 21:16:16.419027 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.419018 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 11 21:16:16.419214 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.419002 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 11 21:16:16.429378 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.429358 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zpqjq"] Jun 11 21:16:16.429497 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.429392 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-f5djm"] Jun 11 21:16:16.429497 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.429479 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-f5djm" Jun 11 21:16:16.431841 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.431822 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 11 21:16:16.431841 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.431838 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 11 21:16:16.431841 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.431824 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 11 21:16:16.432039 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.431838 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gzclq\"" Jun 11 21:16:16.547143 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.547105 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls\") pod \"dns-default-zpqjq\" (UID: \"2c0c15cb-361e-40de-83bf-06ccac34cc83\") " pod="openshift-dns/dns-default-zpqjq" Jun 11 21:16:16.547143 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.547148 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4zpq\" (UniqueName: \"kubernetes.io/projected/2c0c15cb-361e-40de-83bf-06ccac34cc83-kube-api-access-l4zpq\") pod \"dns-default-zpqjq\" (UID: \"2c0c15cb-361e-40de-83bf-06ccac34cc83\") " pod="openshift-dns/dns-default-zpqjq" Jun 11 21:16:16.547353 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.547168 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7n9f\" (UniqueName: \"kubernetes.io/projected/5317cca9-f317-4c21-9dad-a2f1ac629264-kube-api-access-d7n9f\") pod \"ingress-canary-f5djm\" (UID: \"5317cca9-f317-4c21-9dad-a2f1ac629264\") " pod="openshift-ingress-canary/ingress-canary-f5djm" Jun 11 21:16:16.547353 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.547229 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2c0c15cb-361e-40de-83bf-06ccac34cc83-config-volume\") pod \"dns-default-zpqjq\" (UID: \"2c0c15cb-361e-40de-83bf-06ccac34cc83\") " pod="openshift-dns/dns-default-zpqjq" Jun 11 21:16:16.547353 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.547258 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5317cca9-f317-4c21-9dad-a2f1ac629264-cert\") pod \"ingress-canary-f5djm\" (UID: \"5317cca9-f317-4c21-9dad-a2f1ac629264\") " pod="openshift-ingress-canary/ingress-canary-f5djm" Jun 11 21:16:16.547353 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.547308 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c0c15cb-361e-40de-83bf-06ccac34cc83-tmp-dir\") pod \"dns-default-zpqjq\" (UID: \"2c0c15cb-361e-40de-83bf-06ccac34cc83\") " pod="openshift-dns/dns-default-zpqjq" Jun 11 21:16:16.648338 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.648314 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c0c15cb-361e-40de-83bf-06ccac34cc83-tmp-dir\") pod \"dns-default-zpqjq\" (UID: \"2c0c15cb-361e-40de-83bf-06ccac34cc83\") " pod="openshift-dns/dns-default-zpqjq" Jun 11 21:16:16.648468 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.648380 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls\") pod \"dns-default-zpqjq\" (UID: \"2c0c15cb-361e-40de-83bf-06ccac34cc83\") " pod="openshift-dns/dns-default-zpqjq" Jun 11 21:16:16.648468 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.648411 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4zpq\" (UniqueName: \"kubernetes.io/projected/2c0c15cb-361e-40de-83bf-06ccac34cc83-kube-api-access-l4zpq\") pod \"dns-default-zpqjq\" (UID: \"2c0c15cb-361e-40de-83bf-06ccac34cc83\") " pod="openshift-dns/dns-default-zpqjq" Jun 11 21:16:16.648468 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.648441 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7n9f\" (UniqueName: \"kubernetes.io/projected/5317cca9-f317-4c21-9dad-a2f1ac629264-kube-api-access-d7n9f\") pod \"ingress-canary-f5djm\" (UID: \"5317cca9-f317-4c21-9dad-a2f1ac629264\") " pod="openshift-ingress-canary/ingress-canary-f5djm" Jun 11 21:16:16.648646 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.648471 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2c0c15cb-361e-40de-83bf-06ccac34cc83-config-volume\") pod \"dns-default-zpqjq\" (UID: \"2c0c15cb-361e-40de-83bf-06ccac34cc83\") " pod="openshift-dns/dns-default-zpqjq" Jun 11 21:16:16.648646 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.648595 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5317cca9-f317-4c21-9dad-a2f1ac629264-cert\") pod \"ingress-canary-f5djm\" (UID: \"5317cca9-f317-4c21-9dad-a2f1ac629264\") " pod="openshift-ingress-canary/ingress-canary-f5djm" Jun 11 21:16:16.648747 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.648722 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c0c15cb-361e-40de-83bf-06ccac34cc83-tmp-dir\") pod \"dns-default-zpqjq\" (UID: \"2c0c15cb-361e-40de-83bf-06ccac34cc83\") " pod="openshift-dns/dns-default-zpqjq" Jun 11 21:16:16.648833 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:16.648813 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 21:16:16.648895 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:16.648884 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls podName:2c0c15cb-361e-40de-83bf-06ccac34cc83 nodeName:}" failed. No retries permitted until 2026-06-11 21:16:17.148865361 +0000 UTC m=+35.560188896 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls") pod "dns-default-zpqjq" (UID: "2c0c15cb-361e-40de-83bf-06ccac34cc83") : secret "dns-default-metrics-tls" not found Jun 11 21:16:16.649143 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.649123 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2c0c15cb-361e-40de-83bf-06ccac34cc83-config-volume\") pod \"dns-default-zpqjq\" (UID: \"2c0c15cb-361e-40de-83bf-06ccac34cc83\") " pod="openshift-dns/dns-default-zpqjq" Jun 11 21:16:16.652944 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.652921 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5317cca9-f317-4c21-9dad-a2f1ac629264-cert\") pod \"ingress-canary-f5djm\" (UID: \"5317cca9-f317-4c21-9dad-a2f1ac629264\") " pod="openshift-ingress-canary/ingress-canary-f5djm" Jun 11 21:16:16.659828 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.659806 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4zpq\" (UniqueName: \"kubernetes.io/projected/2c0c15cb-361e-40de-83bf-06ccac34cc83-kube-api-access-l4zpq\") pod \"dns-default-zpqjq\" (UID: \"2c0c15cb-361e-40de-83bf-06ccac34cc83\") " pod="openshift-dns/dns-default-zpqjq" Jun 11 21:16:16.660027 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.660010 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7n9f\" (UniqueName: \"kubernetes.io/projected/5317cca9-f317-4c21-9dad-a2f1ac629264-kube-api-access-d7n9f\") pod \"ingress-canary-f5djm\" (UID: \"5317cca9-f317-4c21-9dad-a2f1ac629264\") " pod="openshift-ingress-canary/ingress-canary-f5djm" Jun 11 21:16:16.737962 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.737937 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-f5djm" Jun 11 21:16:16.923656 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:16.923585 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-f5djm"] Jun 11 21:16:16.927730 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:16:16.927547 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5317cca9_f317_4c21_9dad_a2f1ac629264.slice/crio-006b2c844211ca220da917dde5e28355830d58f75213e4b489404fdfc708442a WatchSource:0}: Error finding container 006b2c844211ca220da917dde5e28355830d58f75213e4b489404fdfc708442a: Status 404 returned error can't find the container with id 006b2c844211ca220da917dde5e28355830d58f75213e4b489404fdfc708442a Jun 11 21:16:17.047170 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:17.047133 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:16:17.047170 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:17.047168 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:16:17.050600 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:17.050582 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 11 21:16:17.051295 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:17.051281 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9j4lb\"" Jun 11 21:16:17.051403 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:17.051387 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 11 21:16:17.151977 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:17.151940 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls\") pod \"dns-default-zpqjq\" (UID: \"2c0c15cb-361e-40de-83bf-06ccac34cc83\") " pod="openshift-dns/dns-default-zpqjq" Jun 11 21:16:17.152148 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:17.152051 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 21:16:17.152148 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:17.152105 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls podName:2c0c15cb-361e-40de-83bf-06ccac34cc83 nodeName:}" failed. No retries permitted until 2026-06-11 21:16:18.152090858 +0000 UTC m=+36.563414374 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls") pod "dns-default-zpqjq" (UID: "2c0c15cb-361e-40de-83bf-06ccac34cc83") : secret "dns-default-metrics-tls" not found Jun 11 21:16:17.232246 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:17.232141 2570 generic.go:356] "Generic (PLEG): container finished" podID="5678560c-39b0-4892-a879-3613ac253097" containerID="e6cbca995ae5f193838faeed8ba1b87572d0cb7664e857fd9c18eeb7572b71e4" exitCode=0 Jun 11 21:16:17.232246 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:17.232215 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w26g9" event={"ID":"5678560c-39b0-4892-a879-3613ac253097","Type":"ContainerDied","Data":"e6cbca995ae5f193838faeed8ba1b87572d0cb7664e857fd9c18eeb7572b71e4"} Jun 11 21:16:17.233370 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:17.233347 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-f5djm" event={"ID":"5317cca9-f317-4c21-9dad-a2f1ac629264","Type":"ContainerStarted","Data":"006b2c844211ca220da917dde5e28355830d58f75213e4b489404fdfc708442a"} Jun 11 21:16:18.046692 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:18.046617 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:16:18.049338 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:18.049301 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-v8l2j\"" Jun 11 21:16:18.050080 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:18.050048 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 11 21:16:18.050080 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:18.050065 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 11 21:16:18.159015 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:18.158982 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls\") pod \"dns-default-zpqjq\" (UID: \"2c0c15cb-361e-40de-83bf-06ccac34cc83\") " pod="openshift-dns/dns-default-zpqjq" Jun 11 21:16:18.159429 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:18.159099 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 21:16:18.159429 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:18.159162 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls podName:2c0c15cb-361e-40de-83bf-06ccac34cc83 nodeName:}" failed. No retries permitted until 2026-06-11 21:16:20.15914829 +0000 UTC m=+38.570471807 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls") pod "dns-default-zpqjq" (UID: "2c0c15cb-361e-40de-83bf-06ccac34cc83") : secret "dns-default-metrics-tls" not found Jun 11 21:16:18.238558 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:18.238520 2570 generic.go:356] "Generic (PLEG): container finished" podID="5678560c-39b0-4892-a879-3613ac253097" containerID="bda9a3da9ef162b01f125c7e38eb402ad954f8b5508ed18f126839f136f7ac75" exitCode=0 Jun 11 21:16:18.238751 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:18.238583 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w26g9" event={"ID":"5678560c-39b0-4892-a879-3613ac253097","Type":"ContainerDied","Data":"bda9a3da9ef162b01f125c7e38eb402ad954f8b5508ed18f126839f136f7ac75"} Jun 11 21:16:19.247149 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:19.247121 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w26g9" event={"ID":"5678560c-39b0-4892-a879-3613ac253097","Type":"ContainerStarted","Data":"3f7195138be363f398345b0a799c39e28b384f603b54307a2aabec4ce975b35d"} Jun 11 21:16:19.272528 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:19.272485 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-w26g9" podStartSLOduration=7.651307118 podStartE2EDuration="37.272466562s" totalImagesPullingTime="29.621159444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:15:42 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:16:19.271654941 +0000 UTC m=+37.682978515" watchObservedRunningTime="2026-06-11 21:16:19.272466562 +0000 UTC m=+37.683790100" Jun 11 21:16:20.174638 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:20.174598 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls\") pod \"dns-default-zpqjq\" (UID: \"2c0c15cb-361e-40de-83bf-06ccac34cc83\") " pod="openshift-dns/dns-default-zpqjq" Jun 11 21:16:20.174813 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:20.174703 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 21:16:20.174813 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:20.174752 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls podName:2c0c15cb-361e-40de-83bf-06ccac34cc83 nodeName:}" failed. No retries permitted until 2026-06-11 21:16:24.174738358 +0000 UTC m=+42.586061874 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls") pod "dns-default-zpqjq" (UID: "2c0c15cb-361e-40de-83bf-06ccac34cc83") : secret "dns-default-metrics-tls" not found Jun 11 21:16:20.251315 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:20.251272 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-f5djm" event={"ID":"5317cca9-f317-4c21-9dad-a2f1ac629264","Type":"ContainerStarted","Data":"201bc317178dcfb18a69148bdc7eff2930f8d315fbf760bb4e827b3972f40cb7"} Jun 11 21:16:20.267409 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:20.267176 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-f5djm" podStartSLOduration=1.9737872680000002 podStartE2EDuration="4.267159434s" totalImagesPullingTime="2.293372166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:16:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:16:20.266919978 +0000 UTC m=+38.678243528" watchObservedRunningTime="2026-06-11 21:16:20.267159434 +0000 UTC m=+38.678482976" Jun 11 21:16:24.202040 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:24.201998 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls\") pod \"dns-default-zpqjq\" (UID: \"2c0c15cb-361e-40de-83bf-06ccac34cc83\") " pod="openshift-dns/dns-default-zpqjq" Jun 11 21:16:24.202419 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:24.202111 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 21:16:24.202419 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:24.202160 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls podName:2c0c15cb-361e-40de-83bf-06ccac34cc83 nodeName:}" failed. No retries permitted until 2026-06-11 21:16:32.202147791 +0000 UTC m=+50.613471312 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls") pod "dns-default-zpqjq" (UID: "2c0c15cb-361e-40de-83bf-06ccac34cc83") : secret "dns-default-metrics-tls" not found Jun 11 21:16:32.260256 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:32.260210 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls\") pod \"dns-default-zpqjq\" (UID: \"2c0c15cb-361e-40de-83bf-06ccac34cc83\") " pod="openshift-dns/dns-default-zpqjq" Jun 11 21:16:32.260732 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:32.260334 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 21:16:32.260732 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:32.260410 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls podName:2c0c15cb-361e-40de-83bf-06ccac34cc83 nodeName:}" failed. No retries permitted until 2026-06-11 21:16:48.260395673 +0000 UTC m=+66.671719190 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls") pod "dns-default-zpqjq" (UID: "2c0c15cb-361e-40de-83bf-06ccac34cc83") : secret "dns-default-metrics-tls" not found Jun 11 21:16:33.792148 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:33.792087 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-676bc8cf94-w5rzn"] Jun 11 21:16:33.813864 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:33.813828 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-676bc8cf94-w5rzn"] Jun 11 21:16:33.813864 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:33.813870 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5"] Jun 11 21:16:33.814054 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:33.813950 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-676bc8cf94-w5rzn" Jun 11 21:16:33.816675 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:33.816652 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 11 21:16:33.816675 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:33.816664 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 11 21:16:33.816872 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:33.816655 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 11 21:16:33.817577 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:33.817545 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jun 11 21:16:33.818155 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:33.818127 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-zxztj\"" Jun 11 21:16:33.829801 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:33.829779 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5"] Jun 11 21:16:33.829801 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:33.829799 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" Jun 11 21:16:33.832236 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:33.832209 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jun 11 21:16:33.832341 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:33.832267 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jun 11 21:16:33.832341 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:33.832270 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jun 11 21:16:33.832341 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:33.832269 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jun 11 21:16:33.975254 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:33.975213 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e2e186ae-83b2-47b3-9c3c-9e11cc0801eb-hub\") pod \"cluster-proxy-proxy-agent-5b498c6f65-xwdn5\" (UID: \"e2e186ae-83b2-47b3-9c3c-9e11cc0801eb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" Jun 11 21:16:33.975435 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:33.975260 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2e186ae-83b2-47b3-9c3c-9e11cc0801eb-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5b498c6f65-xwdn5\" (UID: \"e2e186ae-83b2-47b3-9c3c-9e11cc0801eb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" Jun 11 21:16:33.975435 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:33.975288 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e2e186ae-83b2-47b3-9c3c-9e11cc0801eb-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5b498c6f65-xwdn5\" (UID: \"e2e186ae-83b2-47b3-9c3c-9e11cc0801eb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" Jun 11 21:16:33.975435 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:33.975320 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnsb8\" (UniqueName: \"kubernetes.io/projected/e2e186ae-83b2-47b3-9c3c-9e11cc0801eb-kube-api-access-wnsb8\") pod \"cluster-proxy-proxy-agent-5b498c6f65-xwdn5\" (UID: \"e2e186ae-83b2-47b3-9c3c-9e11cc0801eb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" Jun 11 21:16:33.975435 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:33.975346 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36a7a121-ce2f-47db-bacb-5f4c1d32e6fd-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-676bc8cf94-w5rzn\" (UID: \"36a7a121-ce2f-47db-bacb-5f4c1d32e6fd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-676bc8cf94-w5rzn" Jun 11 21:16:33.975435 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:33.975392 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgw8q\" (UniqueName: \"kubernetes.io/projected/36a7a121-ce2f-47db-bacb-5f4c1d32e6fd-kube-api-access-jgw8q\") pod \"managed-serviceaccount-addon-agent-676bc8cf94-w5rzn\" (UID: \"36a7a121-ce2f-47db-bacb-5f4c1d32e6fd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-676bc8cf94-w5rzn" Jun 11 21:16:33.975625 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:33.975440 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2e186ae-83b2-47b3-9c3c-9e11cc0801eb-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5b498c6f65-xwdn5\" (UID: \"e2e186ae-83b2-47b3-9c3c-9e11cc0801eb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" Jun 11 21:16:33.975625 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:33.975468 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e2e186ae-83b2-47b3-9c3c-9e11cc0801eb-ca\") pod \"cluster-proxy-proxy-agent-5b498c6f65-xwdn5\" (UID: \"e2e186ae-83b2-47b3-9c3c-9e11cc0801eb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" Jun 11 21:16:34.076866 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:34.076775 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e2e186ae-83b2-47b3-9c3c-9e11cc0801eb-ca\") pod \"cluster-proxy-proxy-agent-5b498c6f65-xwdn5\" (UID: \"e2e186ae-83b2-47b3-9c3c-9e11cc0801eb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" Jun 11 21:16:34.076866 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:34.076832 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e2e186ae-83b2-47b3-9c3c-9e11cc0801eb-hub\") pod \"cluster-proxy-proxy-agent-5b498c6f65-xwdn5\" (UID: \"e2e186ae-83b2-47b3-9c3c-9e11cc0801eb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" Jun 11 21:16:34.077046 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:34.077011 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2e186ae-83b2-47b3-9c3c-9e11cc0801eb-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5b498c6f65-xwdn5\" (UID: \"e2e186ae-83b2-47b3-9c3c-9e11cc0801eb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" Jun 11 21:16:34.077046 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:34.077038 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e2e186ae-83b2-47b3-9c3c-9e11cc0801eb-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5b498c6f65-xwdn5\" (UID: \"e2e186ae-83b2-47b3-9c3c-9e11cc0801eb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" Jun 11 21:16:34.077113 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:34.077066 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnsb8\" (UniqueName: \"kubernetes.io/projected/e2e186ae-83b2-47b3-9c3c-9e11cc0801eb-kube-api-access-wnsb8\") pod \"cluster-proxy-proxy-agent-5b498c6f65-xwdn5\" (UID: \"e2e186ae-83b2-47b3-9c3c-9e11cc0801eb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" Jun 11 21:16:34.077113 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:34.077092 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36a7a121-ce2f-47db-bacb-5f4c1d32e6fd-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-676bc8cf94-w5rzn\" (UID: \"36a7a121-ce2f-47db-bacb-5f4c1d32e6fd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-676bc8cf94-w5rzn" Jun 11 21:16:34.077208 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:34.077120 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgw8q\" (UniqueName: \"kubernetes.io/projected/36a7a121-ce2f-47db-bacb-5f4c1d32e6fd-kube-api-access-jgw8q\") pod \"managed-serviceaccount-addon-agent-676bc8cf94-w5rzn\" (UID: \"36a7a121-ce2f-47db-bacb-5f4c1d32e6fd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-676bc8cf94-w5rzn" Jun 11 21:16:34.077208 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:34.077164 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2e186ae-83b2-47b3-9c3c-9e11cc0801eb-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5b498c6f65-xwdn5\" (UID: \"e2e186ae-83b2-47b3-9c3c-9e11cc0801eb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" Jun 11 21:16:34.078200 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:34.077914 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2e186ae-83b2-47b3-9c3c-9e11cc0801eb-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5b498c6f65-xwdn5\" (UID: \"e2e186ae-83b2-47b3-9c3c-9e11cc0801eb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" Jun 11 21:16:34.081185 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:34.081159 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e2e186ae-83b2-47b3-9c3c-9e11cc0801eb-ca\") pod \"cluster-proxy-proxy-agent-5b498c6f65-xwdn5\" (UID: \"e2e186ae-83b2-47b3-9c3c-9e11cc0801eb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" Jun 11 21:16:34.081304 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:34.081201 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e2e186ae-83b2-47b3-9c3c-9e11cc0801eb-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5b498c6f65-xwdn5\" (UID: \"e2e186ae-83b2-47b3-9c3c-9e11cc0801eb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" Jun 11 21:16:34.081304 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:34.081266 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e2e186ae-83b2-47b3-9c3c-9e11cc0801eb-hub\") pod \"cluster-proxy-proxy-agent-5b498c6f65-xwdn5\" (UID: \"e2e186ae-83b2-47b3-9c3c-9e11cc0801eb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" Jun 11 21:16:34.081304 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:34.081287 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36a7a121-ce2f-47db-bacb-5f4c1d32e6fd-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-676bc8cf94-w5rzn\" (UID: \"36a7a121-ce2f-47db-bacb-5f4c1d32e6fd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-676bc8cf94-w5rzn" Jun 11 21:16:34.081460 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:34.081289 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2e186ae-83b2-47b3-9c3c-9e11cc0801eb-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5b498c6f65-xwdn5\" (UID: \"e2e186ae-83b2-47b3-9c3c-9e11cc0801eb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" Jun 11 21:16:34.084840 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:34.084813 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnsb8\" (UniqueName: \"kubernetes.io/projected/e2e186ae-83b2-47b3-9c3c-9e11cc0801eb-kube-api-access-wnsb8\") pod \"cluster-proxy-proxy-agent-5b498c6f65-xwdn5\" (UID: \"e2e186ae-83b2-47b3-9c3c-9e11cc0801eb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" Jun 11 21:16:34.085011 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:34.084995 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgw8q\" (UniqueName: \"kubernetes.io/projected/36a7a121-ce2f-47db-bacb-5f4c1d32e6fd-kube-api-access-jgw8q\") pod \"managed-serviceaccount-addon-agent-676bc8cf94-w5rzn\" (UID: \"36a7a121-ce2f-47db-bacb-5f4c1d32e6fd\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-676bc8cf94-w5rzn" Jun 11 21:16:34.139905 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:34.139866 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-676bc8cf94-w5rzn" Jun 11 21:16:34.147037 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:34.147002 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" Jun 11 21:16:34.311675 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:34.311636 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-676bc8cf94-w5rzn"] Jun 11 21:16:34.315461 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:16:34.315431 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36a7a121_ce2f_47db_bacb_5f4c1d32e6fd.slice/crio-3046f7b6582d64d2f5fe854769387bec25bb36156785f4be8ed3afa09835d8f4 WatchSource:0}: Error finding container 3046f7b6582d64d2f5fe854769387bec25bb36156785f4be8ed3afa09835d8f4: Status 404 returned error can't find the container with id 3046f7b6582d64d2f5fe854769387bec25bb36156785f4be8ed3afa09835d8f4 Jun 11 21:16:34.325940 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:34.325912 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5"] Jun 11 21:16:34.329132 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:16:34.329111 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2e186ae_83b2_47b3_9c3c_9e11cc0801eb.slice/crio-b14fd37bcc958ec6ef0cc14bbbc4c1b021385e6297db104d07adb16e0dc517ec WatchSource:0}: Error finding container b14fd37bcc958ec6ef0cc14bbbc4c1b021385e6297db104d07adb16e0dc517ec: Status 404 returned error can't find the container with id b14fd37bcc958ec6ef0cc14bbbc4c1b021385e6297db104d07adb16e0dc517ec Jun 11 21:16:35.290353 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:35.290312 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-676bc8cf94-w5rzn" event={"ID":"36a7a121-ce2f-47db-bacb-5f4c1d32e6fd","Type":"ContainerStarted","Data":"3046f7b6582d64d2f5fe854769387bec25bb36156785f4be8ed3afa09835d8f4"} Jun 11 21:16:35.292727 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:35.292694 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" event={"ID":"e2e186ae-83b2-47b3-9c3c-9e11cc0801eb","Type":"ContainerStarted","Data":"b14fd37bcc958ec6ef0cc14bbbc4c1b021385e6297db104d07adb16e0dc517ec"} Jun 11 21:16:38.303497 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:38.303387 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-676bc8cf94-w5rzn" event={"ID":"36a7a121-ce2f-47db-bacb-5f4c1d32e6fd","Type":"ContainerStarted","Data":"e71302f2cda4ae876157685dad679b7d3deded99fc756b7da636275d00b84fff"} Jun 11 21:16:38.305311 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:38.305286 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" event={"ID":"e2e186ae-83b2-47b3-9c3c-9e11cc0801eb","Type":"ContainerStarted","Data":"07f5c7e516d987dadd538f8b56cd2121a7701ef2f3dd29850b4af6f1b446f143"} Jun 11 21:16:38.305409 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:38.305326 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" event={"ID":"e2e186ae-83b2-47b3-9c3c-9e11cc0801eb","Type":"ContainerStarted","Data":"b6c5a1d4e112f49acbe98b564f38bfc0fdef21a1d6cadf7f844eee3ee389ac40"} Jun 11 21:16:38.305409 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:38.305341 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" event={"ID":"e2e186ae-83b2-47b3-9c3c-9e11cc0801eb","Type":"ContainerStarted","Data":"1bca219881dbc404df89055b3d7f25fdb6326087f80041f5102d7fc449aa1d29"} Jun 11 21:16:38.317877 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:38.317826 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-676bc8cf94-w5rzn" podStartSLOduration=1.64352195 podStartE2EDuration="5.31781077s" totalImagesPullingTime="3.67428882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:16:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:16:38.31747979 +0000 UTC m=+56.728803333" watchObservedRunningTime="2026-06-11 21:16:38.31781077 +0000 UTC m=+56.729134308" Jun 11 21:16:38.335710 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:38.335643 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" podStartSLOduration=1.662518964 podStartE2EDuration="5.335626254s" totalImagesPullingTime="3.67310729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:16:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:16:38.334632925 +0000 UTC m=+56.745956461" watchObservedRunningTime="2026-06-11 21:16:38.335626254 +0000 UTC m=+56.746949836" Jun 11 21:16:42.229138 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:42.229108 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b4gmk" Jun 11 21:16:47.765431 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:47.765389 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2xdc\" (UniqueName: \"kubernetes.io/projected/54688996-c5d4-445d-a6ff-4997a19fe1d6-kube-api-access-g2xdc\") pod \"network-check-target-shxrd\" (UID: \"54688996-c5d4-445d-a6ff-4997a19fe1d6\") " pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:16:47.769397 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:47.769377 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 11 21:16:47.778830 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:47.778809 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 11 21:16:47.789302 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:47.789267 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2xdc\" (UniqueName: \"kubernetes.io/projected/54688996-c5d4-445d-a6ff-4997a19fe1d6-kube-api-access-g2xdc\") pod \"network-check-target-shxrd\" (UID: \"54688996-c5d4-445d-a6ff-4997a19fe1d6\") " pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:16:47.865752 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:47.865709 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs\") pod \"network-metrics-daemon-2bmhl\" (UID: \"329e0949-4197-4a29-afd8-1300e76b75cf\") " pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:16:47.865752 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:47.865753 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f4a1fd8-505b-4061-a432-620cb3c68386-original-pull-secret\") pod \"global-pull-secret-syncer-4c247\" (UID: \"6f4a1fd8-505b-4061-a432-620cb3c68386\") " pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:16:47.868254 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:47.868227 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 11 21:16:47.868348 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:47.868305 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 11 21:16:47.876095 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:47.876077 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 11 21:16:47.876175 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:47.876159 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs podName:329e0949-4197-4a29-afd8-1300e76b75cf nodeName:}" failed. No retries permitted until 2026-06-11 21:17:51.876135859 +0000 UTC m=+130.287459378 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs") pod "network-metrics-daemon-2bmhl" (UID: "329e0949-4197-4a29-afd8-1300e76b75cf") : secret "metrics-daemon-secret" not found Jun 11 21:16:47.878341 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:47.878322 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f4a1fd8-505b-4061-a432-620cb3c68386-original-pull-secret\") pod \"global-pull-secret-syncer-4c247\" (UID: \"6f4a1fd8-505b-4061-a432-620cb3c68386\") " pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:16:47.962580 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:47.962526 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4c247" Jun 11 21:16:48.057818 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:48.057787 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-v8l2j\"" Jun 11 21:16:48.066454 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:48.066032 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:16:48.125187 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:48.125157 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4c247"] Jun 11 21:16:48.129161 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:16:48.129123 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f4a1fd8_505b_4061_a432_620cb3c68386.slice/crio-5e68df2bf847d1e3501db56bf1543c19568a61ece00ee6393785997e8beba8ae WatchSource:0}: Error finding container 5e68df2bf847d1e3501db56bf1543c19568a61ece00ee6393785997e8beba8ae: Status 404 returned error can't find the container with id 5e68df2bf847d1e3501db56bf1543c19568a61ece00ee6393785997e8beba8ae Jun 11 21:16:48.206399 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:48.206360 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-shxrd"] Jun 11 21:16:48.209665 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:16:48.209637 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54688996_c5d4_445d_a6ff_4997a19fe1d6.slice/crio-710d7e219873a356b3013f304ae2728edc40888f6d0987f521bfdde1502fa611 WatchSource:0}: Error finding container 710d7e219873a356b3013f304ae2728edc40888f6d0987f521bfdde1502fa611: Status 404 returned error can't find the container with id 710d7e219873a356b3013f304ae2728edc40888f6d0987f521bfdde1502fa611 Jun 11 21:16:48.269219 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:48.269186 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls\") pod \"dns-default-zpqjq\" (UID: \"2c0c15cb-361e-40de-83bf-06ccac34cc83\") " pod="openshift-dns/dns-default-zpqjq" Jun 11 21:16:48.269335 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:48.269300 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 21:16:48.269371 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:16:48.269354 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls podName:2c0c15cb-361e-40de-83bf-06ccac34cc83 nodeName:}" failed. No retries permitted until 2026-06-11 21:17:20.269338657 +0000 UTC m=+98.680662174 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls") pod "dns-default-zpqjq" (UID: "2c0c15cb-361e-40de-83bf-06ccac34cc83") : secret "dns-default-metrics-tls" not found Jun 11 21:16:48.331382 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:48.331307 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-shxrd" event={"ID":"54688996-c5d4-445d-a6ff-4997a19fe1d6","Type":"ContainerStarted","Data":"710d7e219873a356b3013f304ae2728edc40888f6d0987f521bfdde1502fa611"} Jun 11 21:16:48.332432 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:48.332405 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4c247" event={"ID":"6f4a1fd8-505b-4061-a432-620cb3c68386","Type":"ContainerStarted","Data":"5e68df2bf847d1e3501db56bf1543c19568a61ece00ee6393785997e8beba8ae"} Jun 11 21:16:53.349488 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:53.349449 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-shxrd" event={"ID":"54688996-c5d4-445d-a6ff-4997a19fe1d6","Type":"ContainerStarted","Data":"5f2fbc2ec487ac919ac9f45c75696fdf7adcef0609d16e65ef412e41c9802d8c"} Jun 11 21:16:53.349967 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:53.349593 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:16:53.350951 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:53.350929 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4c247" event={"ID":"6f4a1fd8-505b-4061-a432-620cb3c68386","Type":"ContainerStarted","Data":"08d990d6e568f7dbf103a9d8b881473317f13d3cbadfe2bafcf8af3b6968cc48"} Jun 11 21:16:53.365011 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:53.364960 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-shxrd" podStartSLOduration=66.816846462 podStartE2EDuration="1m11.364945679s" totalImagesPullingTime="4.548099217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:16:53.364426025 +0000 UTC m=+71.775749564" watchObservedRunningTime="2026-06-11 21:16:53.364945679 +0000 UTC m=+71.776269217" Jun 11 21:16:53.378686 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:16:53.378640 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4c247" podStartSLOduration=66.74647832 podStartE2EDuration="1m11.378627629s" totalImagesPullingTime="4.632149309s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:16:53.378278997 +0000 UTC m=+71.789602534" watchObservedRunningTime="2026-06-11 21:16:53.378627629 +0000 UTC m=+71.789951166" Jun 11 21:17:20.288265 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:17:20.288230 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls\") pod \"dns-default-zpqjq\" (UID: \"2c0c15cb-361e-40de-83bf-06ccac34cc83\") " pod="openshift-dns/dns-default-zpqjq" Jun 11 21:17:20.288593 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:17:20.288346 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 21:17:20.288593 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:17:20.288407 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls podName:2c0c15cb-361e-40de-83bf-06ccac34cc83 nodeName:}" failed. No retries permitted until 2026-06-11 21:18:24.288393671 +0000 UTC m=+162.699717186 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls") pod "dns-default-zpqjq" (UID: "2c0c15cb-361e-40de-83bf-06ccac34cc83") : secret "dns-default-metrics-tls" not found Jun 11 21:17:24.356323 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:17:24.356289 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-shxrd" Jun 11 21:17:51.906289 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:17:51.906247 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs\") pod \"network-metrics-daemon-2bmhl\" (UID: \"329e0949-4197-4a29-afd8-1300e76b75cf\") " pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:17:51.906743 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:17:51.906401 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 11 21:17:51.906743 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:17:51.906473 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs podName:329e0949-4197-4a29-afd8-1300e76b75cf nodeName:}" failed. No retries permitted until 2026-06-11 21:19:53.906456439 +0000 UTC m=+252.317779954 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs") pod "network-metrics-daemon-2bmhl" (UID: "329e0949-4197-4a29-afd8-1300e76b75cf") : secret "metrics-daemon-secret" not found Jun 11 21:18:19.425857 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:18:19.425813 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-zpqjq" podUID="2c0c15cb-361e-40de-83bf-06ccac34cc83" Jun 11 21:18:19.597803 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:18:19.597769 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zpqjq" Jun 11 21:18:20.057402 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:18:20.057365 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-2bmhl" podUID="329e0949-4197-4a29-afd8-1300e76b75cf" Jun 11 21:18:24.330537 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:18:24.330483 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls\") pod \"dns-default-zpqjq\" (UID: \"2c0c15cb-361e-40de-83bf-06ccac34cc83\") " pod="openshift-dns/dns-default-zpqjq" Jun 11 21:18:24.330970 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:18:24.330652 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 21:18:24.330970 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:18:24.330723 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls podName:2c0c15cb-361e-40de-83bf-06ccac34cc83 nodeName:}" failed. No retries permitted until 2026-06-11 21:20:26.330706649 +0000 UTC m=+284.742030165 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls") pod "dns-default-zpqjq" (UID: "2c0c15cb-361e-40de-83bf-06ccac34cc83") : secret "dns-default-metrics-tls" not found Jun 11 21:18:33.046636 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:18:33.046504 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:18:38.652061 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:18:38.652030 2570 generic.go:356] "Generic (PLEG): container finished" podID="36a7a121-ce2f-47db-bacb-5f4c1d32e6fd" containerID="e71302f2cda4ae876157685dad679b7d3deded99fc756b7da636275d00b84fff" exitCode=255 Jun 11 21:18:38.652425 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:18:38.652062 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-676bc8cf94-w5rzn" event={"ID":"36a7a121-ce2f-47db-bacb-5f4c1d32e6fd","Type":"ContainerDied","Data":"e71302f2cda4ae876157685dad679b7d3deded99fc756b7da636275d00b84fff"} Jun 11 21:18:38.652425 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:18:38.652352 2570 scope.go:117] "RemoveContainer" containerID="e71302f2cda4ae876157685dad679b7d3deded99fc756b7da636275d00b84fff" Jun 11 21:18:39.656735 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:18:39.656702 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-676bc8cf94-w5rzn" event={"ID":"36a7a121-ce2f-47db-bacb-5f4c1d32e6fd","Type":"ContainerStarted","Data":"5874b27fefac09b92a7e7e657a2f0a3728b9ca40eacf16cd400203046730e145"} Jun 11 21:19:09.180629 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:09.180592 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lbtkn"] Jun 11 21:19:09.182453 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:09.182438 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:09.184886 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:09.184865 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 11 21:19:09.185719 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:09.185694 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9qj9z\"" Jun 11 21:19:09.185719 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:09.185706 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 11 21:19:09.185864 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:09.185737 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 11 21:19:09.186289 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:09.186273 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 11 21:19:09.202355 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:09.202329 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lbtkn"] Jun 11 21:19:09.247314 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:09.247279 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:09.247481 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:09.247320 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpz7g\" (UniqueName: \"kubernetes.io/projected/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-kube-api-access-wpz7g\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:09.247481 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:09.247358 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-crio-socket\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:09.247481 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:09.247374 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:09.247481 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:09.247394 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-data-volume\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:09.348695 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:09.348638 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpz7g\" (UniqueName: \"kubernetes.io/projected/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-kube-api-access-wpz7g\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:09.348695 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:09.348710 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-crio-socket\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:09.348952 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:09.348734 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:09.348952 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:09.348763 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-data-volume\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:09.348952 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:09.348810 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-crio-socket\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:09.348952 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:09.348816 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:09.348952 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:09.348862 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 11 21:19:09.348952 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:09.348928 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-kube-rbac-proxy-cm podName:3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1 nodeName:}" failed. No retries permitted until 2026-06-11 21:19:09.848911454 +0000 UTC m=+208.260234969 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lbtkn" (UID: "3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1") : configmap "kube-rbac-proxy" not found Jun 11 21:19:09.349277 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:09.348962 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 21:19:09.349277 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:09.349012 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-insights-runtime-extractor-tls podName:3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1 nodeName:}" failed. No retries permitted until 2026-06-11 21:19:09.848996457 +0000 UTC m=+208.260319996 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lbtkn" (UID: "3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1") : secret "insights-runtime-extractor-tls" not found Jun 11 21:19:09.349277 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:09.349225 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-data-volume\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:09.360165 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:09.360133 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpz7g\" (UniqueName: \"kubernetes.io/projected/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-kube-api-access-wpz7g\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:09.852215 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:09.852164 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:09.852398 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:09.852309 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:09.852398 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:09.852314 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 21:19:09.852398 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:09.852376 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 11 21:19:09.852398 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:09.852387 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-insights-runtime-extractor-tls podName:3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1 nodeName:}" failed. No retries permitted until 2026-06-11 21:19:10.852370834 +0000 UTC m=+209.263694350 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lbtkn" (UID: "3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1") : secret "insights-runtime-extractor-tls" not found Jun 11 21:19:09.852540 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:09.852423 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-kube-rbac-proxy-cm podName:3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1 nodeName:}" failed. No retries permitted until 2026-06-11 21:19:10.852412088 +0000 UTC m=+209.263735604 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lbtkn" (UID: "3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1") : configmap "kube-rbac-proxy" not found Jun 11 21:19:10.860460 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:10.860422 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:10.860884 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:10.860479 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:10.860884 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:10.860587 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 11 21:19:10.860884 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:10.860641 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-kube-rbac-proxy-cm podName:3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1 nodeName:}" failed. No retries permitted until 2026-06-11 21:19:12.860626851 +0000 UTC m=+211.271950367 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lbtkn" (UID: "3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1") : configmap "kube-rbac-proxy" not found Jun 11 21:19:10.860884 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:10.860587 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 21:19:10.860884 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:10.860716 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-insights-runtime-extractor-tls podName:3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1 nodeName:}" failed. No retries permitted until 2026-06-11 21:19:12.86070214 +0000 UTC m=+211.272025660 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lbtkn" (UID: "3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1") : secret "insights-runtime-extractor-tls" not found Jun 11 21:19:12.168573 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:12.168547 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s8b5p_bdfaf7d3-ae13-48e9-b9ea-040ae02d886a/dns-node-resolver/0.log" Jun 11 21:19:12.873980 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:12.873936 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:12.874183 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:12.874009 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:12.874183 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:12.874076 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 11 21:19:12.874183 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:12.874099 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 21:19:12.874183 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:12.874144 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-kube-rbac-proxy-cm podName:3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1 nodeName:}" failed. No retries permitted until 2026-06-11 21:19:16.874129664 +0000 UTC m=+215.285453180 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lbtkn" (UID: "3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1") : configmap "kube-rbac-proxy" not found Jun 11 21:19:12.874183 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:12.874161 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-insights-runtime-extractor-tls podName:3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1 nodeName:}" failed. No retries permitted until 2026-06-11 21:19:16.874152033 +0000 UTC m=+215.285475548 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lbtkn" (UID: "3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1") : secret "insights-runtime-extractor-tls" not found Jun 11 21:19:13.172767 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:13.172693 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-thgw8_7b88bd69-b4df-4345-a24c-e51adeba9023/node-ca/0.log" Jun 11 21:19:13.781048 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:13.781021 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-f5djm_5317cca9-f317-4c21-9dad-a2f1ac629264/serve-healthcheck-canary/0.log" Jun 11 21:19:16.906490 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:16.906449 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:16.906930 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:16.906510 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:16.906930 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:16.906602 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 11 21:19:16.906930 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:16.906643 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 11 21:19:16.906930 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:16.906668 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-kube-rbac-proxy-cm podName:3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1 nodeName:}" failed. No retries permitted until 2026-06-11 21:19:24.906654154 +0000 UTC m=+223.317977682 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lbtkn" (UID: "3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1") : configmap "kube-rbac-proxy" not found Jun 11 21:19:16.906930 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:16.906692 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-insights-runtime-extractor-tls podName:3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1 nodeName:}" failed. No retries permitted until 2026-06-11 21:19:24.90667963 +0000 UTC m=+223.318003147 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lbtkn" (UID: "3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1") : secret "insights-runtime-extractor-tls" not found Jun 11 21:19:24.973309 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:24.973256 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:24.973309 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:24.973327 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:24.973823 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:24.973803 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:24.975539 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:24.975520 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lbtkn\" (UID: \"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1\") " pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:25.093128 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:25.093084 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lbtkn" Jun 11 21:19:25.212360 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:25.212096 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lbtkn"] Jun 11 21:19:25.214935 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:19:25.214912 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3efb29bd_d2e2_4d55_bcd3_039c4fbf83d1.slice/crio-c65bf22a38331699cde183c82fe714eb4adf95da88d723c3255f167a0ec58003 WatchSource:0}: Error finding container c65bf22a38331699cde183c82fe714eb4adf95da88d723c3255f167a0ec58003: Status 404 returned error can't find the container with id c65bf22a38331699cde183c82fe714eb4adf95da88d723c3255f167a0ec58003 Jun 11 21:19:25.786267 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:25.786226 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lbtkn" event={"ID":"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1","Type":"ContainerStarted","Data":"c65bf22a38331699cde183c82fe714eb4adf95da88d723c3255f167a0ec58003"} Jun 11 21:19:28.797134 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:28.797105 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lbtkn" event={"ID":"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1","Type":"ContainerStarted","Data":"6683accac40a013337ab407c0529ca07847e630cad68ebb56181c35f8daf7664"} Jun 11 21:19:29.802183 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:29.802144 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lbtkn" event={"ID":"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1","Type":"ContainerStarted","Data":"90f11598b0dd207796fc608a0da90a31c10ae15b73fd6239ec81fb94e172eda8"} Jun 11 21:19:31.820978 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:31.820941 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d87ff458-g7pl5"] Jun 11 21:19:31.827432 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:31.827399 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d87ff458-g7pl5" Jun 11 21:19:31.830241 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:31.830217 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 11 21:19:31.831246 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:31.831204 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 11 21:19:31.831360 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:31.831240 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 11 21:19:31.831360 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:31.831297 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 11 21:19:31.831543 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:31.831527 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-z2vmg\"" Jun 11 21:19:31.831677 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:31.831549 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 11 21:19:31.831677 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:31.831595 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 11 21:19:31.831677 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:31.831633 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 11 21:19:31.839097 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:31.839075 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d87ff458-g7pl5"] Jun 11 21:19:31.926744 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:31.926705 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82296dc1-4d45-4b57-bb06-50853f80ccf8-console-serving-cert\") pod \"console-d87ff458-g7pl5\" (UID: \"82296dc1-4d45-4b57-bb06-50853f80ccf8\") " pod="openshift-console/console-d87ff458-g7pl5" Jun 11 21:19:31.926907 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:31.926771 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82296dc1-4d45-4b57-bb06-50853f80ccf8-oauth-serving-cert\") pod \"console-d87ff458-g7pl5\" (UID: \"82296dc1-4d45-4b57-bb06-50853f80ccf8\") " pod="openshift-console/console-d87ff458-g7pl5" Jun 11 21:19:31.926907 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:31.926793 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqp9q\" (UniqueName: \"kubernetes.io/projected/82296dc1-4d45-4b57-bb06-50853f80ccf8-kube-api-access-vqp9q\") pod \"console-d87ff458-g7pl5\" (UID: \"82296dc1-4d45-4b57-bb06-50853f80ccf8\") " pod="openshift-console/console-d87ff458-g7pl5" Jun 11 21:19:31.927020 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:31.926903 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82296dc1-4d45-4b57-bb06-50853f80ccf8-console-oauth-config\") pod \"console-d87ff458-g7pl5\" (UID: \"82296dc1-4d45-4b57-bb06-50853f80ccf8\") " pod="openshift-console/console-d87ff458-g7pl5" Jun 11 21:19:31.927020 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:31.926939 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82296dc1-4d45-4b57-bb06-50853f80ccf8-console-config\") pod \"console-d87ff458-g7pl5\" (UID: \"82296dc1-4d45-4b57-bb06-50853f80ccf8\") " pod="openshift-console/console-d87ff458-g7pl5" Jun 11 21:19:31.927020 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:31.926978 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82296dc1-4d45-4b57-bb06-50853f80ccf8-service-ca\") pod \"console-d87ff458-g7pl5\" (UID: \"82296dc1-4d45-4b57-bb06-50853f80ccf8\") " pod="openshift-console/console-d87ff458-g7pl5" Jun 11 21:19:32.027864 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:32.027828 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82296dc1-4d45-4b57-bb06-50853f80ccf8-console-oauth-config\") pod \"console-d87ff458-g7pl5\" (UID: \"82296dc1-4d45-4b57-bb06-50853f80ccf8\") " pod="openshift-console/console-d87ff458-g7pl5" Jun 11 21:19:32.027864 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:32.027869 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82296dc1-4d45-4b57-bb06-50853f80ccf8-console-config\") pod \"console-d87ff458-g7pl5\" (UID: \"82296dc1-4d45-4b57-bb06-50853f80ccf8\") " pod="openshift-console/console-d87ff458-g7pl5" Jun 11 21:19:32.028085 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:32.027893 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82296dc1-4d45-4b57-bb06-50853f80ccf8-service-ca\") pod \"console-d87ff458-g7pl5\" (UID: \"82296dc1-4d45-4b57-bb06-50853f80ccf8\") " pod="openshift-console/console-d87ff458-g7pl5" Jun 11 21:19:32.028085 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:32.027915 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82296dc1-4d45-4b57-bb06-50853f80ccf8-console-serving-cert\") pod \"console-d87ff458-g7pl5\" (UID: \"82296dc1-4d45-4b57-bb06-50853f80ccf8\") " pod="openshift-console/console-d87ff458-g7pl5" Jun 11 21:19:32.028085 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:32.027946 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82296dc1-4d45-4b57-bb06-50853f80ccf8-oauth-serving-cert\") pod \"console-d87ff458-g7pl5\" (UID: \"82296dc1-4d45-4b57-bb06-50853f80ccf8\") " pod="openshift-console/console-d87ff458-g7pl5" Jun 11 21:19:32.028085 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:32.027967 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqp9q\" (UniqueName: \"kubernetes.io/projected/82296dc1-4d45-4b57-bb06-50853f80ccf8-kube-api-access-vqp9q\") pod \"console-d87ff458-g7pl5\" (UID: \"82296dc1-4d45-4b57-bb06-50853f80ccf8\") " pod="openshift-console/console-d87ff458-g7pl5" Jun 11 21:19:32.028560 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:32.028530 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82296dc1-4d45-4b57-bb06-50853f80ccf8-oauth-serving-cert\") pod \"console-d87ff458-g7pl5\" (UID: \"82296dc1-4d45-4b57-bb06-50853f80ccf8\") " pod="openshift-console/console-d87ff458-g7pl5" Jun 11 21:19:32.028699 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:32.028624 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82296dc1-4d45-4b57-bb06-50853f80ccf8-service-ca\") pod \"console-d87ff458-g7pl5\" (UID: \"82296dc1-4d45-4b57-bb06-50853f80ccf8\") " pod="openshift-console/console-d87ff458-g7pl5" Jun 11 21:19:32.028699 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:32.028634 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82296dc1-4d45-4b57-bb06-50853f80ccf8-console-config\") pod \"console-d87ff458-g7pl5\" (UID: \"82296dc1-4d45-4b57-bb06-50853f80ccf8\") " pod="openshift-console/console-d87ff458-g7pl5" Jun 11 21:19:32.030301 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:32.030271 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82296dc1-4d45-4b57-bb06-50853f80ccf8-console-oauth-config\") pod \"console-d87ff458-g7pl5\" (UID: \"82296dc1-4d45-4b57-bb06-50853f80ccf8\") " pod="openshift-console/console-d87ff458-g7pl5" Jun 11 21:19:32.030469 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:32.030452 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82296dc1-4d45-4b57-bb06-50853f80ccf8-console-serving-cert\") pod \"console-d87ff458-g7pl5\" (UID: \"82296dc1-4d45-4b57-bb06-50853f80ccf8\") " pod="openshift-console/console-d87ff458-g7pl5" Jun 11 21:19:32.041755 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:32.041732 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqp9q\" (UniqueName: \"kubernetes.io/projected/82296dc1-4d45-4b57-bb06-50853f80ccf8-kube-api-access-vqp9q\") pod \"console-d87ff458-g7pl5\" (UID: \"82296dc1-4d45-4b57-bb06-50853f80ccf8\") " pod="openshift-console/console-d87ff458-g7pl5" Jun 11 21:19:32.138510 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:32.138480 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d87ff458-g7pl5" Jun 11 21:19:32.259021 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:32.258981 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d87ff458-g7pl5"] Jun 11 21:19:32.261449 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:19:32.261418 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82296dc1_4d45_4b57_bb06_50853f80ccf8.slice/crio-b1be7ea7c90b8be1fa1d4e19c61d4f4bb199d5109ec79f7f362ee40365d65723 WatchSource:0}: Error finding container b1be7ea7c90b8be1fa1d4e19c61d4f4bb199d5109ec79f7f362ee40365d65723: Status 404 returned error can't find the container with id b1be7ea7c90b8be1fa1d4e19c61d4f4bb199d5109ec79f7f362ee40365d65723 Jun 11 21:19:32.814592 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:32.814508 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d87ff458-g7pl5" event={"ID":"82296dc1-4d45-4b57-bb06-50853f80ccf8","Type":"ContainerStarted","Data":"b1be7ea7c90b8be1fa1d4e19c61d4f4bb199d5109ec79f7f362ee40365d65723"} Jun 11 21:19:32.816758 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:32.816727 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lbtkn" event={"ID":"3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1","Type":"ContainerStarted","Data":"2dcac8c5d856b66f9d4f23d6d01a986bb8e3f0b4a3ae42c1fc321515d050617b"} Jun 11 21:19:32.835515 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:32.835466 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lbtkn" podStartSLOduration=17.116689394 podStartE2EDuration="23.835449984s" totalImagesPullingTime="6.71876059s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:19:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:19:32.83352815 +0000 UTC m=+231.244851721" watchObservedRunningTime="2026-06-11 21:19:32.835449984 +0000 UTC m=+231.246773521" Jun 11 21:19:35.827373 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:35.827340 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d87ff458-g7pl5" event={"ID":"82296dc1-4d45-4b57-bb06-50853f80ccf8","Type":"ContainerStarted","Data":"760c708d8b624b565e786e0528646b662c82b8d01758afd46c17ccf48c29504f"} Jun 11 21:19:35.845094 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:35.845047 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d87ff458-g7pl5" podStartSLOduration=2.08871838 podStartE2EDuration="4.845032944s" totalImagesPullingTime="2.756314564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:19:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:19:35.843549839 +0000 UTC m=+234.254873395" watchObservedRunningTime="2026-06-11 21:19:35.845032944 +0000 UTC m=+234.256356482" Jun 11 21:19:39.377448 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:39.377413 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7gxwt"] Jun 11 21:19:39.379393 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:39.379378 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7gxwt" Jun 11 21:19:39.388809 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:39.388789 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jun 11 21:19:39.388809 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:39.388800 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-t8jc2\"" Jun 11 21:19:39.399088 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:39.399063 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7gxwt"] Jun 11 21:19:39.484157 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:39.484116 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9d605903-7c76-48d9-9a1a-439b29742598-tls-certificates\") pod \"prometheus-operator-admission-webhook-5cf667749c-7gxwt\" (UID: \"9d605903-7c76-48d9-9a1a-439b29742598\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7gxwt" Jun 11 21:19:39.585461 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:39.585421 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9d605903-7c76-48d9-9a1a-439b29742598-tls-certificates\") pod \"prometheus-operator-admission-webhook-5cf667749c-7gxwt\" (UID: \"9d605903-7c76-48d9-9a1a-439b29742598\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7gxwt" Jun 11 21:19:39.585621 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:39.585555 2570 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jun 11 21:19:39.585686 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:39.585651 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d605903-7c76-48d9-9a1a-439b29742598-tls-certificates podName:9d605903-7c76-48d9-9a1a-439b29742598 nodeName:}" failed. No retries permitted until 2026-06-11 21:19:40.085630104 +0000 UTC m=+238.496953620 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/9d605903-7c76-48d9-9a1a-439b29742598-tls-certificates") pod "prometheus-operator-admission-webhook-5cf667749c-7gxwt" (UID: "9d605903-7c76-48d9-9a1a-439b29742598") : secret "prometheus-operator-admission-webhook-tls" not found Jun 11 21:19:40.089687 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:40.089657 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9d605903-7c76-48d9-9a1a-439b29742598-tls-certificates\") pod \"prometheus-operator-admission-webhook-5cf667749c-7gxwt\" (UID: \"9d605903-7c76-48d9-9a1a-439b29742598\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7gxwt" Jun 11 21:19:40.092118 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:40.092096 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9d605903-7c76-48d9-9a1a-439b29742598-tls-certificates\") pod \"prometheus-operator-admission-webhook-5cf667749c-7gxwt\" (UID: \"9d605903-7c76-48d9-9a1a-439b29742598\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7gxwt" Jun 11 21:19:40.287851 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:40.287812 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7gxwt" Jun 11 21:19:40.414986 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:40.411610 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7gxwt"] Jun 11 21:19:40.843095 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:40.843060 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7gxwt" event={"ID":"9d605903-7c76-48d9-9a1a-439b29742598","Type":"ContainerStarted","Data":"b49679affde9e369f9c5d5b94eb97eede8a6ded7973bf4f3c20863efeb408c42"} Jun 11 21:19:41.847807 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:41.847772 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7gxwt" event={"ID":"9d605903-7c76-48d9-9a1a-439b29742598","Type":"ContainerStarted","Data":"6b71adfb07febc2d0809eb154ec32167f75fb3c65dd1bcb23a961c206cc0df67"} Jun 11 21:19:41.848248 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:41.847977 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7gxwt" Jun 11 21:19:41.852638 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:41.852620 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7gxwt" Jun 11 21:19:41.863267 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:41.863052 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5cf667749c-7gxwt" podStartSLOduration=1.872277445 podStartE2EDuration="2.863035429s" totalImagesPullingTime="990.757984ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:19:41.862863298 +0000 UTC m=+240.274186847" watchObservedRunningTime="2026-06-11 21:19:41.863035429 +0000 UTC m=+240.274358966" Jun 11 21:19:42.138836 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:42.138751 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d87ff458-g7pl5" Jun 11 21:19:42.138836 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:42.138804 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d87ff458-g7pl5" Jun 11 21:19:42.140143 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:42.140122 2570 patch_prober.go:28] interesting pod/console-d87ff458-g7pl5 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.11:8443/health\": dial tcp 10.134.0.11:8443: connect: connection refused" start-of-body= Jun 11 21:19:42.140211 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:42.140165 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-d87ff458-g7pl5" podUID="82296dc1-4d45-4b57-bb06-50853f80ccf8" containerName="console" probeResult="failure" output="Get \"https://10.134.0.11:8443/health\": dial tcp 10.134.0.11:8443: connect: connection refused" Jun 11 21:19:47.797279 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:47.797248 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mpqmj"] Jun 11 21:19:47.800443 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:47.800425 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:47.802724 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:47.802699 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 11 21:19:47.802855 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:47.802708 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 11 21:19:47.802855 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:47.802822 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 11 21:19:47.802984 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:47.802828 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 11 21:19:47.803040 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:47.802995 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mzcr4\"" Jun 11 21:19:47.803174 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:47.803151 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 11 21:19:47.803616 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:47.803602 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 11 21:19:47.947740 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:47.947693 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6aac3ed7-198f-44c0-934f-f0606ffb7955-node-exporter-wtmp\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:47.947740 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:47.947750 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klpxv\" (UniqueName: \"kubernetes.io/projected/6aac3ed7-198f-44c0-934f-f0606ffb7955-kube-api-access-klpxv\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:47.947970 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:47.947816 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6aac3ed7-198f-44c0-934f-f0606ffb7955-root\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:47.947970 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:47.947837 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6aac3ed7-198f-44c0-934f-f0606ffb7955-node-exporter-tls\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:47.947970 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:47.947875 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6aac3ed7-198f-44c0-934f-f0606ffb7955-sys\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:47.947970 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:47.947894 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6aac3ed7-198f-44c0-934f-f0606ffb7955-metrics-client-ca\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:47.947970 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:47.947939 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6aac3ed7-198f-44c0-934f-f0606ffb7955-node-exporter-accelerators-collector-config\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:47.947970 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:47.947967 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6aac3ed7-198f-44c0-934f-f0606ffb7955-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:47.948150 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:47.947986 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6aac3ed7-198f-44c0-934f-f0606ffb7955-node-exporter-textfile\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:48.048672 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.048580 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6aac3ed7-198f-44c0-934f-f0606ffb7955-metrics-client-ca\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:48.048672 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.048626 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6aac3ed7-198f-44c0-934f-f0606ffb7955-node-exporter-accelerators-collector-config\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:48.048672 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.048662 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6aac3ed7-198f-44c0-934f-f0606ffb7955-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:48.048946 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.048690 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6aac3ed7-198f-44c0-934f-f0606ffb7955-node-exporter-textfile\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:48.048946 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.048743 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6aac3ed7-198f-44c0-934f-f0606ffb7955-node-exporter-wtmp\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:48.048946 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.048776 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klpxv\" (UniqueName: \"kubernetes.io/projected/6aac3ed7-198f-44c0-934f-f0606ffb7955-kube-api-access-klpxv\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:48.048946 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.048917 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6aac3ed7-198f-44c0-934f-f0606ffb7955-node-exporter-wtmp\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:48.049135 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.048984 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6aac3ed7-198f-44c0-934f-f0606ffb7955-root\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:48.049135 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.049014 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6aac3ed7-198f-44c0-934f-f0606ffb7955-node-exporter-tls\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:48.049135 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.049047 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6aac3ed7-198f-44c0-934f-f0606ffb7955-sys\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:48.049135 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.049090 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6aac3ed7-198f-44c0-934f-f0606ffb7955-node-exporter-textfile\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:48.049135 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.049124 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6aac3ed7-198f-44c0-934f-f0606ffb7955-sys\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:48.049351 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.049139 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6aac3ed7-198f-44c0-934f-f0606ffb7955-root\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:48.049351 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:48.049204 2570 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 11 21:19:48.049351 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.049230 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6aac3ed7-198f-44c0-934f-f0606ffb7955-metrics-client-ca\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:48.049351 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:48.049284 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6aac3ed7-198f-44c0-934f-f0606ffb7955-node-exporter-tls podName:6aac3ed7-198f-44c0-934f-f0606ffb7955 nodeName:}" failed. No retries permitted until 2026-06-11 21:19:48.549266164 +0000 UTC m=+246.960589685 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/6aac3ed7-198f-44c0-934f-f0606ffb7955-node-exporter-tls") pod "node-exporter-mpqmj" (UID: "6aac3ed7-198f-44c0-934f-f0606ffb7955") : secret "node-exporter-tls" not found Jun 11 21:19:48.049618 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.049599 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6aac3ed7-198f-44c0-934f-f0606ffb7955-node-exporter-accelerators-collector-config\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:48.051286 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.051261 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6aac3ed7-198f-44c0-934f-f0606ffb7955-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:48.066246 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.066215 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klpxv\" (UniqueName: \"kubernetes.io/projected/6aac3ed7-198f-44c0-934f-f0606ffb7955-kube-api-access-klpxv\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:48.554042 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.554007 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6aac3ed7-198f-44c0-934f-f0606ffb7955-node-exporter-tls\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:48.556313 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.556283 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6aac3ed7-198f-44c0-934f-f0606ffb7955-node-exporter-tls\") pod \"node-exporter-mpqmj\" (UID: \"6aac3ed7-198f-44c0-934f-f0606ffb7955\") " pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:48.709080 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.709051 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mpqmj" Jun 11 21:19:48.719092 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:19:48.719063 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aac3ed7_198f_44c0_934f_f0606ffb7955.slice/crio-567ea7932a4611cc3a26cdd4d92cd8627a7822e7f57b8045cca39fdf19ef754c WatchSource:0}: Error finding container 567ea7932a4611cc3a26cdd4d92cd8627a7822e7f57b8045cca39fdf19ef754c: Status 404 returned error can't find the container with id 567ea7932a4611cc3a26cdd4d92cd8627a7822e7f57b8045cca39fdf19ef754c Jun 11 21:19:48.864625 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.864588 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 11 21:19:48.869000 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.868974 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:48.870284 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.870261 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mpqmj" event={"ID":"6aac3ed7-198f-44c0-934f-f0606ffb7955","Type":"ContainerStarted","Data":"567ea7932a4611cc3a26cdd4d92cd8627a7822e7f57b8045cca39fdf19ef754c"} Jun 11 21:19:48.871383 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.871355 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 11 21:19:48.871538 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.871413 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 11 21:19:48.871538 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.871475 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 11 21:19:48.871538 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.871415 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 11 21:19:48.871538 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.871479 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-4mdd5\"" Jun 11 21:19:48.871538 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.871484 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 11 21:19:48.871888 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.871475 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 11 21:19:48.871888 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.871810 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 11 21:19:48.871888 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.871815 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 11 21:19:48.871888 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.871812 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 11 21:19:48.884357 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:48.884333 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 11 21:19:49.058664 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.058621 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a22dba60-3268-4a0a-a94d-d03c33d5b38a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.058664 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.058669 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a22dba60-3268-4a0a-a94d-d03c33d5b38a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.058870 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.058686 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6lm2\" (UniqueName: \"kubernetes.io/projected/a22dba60-3268-4a0a-a94d-d03c33d5b38a-kube-api-access-n6lm2\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.058870 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.058716 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a22dba60-3268-4a0a-a94d-d03c33d5b38a-web-config\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.058870 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.058737 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a22dba60-3268-4a0a-a94d-d03c33d5b38a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.058870 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.058774 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a22dba60-3268-4a0a-a94d-d03c33d5b38a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.058870 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.058832 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a22dba60-3268-4a0a-a94d-d03c33d5b38a-config-volume\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.058870 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.058870 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a22dba60-3268-4a0a-a94d-d03c33d5b38a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.059088 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.058890 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a22dba60-3268-4a0a-a94d-d03c33d5b38a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.059088 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.058916 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a22dba60-3268-4a0a-a94d-d03c33d5b38a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.059088 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.058981 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a22dba60-3268-4a0a-a94d-d03c33d5b38a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.059088 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.059007 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a22dba60-3268-4a0a-a94d-d03c33d5b38a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.059088 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.059045 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a22dba60-3268-4a0a-a94d-d03c33d5b38a-config-out\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.159962 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.159872 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a22dba60-3268-4a0a-a94d-d03c33d5b38a-web-config\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.159962 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.159912 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a22dba60-3268-4a0a-a94d-d03c33d5b38a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.159962 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.159946 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a22dba60-3268-4a0a-a94d-d03c33d5b38a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.159962 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.159967 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a22dba60-3268-4a0a-a94d-d03c33d5b38a-config-volume\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.160281 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.159986 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a22dba60-3268-4a0a-a94d-d03c33d5b38a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.160281 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.160005 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a22dba60-3268-4a0a-a94d-d03c33d5b38a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.160281 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.160020 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a22dba60-3268-4a0a-a94d-d03c33d5b38a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.160281 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.160095 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a22dba60-3268-4a0a-a94d-d03c33d5b38a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.160281 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.160154 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a22dba60-3268-4a0a-a94d-d03c33d5b38a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.160281 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.160190 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a22dba60-3268-4a0a-a94d-d03c33d5b38a-config-out\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.160281 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.160267 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a22dba60-3268-4a0a-a94d-d03c33d5b38a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.160650 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.160299 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a22dba60-3268-4a0a-a94d-d03c33d5b38a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.160650 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.160325 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6lm2\" (UniqueName: \"kubernetes.io/projected/a22dba60-3268-4a0a-a94d-d03c33d5b38a-kube-api-access-n6lm2\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.160752 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.160734 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a22dba60-3268-4a0a-a94d-d03c33d5b38a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.160821 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.160798 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a22dba60-3268-4a0a-a94d-d03c33d5b38a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.161806 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.161783 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a22dba60-3268-4a0a-a94d-d03c33d5b38a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.163065 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.163037 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a22dba60-3268-4a0a-a94d-d03c33d5b38a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.163065 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.163062 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a22dba60-3268-4a0a-a94d-d03c33d5b38a-config-volume\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.163240 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.163171 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a22dba60-3268-4a0a-a94d-d03c33d5b38a-web-config\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.163444 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.163425 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a22dba60-3268-4a0a-a94d-d03c33d5b38a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.163502 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.163471 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a22dba60-3268-4a0a-a94d-d03c33d5b38a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.163502 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.163479 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a22dba60-3268-4a0a-a94d-d03c33d5b38a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.164173 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.164154 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a22dba60-3268-4a0a-a94d-d03c33d5b38a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.165117 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.165094 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a22dba60-3268-4a0a-a94d-d03c33d5b38a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.165192 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.165133 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a22dba60-3268-4a0a-a94d-d03c33d5b38a-config-out\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.185861 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.185834 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6lm2\" (UniqueName: \"kubernetes.io/projected/a22dba60-3268-4a0a-a94d-d03c33d5b38a-kube-api-access-n6lm2\") pod \"alertmanager-main-0\" (UID: \"a22dba60-3268-4a0a-a94d-d03c33d5b38a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.478863 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.478780 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 11 21:19:49.835382 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.835358 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 11 21:19:49.837830 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:19:49.837802 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda22dba60_3268_4a0a_a94d_d03c33d5b38a.slice/crio-d191d82f2bda70c72a99ce11a7f4ad671fda641eb10479e100f6d4337189ab4e WatchSource:0}: Error finding container d191d82f2bda70c72a99ce11a7f4ad671fda641eb10479e100f6d4337189ab4e: Status 404 returned error can't find the container with id d191d82f2bda70c72a99ce11a7f4ad671fda641eb10479e100f6d4337189ab4e Jun 11 21:19:49.875659 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.875619 2570 generic.go:356] "Generic (PLEG): container finished" podID="6aac3ed7-198f-44c0-934f-f0606ffb7955" containerID="18a86f1239997bb95e5cafb462f62ce68cb1f50edfda1007f80833cd2b4b452d" exitCode=0 Jun 11 21:19:49.876059 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.875675 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mpqmj" event={"ID":"6aac3ed7-198f-44c0-934f-f0606ffb7955","Type":"ContainerDied","Data":"18a86f1239997bb95e5cafb462f62ce68cb1f50edfda1007f80833cd2b4b452d"} Jun 11 21:19:49.877076 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:49.877053 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a22dba60-3268-4a0a-a94d-d03c33d5b38a","Type":"ContainerStarted","Data":"d191d82f2bda70c72a99ce11a7f4ad671fda641eb10479e100f6d4337189ab4e"} Jun 11 21:19:50.882015 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:50.881980 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mpqmj" event={"ID":"6aac3ed7-198f-44c0-934f-f0606ffb7955","Type":"ContainerStarted","Data":"8d2bff659a6a58d3da6b934615ef9bfaf67b6ae5ec916da664fa8bb7c246f106"} Jun 11 21:19:50.882375 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:50.882042 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mpqmj" event={"ID":"6aac3ed7-198f-44c0-934f-f0606ffb7955","Type":"ContainerStarted","Data":"c74c8b62278564df2468478f8dc9241da24acb7dd02cdd76657069740c18bad3"} Jun 11 21:19:50.906633 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:50.906584 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mpqmj" podStartSLOduration=2.839707119 podStartE2EDuration="3.906545084s" totalImagesPullingTime="999.796475ms" totalInitContainerRuntime="67.04149ms" isStatefulPod=true podCreationTimestamp="2026-06-11 21:19:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:19:50.904263379 +0000 UTC m=+249.315586936" watchObservedRunningTime="2026-06-11 21:19:50.906545084 +0000 UTC m=+249.317868623" Jun 11 21:19:51.887720 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:51.887688 2570 generic.go:356] "Generic (PLEG): container finished" podID="a22dba60-3268-4a0a-a94d-d03c33d5b38a" containerID="6c6832a3e65f5e81cb3d4d1ebb1dce5ffc0184c2702d73b5c16cf77005a7b95c" exitCode=0 Jun 11 21:19:51.888118 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:51.887777 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a22dba60-3268-4a0a-a94d-d03c33d5b38a","Type":"ContainerDied","Data":"6c6832a3e65f5e81cb3d4d1ebb1dce5ffc0184c2702d73b5c16cf77005a7b95c"} Jun 11 21:19:52.139640 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.139541 2570 patch_prober.go:28] interesting pod/console-d87ff458-g7pl5 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.11:8443/health\": dial tcp 10.134.0.11:8443: connect: connection refused" start-of-body= Jun 11 21:19:52.139777 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.139642 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-d87ff458-g7pl5" podUID="82296dc1-4d45-4b57-bb06-50853f80ccf8" containerName="console" probeResult="failure" output="Get \"https://10.134.0.11:8443/health\": dial tcp 10.134.0.11:8443: connect: connection refused" Jun 11 21:19:52.250030 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.249987 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7d868dd696-hhxbl"] Jun 11 21:19:52.253508 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.253492 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:19:52.256261 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.256233 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 11 21:19:52.256261 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.256233 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-9nsrk\"" Jun 11 21:19:52.256492 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.256478 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 11 21:19:52.256660 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.256646 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 11 21:19:52.257191 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.257170 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 11 21:19:52.257353 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.257170 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4fuu5nud3g5vp\"" Jun 11 21:19:52.261377 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.261357 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7d868dd696-hhxbl"] Jun 11 21:19:52.286590 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.286546 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/61bce198-7787-4f9d-83a4-75133afaca31-audit-log\") pod \"metrics-server-7d868dd696-hhxbl\" (UID: \"61bce198-7787-4f9d-83a4-75133afaca31\") " pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:19:52.286590 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.286599 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/61bce198-7787-4f9d-83a4-75133afaca31-client-ca-bundle\") pod \"metrics-server-7d868dd696-hhxbl\" (UID: \"61bce198-7787-4f9d-83a4-75133afaca31\") " pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:19:52.286742 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.286646 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/61bce198-7787-4f9d-83a4-75133afaca31-secret-metrics-server-tls\") pod \"metrics-server-7d868dd696-hhxbl\" (UID: \"61bce198-7787-4f9d-83a4-75133afaca31\") " pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:19:52.286742 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.286686 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn5mx\" (UniqueName: \"kubernetes.io/projected/61bce198-7787-4f9d-83a4-75133afaca31-kube-api-access-kn5mx\") pod \"metrics-server-7d868dd696-hhxbl\" (UID: \"61bce198-7787-4f9d-83a4-75133afaca31\") " pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:19:52.286803 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.286746 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61bce198-7787-4f9d-83a4-75133afaca31-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7d868dd696-hhxbl\" (UID: \"61bce198-7787-4f9d-83a4-75133afaca31\") " pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:19:52.286803 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.286774 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/61bce198-7787-4f9d-83a4-75133afaca31-secret-metrics-server-client-certs\") pod \"metrics-server-7d868dd696-hhxbl\" (UID: \"61bce198-7787-4f9d-83a4-75133afaca31\") " pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:19:52.286803 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.286796 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/61bce198-7787-4f9d-83a4-75133afaca31-metrics-server-audit-profiles\") pod \"metrics-server-7d868dd696-hhxbl\" (UID: \"61bce198-7787-4f9d-83a4-75133afaca31\") " pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:19:52.387311 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.387271 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61bce198-7787-4f9d-83a4-75133afaca31-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7d868dd696-hhxbl\" (UID: \"61bce198-7787-4f9d-83a4-75133afaca31\") " pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:19:52.387311 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.387313 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/61bce198-7787-4f9d-83a4-75133afaca31-secret-metrics-server-client-certs\") pod \"metrics-server-7d868dd696-hhxbl\" (UID: \"61bce198-7787-4f9d-83a4-75133afaca31\") " pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:19:52.387488 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.387341 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/61bce198-7787-4f9d-83a4-75133afaca31-metrics-server-audit-profiles\") pod \"metrics-server-7d868dd696-hhxbl\" (UID: \"61bce198-7787-4f9d-83a4-75133afaca31\") " pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:19:52.387488 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.387384 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/61bce198-7787-4f9d-83a4-75133afaca31-audit-log\") pod \"metrics-server-7d868dd696-hhxbl\" (UID: \"61bce198-7787-4f9d-83a4-75133afaca31\") " pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:19:52.387488 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.387402 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/61bce198-7787-4f9d-83a4-75133afaca31-client-ca-bundle\") pod \"metrics-server-7d868dd696-hhxbl\" (UID: \"61bce198-7787-4f9d-83a4-75133afaca31\") " pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:19:52.387488 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.387466 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/61bce198-7787-4f9d-83a4-75133afaca31-secret-metrics-server-tls\") pod \"metrics-server-7d868dd696-hhxbl\" (UID: \"61bce198-7787-4f9d-83a4-75133afaca31\") " pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:19:52.387737 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.387518 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn5mx\" (UniqueName: \"kubernetes.io/projected/61bce198-7787-4f9d-83a4-75133afaca31-kube-api-access-kn5mx\") pod \"metrics-server-7d868dd696-hhxbl\" (UID: \"61bce198-7787-4f9d-83a4-75133afaca31\") " pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:19:52.387892 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.387857 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/61bce198-7787-4f9d-83a4-75133afaca31-audit-log\") pod \"metrics-server-7d868dd696-hhxbl\" (UID: \"61bce198-7787-4f9d-83a4-75133afaca31\") " pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:19:52.388300 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.388270 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61bce198-7787-4f9d-83a4-75133afaca31-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7d868dd696-hhxbl\" (UID: \"61bce198-7787-4f9d-83a4-75133afaca31\") " pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:19:52.388417 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.388391 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/61bce198-7787-4f9d-83a4-75133afaca31-metrics-server-audit-profiles\") pod \"metrics-server-7d868dd696-hhxbl\" (UID: \"61bce198-7787-4f9d-83a4-75133afaca31\") " pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:19:52.390126 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.390065 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/61bce198-7787-4f9d-83a4-75133afaca31-secret-metrics-server-tls\") pod \"metrics-server-7d868dd696-hhxbl\" (UID: \"61bce198-7787-4f9d-83a4-75133afaca31\") " pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:19:52.390531 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.390514 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/61bce198-7787-4f9d-83a4-75133afaca31-client-ca-bundle\") pod \"metrics-server-7d868dd696-hhxbl\" (UID: \"61bce198-7787-4f9d-83a4-75133afaca31\") " pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:19:52.390640 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.390624 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/61bce198-7787-4f9d-83a4-75133afaca31-secret-metrics-server-client-certs\") pod \"metrics-server-7d868dd696-hhxbl\" (UID: \"61bce198-7787-4f9d-83a4-75133afaca31\") " pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:19:52.396984 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.396959 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn5mx\" (UniqueName: \"kubernetes.io/projected/61bce198-7787-4f9d-83a4-75133afaca31-kube-api-access-kn5mx\") pod \"metrics-server-7d868dd696-hhxbl\" (UID: \"61bce198-7787-4f9d-83a4-75133afaca31\") " pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:19:52.562862 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.562832 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:19:52.568071 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.568041 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-589576dfc7-bplr4"] Jun 11 21:19:52.571161 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.571136 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-589576dfc7-bplr4" Jun 11 21:19:52.574493 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.574469 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 11 21:19:52.574640 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.574556 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-9dqxb\"" Jun 11 21:19:52.582007 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.581980 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-589576dfc7-bplr4"] Jun 11 21:19:52.588608 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.588556 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e630b187-62f5-4152-9b03-887853c93376-monitoring-plugin-cert\") pod \"monitoring-plugin-589576dfc7-bplr4\" (UID: \"e630b187-62f5-4152-9b03-887853c93376\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-bplr4" Jun 11 21:19:52.588741 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.588672 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvm6s\" (UniqueName: \"kubernetes.io/projected/e630b187-62f5-4152-9b03-887853c93376-kube-api-access-wvm6s\") pod \"monitoring-plugin-589576dfc7-bplr4\" (UID: \"e630b187-62f5-4152-9b03-887853c93376\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-bplr4" Jun 11 21:19:52.590423 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.590399 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d87ff458-g7pl5"] Jun 11 21:19:52.621287 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.621251 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5756c675b8-xpf29"] Jun 11 21:19:52.624450 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.624424 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:19:52.635438 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.635401 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 11 21:19:52.639113 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.639086 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5756c675b8-xpf29"] Jun 11 21:19:52.689620 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.689592 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6123c52f-479c-4c94-8427-407efd3d891f-console-oauth-config\") pod \"console-5756c675b8-xpf29\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:19:52.689800 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.689635 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e630b187-62f5-4152-9b03-887853c93376-monitoring-plugin-cert\") pod \"monitoring-plugin-589576dfc7-bplr4\" (UID: \"e630b187-62f5-4152-9b03-887853c93376\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-bplr4" Jun 11 21:19:52.689800 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.689673 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6123c52f-479c-4c94-8427-407efd3d891f-service-ca\") pod \"console-5756c675b8-xpf29\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:19:52.689800 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.689694 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6123c52f-479c-4c94-8427-407efd3d891f-trusted-ca-bundle\") pod \"console-5756c675b8-xpf29\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:19:52.689800 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.689740 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6123c52f-479c-4c94-8427-407efd3d891f-oauth-serving-cert\") pod \"console-5756c675b8-xpf29\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:19:52.689800 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:52.689773 2570 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jun 11 21:19:52.690036 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:52.689836 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e630b187-62f5-4152-9b03-887853c93376-monitoring-plugin-cert podName:e630b187-62f5-4152-9b03-887853c93376 nodeName:}" failed. No retries permitted until 2026-06-11 21:19:53.189815169 +0000 UTC m=+251.601138688 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/e630b187-62f5-4152-9b03-887853c93376-monitoring-plugin-cert") pod "monitoring-plugin-589576dfc7-bplr4" (UID: "e630b187-62f5-4152-9b03-887853c93376") : secret "monitoring-plugin-cert" not found Jun 11 21:19:52.690036 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.689775 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6123c52f-479c-4c94-8427-407efd3d891f-console-serving-cert\") pod \"console-5756c675b8-xpf29\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:19:52.690036 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.689871 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltqp9\" (UniqueName: \"kubernetes.io/projected/6123c52f-479c-4c94-8427-407efd3d891f-kube-api-access-ltqp9\") pod \"console-5756c675b8-xpf29\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:19:52.690036 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.689906 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6123c52f-479c-4c94-8427-407efd3d891f-console-config\") pod \"console-5756c675b8-xpf29\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:19:52.690036 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.689934 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvm6s\" (UniqueName: \"kubernetes.io/projected/e630b187-62f5-4152-9b03-887853c93376-kube-api-access-wvm6s\") pod \"monitoring-plugin-589576dfc7-bplr4\" (UID: \"e630b187-62f5-4152-9b03-887853c93376\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-bplr4" Jun 11 21:19:52.699331 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.699303 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvm6s\" (UniqueName: \"kubernetes.io/projected/e630b187-62f5-4152-9b03-887853c93376-kube-api-access-wvm6s\") pod \"monitoring-plugin-589576dfc7-bplr4\" (UID: \"e630b187-62f5-4152-9b03-887853c93376\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-bplr4" Jun 11 21:19:52.712126 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.712028 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7d868dd696-hhxbl"] Jun 11 21:19:52.714924 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:19:52.714891 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61bce198_7787_4f9d_83a4_75133afaca31.slice/crio-fc39e09bbf0069f90863d243cc610026e887bee3575e4338145d6529f869460f WatchSource:0}: Error finding container fc39e09bbf0069f90863d243cc610026e887bee3575e4338145d6529f869460f: Status 404 returned error can't find the container with id fc39e09bbf0069f90863d243cc610026e887bee3575e4338145d6529f869460f Jun 11 21:19:52.790440 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.790409 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6123c52f-479c-4c94-8427-407efd3d891f-oauth-serving-cert\") pod \"console-5756c675b8-xpf29\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:19:52.790440 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.790447 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6123c52f-479c-4c94-8427-407efd3d891f-console-serving-cert\") pod \"console-5756c675b8-xpf29\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:19:52.790659 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.790468 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltqp9\" (UniqueName: \"kubernetes.io/projected/6123c52f-479c-4c94-8427-407efd3d891f-kube-api-access-ltqp9\") pod \"console-5756c675b8-xpf29\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:19:52.790659 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.790527 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6123c52f-479c-4c94-8427-407efd3d891f-console-config\") pod \"console-5756c675b8-xpf29\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:19:52.790854 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.790832 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6123c52f-479c-4c94-8427-407efd3d891f-console-oauth-config\") pod \"console-5756c675b8-xpf29\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:19:52.790921 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.790904 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6123c52f-479c-4c94-8427-407efd3d891f-service-ca\") pod \"console-5756c675b8-xpf29\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:19:52.790974 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.790933 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6123c52f-479c-4c94-8427-407efd3d891f-trusted-ca-bundle\") pod \"console-5756c675b8-xpf29\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:19:52.791310 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.791288 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6123c52f-479c-4c94-8427-407efd3d891f-console-config\") pod \"console-5756c675b8-xpf29\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:19:52.791430 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.791416 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6123c52f-479c-4c94-8427-407efd3d891f-oauth-serving-cert\") pod \"console-5756c675b8-xpf29\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:19:52.791501 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.791485 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6123c52f-479c-4c94-8427-407efd3d891f-service-ca\") pod \"console-5756c675b8-xpf29\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:19:52.791703 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.791687 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6123c52f-479c-4c94-8427-407efd3d891f-trusted-ca-bundle\") pod \"console-5756c675b8-xpf29\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:19:52.793496 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.793465 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6123c52f-479c-4c94-8427-407efd3d891f-console-oauth-config\") pod \"console-5756c675b8-xpf29\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:19:52.793641 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.793624 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6123c52f-479c-4c94-8427-407efd3d891f-console-serving-cert\") pod \"console-5756c675b8-xpf29\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:19:52.798437 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.798418 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltqp9\" (UniqueName: \"kubernetes.io/projected/6123c52f-479c-4c94-8427-407efd3d891f-kube-api-access-ltqp9\") pod \"console-5756c675b8-xpf29\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:19:52.892486 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.892442 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" event={"ID":"61bce198-7787-4f9d-83a4-75133afaca31","Type":"ContainerStarted","Data":"fc39e09bbf0069f90863d243cc610026e887bee3575e4338145d6529f869460f"} Jun 11 21:19:52.938202 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:52.938117 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:19:53.070798 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:53.070761 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5756c675b8-xpf29"] Jun 11 21:19:53.194945 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:53.194861 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e630b187-62f5-4152-9b03-887853c93376-monitoring-plugin-cert\") pod \"monitoring-plugin-589576dfc7-bplr4\" (UID: \"e630b187-62f5-4152-9b03-887853c93376\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-bplr4" Jun 11 21:19:53.195112 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:53.195024 2570 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jun 11 21:19:53.195112 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:19:53.195100 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e630b187-62f5-4152-9b03-887853c93376-monitoring-plugin-cert podName:e630b187-62f5-4152-9b03-887853c93376 nodeName:}" failed. No retries permitted until 2026-06-11 21:19:54.195079549 +0000 UTC m=+252.606403067 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/e630b187-62f5-4152-9b03-887853c93376-monitoring-plugin-cert") pod "monitoring-plugin-589576dfc7-bplr4" (UID: "e630b187-62f5-4152-9b03-887853c93376") : secret "monitoring-plugin-cert" not found Jun 11 21:19:53.340391 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:19:53.340358 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6123c52f_479c_4c94_8427_407efd3d891f.slice/crio-254be746b85627a786e76060e022e1ed07a8f8f1ecc39be5f126344261c7cdd6 WatchSource:0}: Error finding container 254be746b85627a786e76060e022e1ed07a8f8f1ecc39be5f126344261c7cdd6: Status 404 returned error can't find the container with id 254be746b85627a786e76060e022e1ed07a8f8f1ecc39be5f126344261c7cdd6 Jun 11 21:19:53.901705 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:53.901661 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a22dba60-3268-4a0a-a94d-d03c33d5b38a","Type":"ContainerStarted","Data":"780934154e6acfd4eb3ec48776c696ad84bb66852a65a4295c6043d86fb722bb"} Jun 11 21:19:53.902127 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:53.901754 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a22dba60-3268-4a0a-a94d-d03c33d5b38a","Type":"ContainerStarted","Data":"d8ed4aa99ae01d764827815795e1915741969680e9bbfa96851f42053e51ae14"} Jun 11 21:19:53.902127 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:53.901795 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a22dba60-3268-4a0a-a94d-d03c33d5b38a","Type":"ContainerStarted","Data":"10ec41c380cabfabae215a350a5005dde854a96a2998c0ffcf559bff3c3eaea6"} Jun 11 21:19:53.902127 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:53.901834 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a22dba60-3268-4a0a-a94d-d03c33d5b38a","Type":"ContainerStarted","Data":"2bd9aac89bf1469224748d565946acf2a24cc4f83cf1ab26f1cce21252738a33"} Jun 11 21:19:53.902127 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:53.901874 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a22dba60-3268-4a0a-a94d-d03c33d5b38a","Type":"ContainerStarted","Data":"bc1eca55073496408fc3093f42c301d711299ea6b3a2d5ce6d7c023c877fd6ab"} Jun 11 21:19:53.903396 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:53.903368 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5756c675b8-xpf29" event={"ID":"6123c52f-479c-4c94-8427-407efd3d891f","Type":"ContainerStarted","Data":"85a2014e904f965c080d11290a2cb44e3e643c1610c8b90a9ed93d9df61aea29"} Jun 11 21:19:53.903493 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:53.903417 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5756c675b8-xpf29" event={"ID":"6123c52f-479c-4c94-8427-407efd3d891f","Type":"ContainerStarted","Data":"254be746b85627a786e76060e022e1ed07a8f8f1ecc39be5f126344261c7cdd6"} Jun 11 21:19:53.923181 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:53.923114 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5756c675b8-xpf29" podStartSLOduration=1.923088277 podStartE2EDuration="1.923088277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:19:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:19:53.921466829 +0000 UTC m=+252.332790369" watchObservedRunningTime="2026-06-11 21:19:53.923088277 +0000 UTC m=+252.334411815" Jun 11 21:19:54.000963 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:54.000870 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs\") pod \"network-metrics-daemon-2bmhl\" (UID: \"329e0949-4197-4a29-afd8-1300e76b75cf\") " pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:19:54.003123 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:54.003099 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/329e0949-4197-4a29-afd8-1300e76b75cf-metrics-certs\") pod \"network-metrics-daemon-2bmhl\" (UID: \"329e0949-4197-4a29-afd8-1300e76b75cf\") " pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:19:54.052788 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:54.049426 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9j4lb\"" Jun 11 21:19:54.058076 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:54.058050 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bmhl" Jun 11 21:19:54.203879 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:54.203845 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e630b187-62f5-4152-9b03-887853c93376-monitoring-plugin-cert\") pod \"monitoring-plugin-589576dfc7-bplr4\" (UID: \"e630b187-62f5-4152-9b03-887853c93376\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-bplr4" Jun 11 21:19:54.206625 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:54.206597 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e630b187-62f5-4152-9b03-887853c93376-monitoring-plugin-cert\") pod \"monitoring-plugin-589576dfc7-bplr4\" (UID: \"e630b187-62f5-4152-9b03-887853c93376\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-bplr4" Jun 11 21:19:54.399050 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:54.398943 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-589576dfc7-bplr4" Jun 11 21:19:54.502991 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:54.502967 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2bmhl"] Jun 11 21:19:54.504704 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:19:54.504632 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod329e0949_4197_4a29_afd8_1300e76b75cf.slice/crio-d183f79bb77c4669590cc5cd25598878cfdf4c96e7e0b47150d070c282ec6c9d WatchSource:0}: Error finding container d183f79bb77c4669590cc5cd25598878cfdf4c96e7e0b47150d070c282ec6c9d: Status 404 returned error can't find the container with id d183f79bb77c4669590cc5cd25598878cfdf4c96e7e0b47150d070c282ec6c9d Jun 11 21:19:54.568410 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:54.568378 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-589576dfc7-bplr4"] Jun 11 21:19:54.572671 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:19:54.572631 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode630b187_62f5_4152_9b03_887853c93376.slice/crio-a364fdeaa38e571485a44f6f02e9e50307a86d12384f6cba7e0c44459f597613 WatchSource:0}: Error finding container a364fdeaa38e571485a44f6f02e9e50307a86d12384f6cba7e0c44459f597613: Status 404 returned error can't find the container with id a364fdeaa38e571485a44f6f02e9e50307a86d12384f6cba7e0c44459f597613 Jun 11 21:19:54.908358 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:54.908323 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" event={"ID":"61bce198-7787-4f9d-83a4-75133afaca31","Type":"ContainerStarted","Data":"4d6936e99071e08980e46830798a957f452126db129cb5c3c43bf6009d5a4531"} Jun 11 21:19:54.910020 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:54.909993 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2bmhl" event={"ID":"329e0949-4197-4a29-afd8-1300e76b75cf","Type":"ContainerStarted","Data":"d183f79bb77c4669590cc5cd25598878cfdf4c96e7e0b47150d070c282ec6c9d"} Jun 11 21:19:54.911254 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:54.911215 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-589576dfc7-bplr4" event={"ID":"e630b187-62f5-4152-9b03-887853c93376","Type":"ContainerStarted","Data":"a364fdeaa38e571485a44f6f02e9e50307a86d12384f6cba7e0c44459f597613"} Jun 11 21:19:54.914805 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:54.914780 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a22dba60-3268-4a0a-a94d-d03c33d5b38a","Type":"ContainerStarted","Data":"1e7599f601c4838b6a852baa5ebdc4f5af8473a7ebab3caa74508b9cc07cc803"} Jun 11 21:19:54.925845 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:54.925799 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" podStartSLOduration=1.227773061 podStartE2EDuration="2.92578422s" totalImagesPullingTime="1.698011159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:19:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:19:54.925348707 +0000 UTC m=+253.336672245" watchObservedRunningTime="2026-06-11 21:19:54.92578422 +0000 UTC m=+253.337107759" Jun 11 21:19:54.949105 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:54.949062 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.271335155 podStartE2EDuration="6.949050093s" totalImagesPullingTime="3.661584713s" totalInitContainerRuntime="16.130225ms" isStatefulPod=false podCreationTimestamp="2026-06-11 21:19:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:19:54.947888997 +0000 UTC m=+253.359212553" watchObservedRunningTime="2026-06-11 21:19:54.949050093 +0000 UTC m=+253.360373691" Jun 11 21:19:55.921050 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:55.920962 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2bmhl" event={"ID":"329e0949-4197-4a29-afd8-1300e76b75cf","Type":"ContainerStarted","Data":"a1add004e8ab35612cf3cd076a84bb55b7216dc8ab26f5ca0da3195626fb0b4f"} Jun 11 21:19:56.925857 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:56.925818 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2bmhl" event={"ID":"329e0949-4197-4a29-afd8-1300e76b75cf","Type":"ContainerStarted","Data":"51c680714403447385ead325015610b3ba75e77ac314612cd8fb34350c94acec"} Jun 11 21:19:56.927108 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:56.927085 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-589576dfc7-bplr4" event={"ID":"e630b187-62f5-4152-9b03-887853c93376","Type":"ContainerStarted","Data":"822c0c8b718250cb9825df5b57e414b879d1ed2607417da9f50507f26e6dd403"} Jun 11 21:19:56.927299 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:56.927283 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-589576dfc7-bplr4" Jun 11 21:19:56.931987 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:56.931968 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-589576dfc7-bplr4" Jun 11 21:19:56.943088 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:56.943048 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2bmhl" podStartSLOduration=253.825979004 podStartE2EDuration="4m14.943034323s" totalImagesPullingTime="1.117055319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:19:56.942219567 +0000 UTC m=+255.353543105" watchObservedRunningTime="2026-06-11 21:19:56.943034323 +0000 UTC m=+255.354357870" Jun 11 21:19:56.957438 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:19:56.957391 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-589576dfc7-bplr4" podStartSLOduration=3.4895969989999998 podStartE2EDuration="4.957377166s" totalImagesPullingTime="1.467780167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:19:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:19:56.956717703 +0000 UTC m=+255.368041275" watchObservedRunningTime="2026-06-11 21:19:56.957377166 +0000 UTC m=+255.368700704" Jun 11 21:20:00.014175 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.014137 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6cd464788f-6csp8"] Jun 11 21:20:00.017136 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.017120 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:00.028494 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.028469 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cd464788f-6csp8"] Jun 11 21:20:00.048372 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.048343 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/91913017-a3ef-4324-9546-22e0fead3df1-console-serving-cert\") pod \"console-6cd464788f-6csp8\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:00.048511 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.048431 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/91913017-a3ef-4324-9546-22e0fead3df1-oauth-serving-cert\") pod \"console-6cd464788f-6csp8\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:00.048511 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.048457 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/91913017-a3ef-4324-9546-22e0fead3df1-console-oauth-config\") pod \"console-6cd464788f-6csp8\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:00.048511 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.048472 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/91913017-a3ef-4324-9546-22e0fead3df1-console-config\") pod \"console-6cd464788f-6csp8\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:00.048511 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.048500 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h2cb\" (UniqueName: \"kubernetes.io/projected/91913017-a3ef-4324-9546-22e0fead3df1-kube-api-access-2h2cb\") pod \"console-6cd464788f-6csp8\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:00.048679 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.048541 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91913017-a3ef-4324-9546-22e0fead3df1-trusted-ca-bundle\") pod \"console-6cd464788f-6csp8\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:00.048679 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.048611 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/91913017-a3ef-4324-9546-22e0fead3df1-service-ca\") pod \"console-6cd464788f-6csp8\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:00.149615 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.149553 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h2cb\" (UniqueName: \"kubernetes.io/projected/91913017-a3ef-4324-9546-22e0fead3df1-kube-api-access-2h2cb\") pod \"console-6cd464788f-6csp8\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:00.149615 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.149612 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91913017-a3ef-4324-9546-22e0fead3df1-trusted-ca-bundle\") pod \"console-6cd464788f-6csp8\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:00.149876 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.149723 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/91913017-a3ef-4324-9546-22e0fead3df1-service-ca\") pod \"console-6cd464788f-6csp8\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:00.149876 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.149779 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/91913017-a3ef-4324-9546-22e0fead3df1-console-serving-cert\") pod \"console-6cd464788f-6csp8\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:00.149973 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.149959 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/91913017-a3ef-4324-9546-22e0fead3df1-oauth-serving-cert\") pod \"console-6cd464788f-6csp8\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:00.150015 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.149986 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/91913017-a3ef-4324-9546-22e0fead3df1-console-oauth-config\") pod \"console-6cd464788f-6csp8\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:00.150015 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.150002 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/91913017-a3ef-4324-9546-22e0fead3df1-console-config\") pod \"console-6cd464788f-6csp8\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:00.150423 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.150390 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/91913017-a3ef-4324-9546-22e0fead3df1-service-ca\") pod \"console-6cd464788f-6csp8\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:00.150544 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.150514 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/91913017-a3ef-4324-9546-22e0fead3df1-console-config\") pod \"console-6cd464788f-6csp8\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:00.150643 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.150621 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/91913017-a3ef-4324-9546-22e0fead3df1-oauth-serving-cert\") pod \"console-6cd464788f-6csp8\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:00.150792 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.150766 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91913017-a3ef-4324-9546-22e0fead3df1-trusted-ca-bundle\") pod \"console-6cd464788f-6csp8\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:00.152365 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.152341 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/91913017-a3ef-4324-9546-22e0fead3df1-console-serving-cert\") pod \"console-6cd464788f-6csp8\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:00.152459 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.152381 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/91913017-a3ef-4324-9546-22e0fead3df1-console-oauth-config\") pod \"console-6cd464788f-6csp8\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:00.160773 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.160754 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h2cb\" (UniqueName: \"kubernetes.io/projected/91913017-a3ef-4324-9546-22e0fead3df1-kube-api-access-2h2cb\") pod \"console-6cd464788f-6csp8\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:00.325972 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.325868 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:00.453468 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.453363 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cd464788f-6csp8"] Jun 11 21:20:00.455866 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:20:00.455839 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91913017_a3ef_4324_9546_22e0fead3df1.slice/crio-d0cab8c33c77c819cb1c2c604a2651f5caa8b00a5b7bf7295da4c84b90371c97 WatchSource:0}: Error finding container d0cab8c33c77c819cb1c2c604a2651f5caa8b00a5b7bf7295da4c84b90371c97: Status 404 returned error can't find the container with id d0cab8c33c77c819cb1c2c604a2651f5caa8b00a5b7bf7295da4c84b90371c97 Jun 11 21:20:00.944628 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.944594 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cd464788f-6csp8" event={"ID":"91913017-a3ef-4324-9546-22e0fead3df1","Type":"ContainerStarted","Data":"4cffe9eeedeed0853d102a2d7309801437702b6da7b98a3f54cc05c45cebc427"} Jun 11 21:20:00.944796 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.944646 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cd464788f-6csp8" event={"ID":"91913017-a3ef-4324-9546-22e0fead3df1","Type":"ContainerStarted","Data":"d0cab8c33c77c819cb1c2c604a2651f5caa8b00a5b7bf7295da4c84b90371c97"} Jun 11 21:20:00.963436 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:00.963391 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6cd464788f-6csp8" podStartSLOduration=1.9633746159999999 podStartE2EDuration="1.963374616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:19:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:20:00.962153569 +0000 UTC m=+259.373477111" watchObservedRunningTime="2026-06-11 21:20:00.963374616 +0000 UTC m=+259.374698166" Jun 11 21:20:02.938344 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:02.938299 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:20:02.938760 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:02.938384 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:20:02.939497 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:02.939477 2570 patch_prober.go:28] interesting pod/console-5756c675b8-xpf29 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.16:8443/health\": dial tcp 10.134.0.16:8443: connect: connection refused" start-of-body= Jun 11 21:20:02.939542 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:02.939517 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-5756c675b8-xpf29" podUID="6123c52f-479c-4c94-8427-407efd3d891f" containerName="console" probeResult="failure" output="Get \"https://10.134.0.16:8443/health\": dial tcp 10.134.0.16:8443: connect: connection refused" Jun 11 21:20:10.326004 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:10.325963 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:10.326659 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:10.326037 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:10.327439 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:10.327413 2570 patch_prober.go:28] interesting pod/console-6cd464788f-6csp8 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.17:8443/health\": dial tcp 10.134.0.17:8443: connect: connection refused" start-of-body= Jun 11 21:20:10.327528 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:10.327463 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-6cd464788f-6csp8" podUID="91913017-a3ef-4324-9546-22e0fead3df1" containerName="console" probeResult="failure" output="Get \"https://10.134.0.17:8443/health\": dial tcp 10.134.0.17:8443: connect: connection refused" Jun 11 21:20:12.563293 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:12.563242 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:20:12.563710 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:12.563317 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:20:12.939329 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:12.939287 2570 patch_prober.go:28] interesting pod/console-5756c675b8-xpf29 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.16:8443/health\": dial tcp 10.134.0.16:8443: connect: connection refused" start-of-body= Jun 11 21:20:12.939519 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:12.939346 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-5756c675b8-xpf29" podUID="6123c52f-479c-4c94-8427-407efd3d891f" containerName="console" probeResult="failure" output="Get \"https://10.134.0.16:8443/health\": dial tcp 10.134.0.16:8443: connect: connection refused" Jun 11 21:20:14.148198 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:14.148158 2570 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" podUID="e2e186ae-83b2-47b3-9c3c-9e11cc0801eb" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 11 21:20:17.617454 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:17.617410 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-d87ff458-g7pl5" podUID="82296dc1-4d45-4b57-bb06-50853f80ccf8" containerName="console" containerID="cri-o://760c708d8b624b565e786e0528646b662c82b8d01758afd46c17ccf48c29504f" gracePeriod=15 Jun 11 21:20:17.857310 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:17.857285 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d87ff458-g7pl5" Jun 11 21:20:17.992790 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:17.992695 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82296dc1-4d45-4b57-bb06-50853f80ccf8-console-serving-cert\") pod \"82296dc1-4d45-4b57-bb06-50853f80ccf8\" (UID: \"82296dc1-4d45-4b57-bb06-50853f80ccf8\") " Jun 11 21:20:17.992790 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:17.992751 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82296dc1-4d45-4b57-bb06-50853f80ccf8-service-ca\") pod \"82296dc1-4d45-4b57-bb06-50853f80ccf8\" (UID: \"82296dc1-4d45-4b57-bb06-50853f80ccf8\") " Jun 11 21:20:17.992790 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:17.992781 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vqp9q\" (UniqueName: \"kubernetes.io/projected/82296dc1-4d45-4b57-bb06-50853f80ccf8-kube-api-access-vqp9q\") pod \"82296dc1-4d45-4b57-bb06-50853f80ccf8\" (UID: \"82296dc1-4d45-4b57-bb06-50853f80ccf8\") " Jun 11 21:20:17.993065 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:17.992806 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82296dc1-4d45-4b57-bb06-50853f80ccf8-oauth-serving-cert\") pod \"82296dc1-4d45-4b57-bb06-50853f80ccf8\" (UID: \"82296dc1-4d45-4b57-bb06-50853f80ccf8\") " Jun 11 21:20:17.993065 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:17.992823 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82296dc1-4d45-4b57-bb06-50853f80ccf8-console-config\") pod \"82296dc1-4d45-4b57-bb06-50853f80ccf8\" (UID: \"82296dc1-4d45-4b57-bb06-50853f80ccf8\") " Jun 11 21:20:17.993065 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:17.992850 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82296dc1-4d45-4b57-bb06-50853f80ccf8-console-oauth-config\") pod \"82296dc1-4d45-4b57-bb06-50853f80ccf8\" (UID: \"82296dc1-4d45-4b57-bb06-50853f80ccf8\") " Jun 11 21:20:17.993257 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:17.993218 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82296dc1-4d45-4b57-bb06-50853f80ccf8-service-ca" (OuterVolumeSpecName: "service-ca") pod "82296dc1-4d45-4b57-bb06-50853f80ccf8" (UID: "82296dc1-4d45-4b57-bb06-50853f80ccf8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 21:20:17.993316 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:17.993226 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82296dc1-4d45-4b57-bb06-50853f80ccf8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "82296dc1-4d45-4b57-bb06-50853f80ccf8" (UID: "82296dc1-4d45-4b57-bb06-50853f80ccf8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 21:20:17.993365 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:17.993239 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82296dc1-4d45-4b57-bb06-50853f80ccf8-console-config" (OuterVolumeSpecName: "console-config") pod "82296dc1-4d45-4b57-bb06-50853f80ccf8" (UID: "82296dc1-4d45-4b57-bb06-50853f80ccf8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 21:20:17.995088 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:17.995061 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82296dc1-4d45-4b57-bb06-50853f80ccf8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "82296dc1-4d45-4b57-bb06-50853f80ccf8" (UID: "82296dc1-4d45-4b57-bb06-50853f80ccf8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:20:17.995088 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:17.995082 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82296dc1-4d45-4b57-bb06-50853f80ccf8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "82296dc1-4d45-4b57-bb06-50853f80ccf8" (UID: "82296dc1-4d45-4b57-bb06-50853f80ccf8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:20:17.995231 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:17.995092 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82296dc1-4d45-4b57-bb06-50853f80ccf8-kube-api-access-vqp9q" (OuterVolumeSpecName: "kube-api-access-vqp9q") pod "82296dc1-4d45-4b57-bb06-50853f80ccf8" (UID: "82296dc1-4d45-4b57-bb06-50853f80ccf8"). InnerVolumeSpecName "kube-api-access-vqp9q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 21:20:18.005362 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:18.005332 2570 generic.go:356] "Generic (PLEG): container finished" podID="82296dc1-4d45-4b57-bb06-50853f80ccf8" containerID="760c708d8b624b565e786e0528646b662c82b8d01758afd46c17ccf48c29504f" exitCode=0 Jun 11 21:20:18.005475 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:18.005375 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d87ff458-g7pl5" Jun 11 21:20:18.005475 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:18.005412 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d87ff458-g7pl5" event={"ID":"82296dc1-4d45-4b57-bb06-50853f80ccf8","Type":"ContainerDied","Data":"760c708d8b624b565e786e0528646b662c82b8d01758afd46c17ccf48c29504f"} Jun 11 21:20:18.005475 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:18.005460 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d87ff458-g7pl5" event={"ID":"82296dc1-4d45-4b57-bb06-50853f80ccf8","Type":"ContainerDied","Data":"b1be7ea7c90b8be1fa1d4e19c61d4f4bb199d5109ec79f7f362ee40365d65723"} Jun 11 21:20:18.005598 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:18.005481 2570 scope.go:117] "RemoveContainer" containerID="760c708d8b624b565e786e0528646b662c82b8d01758afd46c17ccf48c29504f" Jun 11 21:20:18.013970 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:18.013954 2570 scope.go:117] "RemoveContainer" containerID="760c708d8b624b565e786e0528646b662c82b8d01758afd46c17ccf48c29504f" Jun 11 21:20:18.014249 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:20:18.014225 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"760c708d8b624b565e786e0528646b662c82b8d01758afd46c17ccf48c29504f\": container with ID starting with 760c708d8b624b565e786e0528646b662c82b8d01758afd46c17ccf48c29504f not found: ID does not exist" containerID="760c708d8b624b565e786e0528646b662c82b8d01758afd46c17ccf48c29504f" Jun 11 21:20:18.014342 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:18.014256 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"760c708d8b624b565e786e0528646b662c82b8d01758afd46c17ccf48c29504f"} err="failed to get container status \"760c708d8b624b565e786e0528646b662c82b8d01758afd46c17ccf48c29504f\": rpc error: code = NotFound desc = could not find container \"760c708d8b624b565e786e0528646b662c82b8d01758afd46c17ccf48c29504f\": container with ID starting with 760c708d8b624b565e786e0528646b662c82b8d01758afd46c17ccf48c29504f not found: ID does not exist" Jun 11 21:20:18.025772 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:18.025745 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d87ff458-g7pl5"] Jun 11 21:20:18.027602 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:18.027582 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d87ff458-g7pl5"] Jun 11 21:20:18.051385 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:18.051350 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82296dc1-4d45-4b57-bb06-50853f80ccf8" path="/var/lib/kubelet/pods/82296dc1-4d45-4b57-bb06-50853f80ccf8/volumes" Jun 11 21:20:18.093650 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:18.093618 2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82296dc1-4d45-4b57-bb06-50853f80ccf8-console-serving-cert\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:20:18.093650 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:18.093646 2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82296dc1-4d45-4b57-bb06-50853f80ccf8-service-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:20:18.093650 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:18.093656 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqp9q\" (UniqueName: \"kubernetes.io/projected/82296dc1-4d45-4b57-bb06-50853f80ccf8-kube-api-access-vqp9q\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:20:18.093898 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:18.093665 2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82296dc1-4d45-4b57-bb06-50853f80ccf8-oauth-serving-cert\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:20:18.093898 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:18.093674 2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82296dc1-4d45-4b57-bb06-50853f80ccf8-console-config\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:20:18.093898 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:18.093682 2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82296dc1-4d45-4b57-bb06-50853f80ccf8-console-oauth-config\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:20:20.330997 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:20.330968 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:20.334686 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:20.334666 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:20:20.462710 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:20.462664 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5756c675b8-xpf29"] Jun 11 21:20:22.598552 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:20:22.598505 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-zpqjq" podUID="2c0c15cb-361e-40de-83bf-06ccac34cc83" Jun 11 21:20:23.024115 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:23.024037 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zpqjq" Jun 11 21:20:24.148404 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:24.148368 2570 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" podUID="e2e186ae-83b2-47b3-9c3c-9e11cc0801eb" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 11 21:20:26.363672 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:26.363634 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls\") pod \"dns-default-zpqjq\" (UID: \"2c0c15cb-361e-40de-83bf-06ccac34cc83\") " pod="openshift-dns/dns-default-zpqjq" Jun 11 21:20:26.365948 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:26.365927 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c0c15cb-361e-40de-83bf-06ccac34cc83-metrics-tls\") pod \"dns-default-zpqjq\" (UID: \"2c0c15cb-361e-40de-83bf-06ccac34cc83\") " pod="openshift-dns/dns-default-zpqjq" Jun 11 21:20:26.630209 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:26.630133 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-td2lf\"" Jun 11 21:20:26.634963 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:26.634943 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zpqjq" Jun 11 21:20:26.645092 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:26.645063 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s8b5p_bdfaf7d3-ae13-48e9-b9ea-040ae02d886a/dns-node-resolver/0.log" Jun 11 21:20:26.788863 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:26.788709 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zpqjq"] Jun 11 21:20:26.791623 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:20:26.791595 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c0c15cb_361e_40de_83bf_06ccac34cc83.slice/crio-78f9926eb407fb1b48396bd29640826d709732499faec69c70a5719d9b5889ce WatchSource:0}: Error finding container 78f9926eb407fb1b48396bd29640826d709732499faec69c70a5719d9b5889ce: Status 404 returned error can't find the container with id 78f9926eb407fb1b48396bd29640826d709732499faec69c70a5719d9b5889ce Jun 11 21:20:27.041022 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:27.040933 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zpqjq" event={"ID":"2c0c15cb-361e-40de-83bf-06ccac34cc83","Type":"ContainerStarted","Data":"78f9926eb407fb1b48396bd29640826d709732499faec69c70a5719d9b5889ce"} Jun 11 21:20:27.229470 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:27.229439 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-f5djm_5317cca9-f317-4c21-9dad-a2f1ac629264/serve-healthcheck-canary/0.log" Jun 11 21:20:29.050250 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:29.050162 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zpqjq" event={"ID":"2c0c15cb-361e-40de-83bf-06ccac34cc83","Type":"ContainerStarted","Data":"430581ea9754afb3899b5c18ffea93c0618efdb900cbdc8736127be457b1da6f"} Jun 11 21:20:29.050250 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:29.050216 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zpqjq" event={"ID":"2c0c15cb-361e-40de-83bf-06ccac34cc83","Type":"ContainerStarted","Data":"6d1f3a6174e1fdbad54f5204da3c9985936d02dd4c1f6e22ae95223bd028f003"} Jun 11 21:20:29.050657 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:29.050296 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zpqjq" Jun 11 21:20:29.072397 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:29.072225 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zpqjq" podStartSLOduration=251.204440616 podStartE2EDuration="4m13.072200657s" totalImagesPullingTime="1.867760041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:16:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:20:29.072050161 +0000 UTC m=+287.483373700" watchObservedRunningTime="2026-06-11 21:20:29.072200657 +0000 UTC m=+287.483524196" Jun 11 21:20:32.569185 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:32.569153 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:20:32.573032 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:32.573007 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7d868dd696-hhxbl" Jun 11 21:20:34.148333 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:34.148291 2570 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" podUID="e2e186ae-83b2-47b3-9c3c-9e11cc0801eb" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 11 21:20:34.148826 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:34.148379 2570 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" Jun 11 21:20:34.149007 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:34.148986 2570 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"07f5c7e516d987dadd538f8b56cd2121a7701ef2f3dd29850b4af6f1b446f143"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jun 11 21:20:34.149080 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:34.149034 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" podUID="e2e186ae-83b2-47b3-9c3c-9e11cc0801eb" containerName="service-proxy" containerID="cri-o://07f5c7e516d987dadd538f8b56cd2121a7701ef2f3dd29850b4af6f1b446f143" gracePeriod=30 Jun 11 21:20:35.074857 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:35.074816 2570 generic.go:356] "Generic (PLEG): container finished" podID="e2e186ae-83b2-47b3-9c3c-9e11cc0801eb" containerID="07f5c7e516d987dadd538f8b56cd2121a7701ef2f3dd29850b4af6f1b446f143" exitCode=2 Jun 11 21:20:35.075042 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:35.074877 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" event={"ID":"e2e186ae-83b2-47b3-9c3c-9e11cc0801eb","Type":"ContainerDied","Data":"07f5c7e516d987dadd538f8b56cd2121a7701ef2f3dd29850b4af6f1b446f143"} Jun 11 21:20:35.075042 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:35.074919 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b498c6f65-xwdn5" event={"ID":"e2e186ae-83b2-47b3-9c3c-9e11cc0801eb","Type":"ContainerStarted","Data":"8a31e7f86eee81f42daf74adc9e1ce11fe1b3e0204e4bb71ddc5a6e941440c48"} Jun 11 21:20:39.055510 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:39.055478 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zpqjq" Jun 11 21:20:42.011518 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:42.011491 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/ovn-acl-logging/0.log" Jun 11 21:20:42.011961 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:42.011711 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/ovn-acl-logging/0.log" Jun 11 21:20:42.017178 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:42.017158 2570 kubelet.go:1648] "Image garbage collection succeeded" Jun 11 21:20:45.481712 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:45.481645 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5756c675b8-xpf29" podUID="6123c52f-479c-4c94-8427-407efd3d891f" containerName="console" containerID="cri-o://85a2014e904f965c080d11290a2cb44e3e643c1610c8b90a9ed93d9df61aea29" gracePeriod=15 Jun 11 21:20:45.723094 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:45.723065 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:20:45.826883 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:45.826787 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6123c52f-479c-4c94-8427-407efd3d891f-service-ca\") pod \"6123c52f-479c-4c94-8427-407efd3d891f\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " Jun 11 21:20:45.826883 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:45.826859 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6123c52f-479c-4c94-8427-407efd3d891f-oauth-serving-cert\") pod \"6123c52f-479c-4c94-8427-407efd3d891f\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " Jun 11 21:20:45.827116 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:45.826890 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6123c52f-479c-4c94-8427-407efd3d891f-console-oauth-config\") pod \"6123c52f-479c-4c94-8427-407efd3d891f\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " Jun 11 21:20:45.827116 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:45.826908 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltqp9\" (UniqueName: \"kubernetes.io/projected/6123c52f-479c-4c94-8427-407efd3d891f-kube-api-access-ltqp9\") pod \"6123c52f-479c-4c94-8427-407efd3d891f\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " Jun 11 21:20:45.827116 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:45.826934 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6123c52f-479c-4c94-8427-407efd3d891f-console-config\") pod \"6123c52f-479c-4c94-8427-407efd3d891f\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " Jun 11 21:20:45.827116 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:45.826988 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6123c52f-479c-4c94-8427-407efd3d891f-trusted-ca-bundle\") pod \"6123c52f-479c-4c94-8427-407efd3d891f\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " Jun 11 21:20:45.827116 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:45.827013 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6123c52f-479c-4c94-8427-407efd3d891f-console-serving-cert\") pod \"6123c52f-479c-4c94-8427-407efd3d891f\" (UID: \"6123c52f-479c-4c94-8427-407efd3d891f\") " Jun 11 21:20:45.827370 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:45.827336 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6123c52f-479c-4c94-8427-407efd3d891f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6123c52f-479c-4c94-8427-407efd3d891f" (UID: "6123c52f-479c-4c94-8427-407efd3d891f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 21:20:45.827415 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:45.827357 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6123c52f-479c-4c94-8427-407efd3d891f-service-ca" (OuterVolumeSpecName: "service-ca") pod "6123c52f-479c-4c94-8427-407efd3d891f" (UID: "6123c52f-479c-4c94-8427-407efd3d891f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 21:20:45.827456 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:45.827428 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6123c52f-479c-4c94-8427-407efd3d891f-console-config" (OuterVolumeSpecName: "console-config") pod "6123c52f-479c-4c94-8427-407efd3d891f" (UID: "6123c52f-479c-4c94-8427-407efd3d891f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 21:20:45.827456 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:45.827439 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6123c52f-479c-4c94-8427-407efd3d891f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6123c52f-479c-4c94-8427-407efd3d891f" (UID: "6123c52f-479c-4c94-8427-407efd3d891f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 21:20:45.829131 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:45.829106 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6123c52f-479c-4c94-8427-407efd3d891f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6123c52f-479c-4c94-8427-407efd3d891f" (UID: "6123c52f-479c-4c94-8427-407efd3d891f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:20:45.829249 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:45.829156 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6123c52f-479c-4c94-8427-407efd3d891f-kube-api-access-ltqp9" (OuterVolumeSpecName: "kube-api-access-ltqp9") pod "6123c52f-479c-4c94-8427-407efd3d891f" (UID: "6123c52f-479c-4c94-8427-407efd3d891f"). InnerVolumeSpecName "kube-api-access-ltqp9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 21:20:45.829249 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:45.829179 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6123c52f-479c-4c94-8427-407efd3d891f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6123c52f-479c-4c94-8427-407efd3d891f" (UID: "6123c52f-479c-4c94-8427-407efd3d891f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:20:45.928446 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:45.928415 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6123c52f-479c-4c94-8427-407efd3d891f-trusted-ca-bundle\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:20:45.928446 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:45.928441 2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6123c52f-479c-4c94-8427-407efd3d891f-console-serving-cert\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:20:45.928446 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:45.928451 2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6123c52f-479c-4c94-8427-407efd3d891f-service-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:20:45.928710 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:45.928460 2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6123c52f-479c-4c94-8427-407efd3d891f-oauth-serving-cert\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:20:45.928710 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:45.928469 2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6123c52f-479c-4c94-8427-407efd3d891f-console-oauth-config\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:20:45.928710 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:45.928479 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltqp9\" (UniqueName: \"kubernetes.io/projected/6123c52f-479c-4c94-8427-407efd3d891f-kube-api-access-ltqp9\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:20:45.928710 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:45.928487 2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6123c52f-479c-4c94-8427-407efd3d891f-console-config\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:20:46.124066 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:46.124033 2570 generic.go:356] "Generic (PLEG): container finished" podID="6123c52f-479c-4c94-8427-407efd3d891f" containerID="85a2014e904f965c080d11290a2cb44e3e643c1610c8b90a9ed93d9df61aea29" exitCode=0 Jun 11 21:20:46.124259 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:46.124088 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5756c675b8-xpf29" event={"ID":"6123c52f-479c-4c94-8427-407efd3d891f","Type":"ContainerDied","Data":"85a2014e904f965c080d11290a2cb44e3e643c1610c8b90a9ed93d9df61aea29"} Jun 11 21:20:46.124259 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:46.124104 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5756c675b8-xpf29" Jun 11 21:20:46.124259 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:46.124150 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5756c675b8-xpf29" event={"ID":"6123c52f-479c-4c94-8427-407efd3d891f","Type":"ContainerDied","Data":"254be746b85627a786e76060e022e1ed07a8f8f1ecc39be5f126344261c7cdd6"} Jun 11 21:20:46.124259 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:46.124169 2570 scope.go:117] "RemoveContainer" containerID="85a2014e904f965c080d11290a2cb44e3e643c1610c8b90a9ed93d9df61aea29" Jun 11 21:20:46.132481 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:46.132465 2570 scope.go:117] "RemoveContainer" containerID="85a2014e904f965c080d11290a2cb44e3e643c1610c8b90a9ed93d9df61aea29" Jun 11 21:20:46.132762 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:20:46.132742 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85a2014e904f965c080d11290a2cb44e3e643c1610c8b90a9ed93d9df61aea29\": container with ID starting with 85a2014e904f965c080d11290a2cb44e3e643c1610c8b90a9ed93d9df61aea29 not found: ID does not exist" containerID="85a2014e904f965c080d11290a2cb44e3e643c1610c8b90a9ed93d9df61aea29" Jun 11 21:20:46.132834 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:46.132774 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85a2014e904f965c080d11290a2cb44e3e643c1610c8b90a9ed93d9df61aea29"} err="failed to get container status \"85a2014e904f965c080d11290a2cb44e3e643c1610c8b90a9ed93d9df61aea29\": rpc error: code = NotFound desc = could not find container \"85a2014e904f965c080d11290a2cb44e3e643c1610c8b90a9ed93d9df61aea29\": container with ID starting with 85a2014e904f965c080d11290a2cb44e3e643c1610c8b90a9ed93d9df61aea29 not found: ID does not exist" Jun 11 21:20:46.139431 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:46.139410 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5756c675b8-xpf29"] Jun 11 21:20:46.146641 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:46.146622 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5756c675b8-xpf29"] Jun 11 21:20:48.051192 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:20:48.051149 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6123c52f-479c-4c94-8427-407efd3d891f" path="/var/lib/kubelet/pods/6123c52f-479c-4c94-8427-407efd3d891f/volumes" Jun 11 21:21:20.284161 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:20.284124 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cd464788f-6csp8"] Jun 11 21:21:45.303772 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:45.303725 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6cd464788f-6csp8" podUID="91913017-a3ef-4324-9546-22e0fead3df1" containerName="console" containerID="cri-o://4cffe9eeedeed0853d102a2d7309801437702b6da7b98a3f54cc05c45cebc427" gracePeriod=15 Jun 11 21:21:45.544368 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:45.544344 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:21:45.665937 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:45.665899 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/91913017-a3ef-4324-9546-22e0fead3df1-console-config\") pod \"91913017-a3ef-4324-9546-22e0fead3df1\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " Jun 11 21:21:45.666137 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:45.665950 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/91913017-a3ef-4324-9546-22e0fead3df1-oauth-serving-cert\") pod \"91913017-a3ef-4324-9546-22e0fead3df1\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " Jun 11 21:21:45.666137 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:45.665971 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/91913017-a3ef-4324-9546-22e0fead3df1-console-serving-cert\") pod \"91913017-a3ef-4324-9546-22e0fead3df1\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " Jun 11 21:21:45.666137 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:45.666009 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2h2cb\" (UniqueName: \"kubernetes.io/projected/91913017-a3ef-4324-9546-22e0fead3df1-kube-api-access-2h2cb\") pod \"91913017-a3ef-4324-9546-22e0fead3df1\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " Jun 11 21:21:45.666312 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:45.666177 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91913017-a3ef-4324-9546-22e0fead3df1-trusted-ca-bundle\") pod \"91913017-a3ef-4324-9546-22e0fead3df1\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " Jun 11 21:21:45.666312 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:45.666248 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/91913017-a3ef-4324-9546-22e0fead3df1-console-oauth-config\") pod \"91913017-a3ef-4324-9546-22e0fead3df1\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " Jun 11 21:21:45.666312 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:45.666283 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/91913017-a3ef-4324-9546-22e0fead3df1-service-ca\") pod \"91913017-a3ef-4324-9546-22e0fead3df1\" (UID: \"91913017-a3ef-4324-9546-22e0fead3df1\") " Jun 11 21:21:45.666486 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:45.666461 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91913017-a3ef-4324-9546-22e0fead3df1-console-config" (OuterVolumeSpecName: "console-config") pod "91913017-a3ef-4324-9546-22e0fead3df1" (UID: "91913017-a3ef-4324-9546-22e0fead3df1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 21:21:45.666486 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:45.666469 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91913017-a3ef-4324-9546-22e0fead3df1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "91913017-a3ef-4324-9546-22e0fead3df1" (UID: "91913017-a3ef-4324-9546-22e0fead3df1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 21:21:45.666614 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:45.666582 2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/91913017-a3ef-4324-9546-22e0fead3df1-console-config\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:21:45.666614 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:45.666602 2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/91913017-a3ef-4324-9546-22e0fead3df1-oauth-serving-cert\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:21:45.666688 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:45.666651 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91913017-a3ef-4324-9546-22e0fead3df1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "91913017-a3ef-4324-9546-22e0fead3df1" (UID: "91913017-a3ef-4324-9546-22e0fead3df1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 21:21:45.666734 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:45.666703 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91913017-a3ef-4324-9546-22e0fead3df1-service-ca" (OuterVolumeSpecName: "service-ca") pod "91913017-a3ef-4324-9546-22e0fead3df1" (UID: "91913017-a3ef-4324-9546-22e0fead3df1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 21:21:45.668216 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:45.668188 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91913017-a3ef-4324-9546-22e0fead3df1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "91913017-a3ef-4324-9546-22e0fead3df1" (UID: "91913017-a3ef-4324-9546-22e0fead3df1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:21:45.668216 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:45.668202 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91913017-a3ef-4324-9546-22e0fead3df1-kube-api-access-2h2cb" (OuterVolumeSpecName: "kube-api-access-2h2cb") pod "91913017-a3ef-4324-9546-22e0fead3df1" (UID: "91913017-a3ef-4324-9546-22e0fead3df1"). InnerVolumeSpecName "kube-api-access-2h2cb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 21:21:45.668492 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:45.668470 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91913017-a3ef-4324-9546-22e0fead3df1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "91913017-a3ef-4324-9546-22e0fead3df1" (UID: "91913017-a3ef-4324-9546-22e0fead3df1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:21:45.767585 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:45.767505 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2h2cb\" (UniqueName: \"kubernetes.io/projected/91913017-a3ef-4324-9546-22e0fead3df1-kube-api-access-2h2cb\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:21:45.767585 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:45.767555 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91913017-a3ef-4324-9546-22e0fead3df1-trusted-ca-bundle\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:21:45.767585 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:45.767593 2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/91913017-a3ef-4324-9546-22e0fead3df1-console-oauth-config\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:21:45.767826 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:45.767605 2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/91913017-a3ef-4324-9546-22e0fead3df1-service-ca\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:21:45.767826 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:45.767618 2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/91913017-a3ef-4324-9546-22e0fead3df1-console-serving-cert\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:21:46.339307 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:46.339273 2570 generic.go:356] "Generic (PLEG): container finished" podID="91913017-a3ef-4324-9546-22e0fead3df1" containerID="4cffe9eeedeed0853d102a2d7309801437702b6da7b98a3f54cc05c45cebc427" exitCode=0 Jun 11 21:21:46.339779 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:46.339321 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cd464788f-6csp8" event={"ID":"91913017-a3ef-4324-9546-22e0fead3df1","Type":"ContainerDied","Data":"4cffe9eeedeed0853d102a2d7309801437702b6da7b98a3f54cc05c45cebc427"} Jun 11 21:21:46.339779 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:46.339336 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cd464788f-6csp8" Jun 11 21:21:46.339779 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:46.339359 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cd464788f-6csp8" event={"ID":"91913017-a3ef-4324-9546-22e0fead3df1","Type":"ContainerDied","Data":"d0cab8c33c77c819cb1c2c604a2651f5caa8b00a5b7bf7295da4c84b90371c97"} Jun 11 21:21:46.339779 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:46.339377 2570 scope.go:117] "RemoveContainer" containerID="4cffe9eeedeed0853d102a2d7309801437702b6da7b98a3f54cc05c45cebc427" Jun 11 21:21:46.347148 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:46.347124 2570 scope.go:117] "RemoveContainer" containerID="4cffe9eeedeed0853d102a2d7309801437702b6da7b98a3f54cc05c45cebc427" Jun 11 21:21:46.347399 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:21:46.347381 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cffe9eeedeed0853d102a2d7309801437702b6da7b98a3f54cc05c45cebc427\": container with ID starting with 4cffe9eeedeed0853d102a2d7309801437702b6da7b98a3f54cc05c45cebc427 not found: ID does not exist" containerID="4cffe9eeedeed0853d102a2d7309801437702b6da7b98a3f54cc05c45cebc427" Jun 11 21:21:46.347459 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:46.347412 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cffe9eeedeed0853d102a2d7309801437702b6da7b98a3f54cc05c45cebc427"} err="failed to get container status \"4cffe9eeedeed0853d102a2d7309801437702b6da7b98a3f54cc05c45cebc427\": rpc error: code = NotFound desc = could not find container \"4cffe9eeedeed0853d102a2d7309801437702b6da7b98a3f54cc05c45cebc427\": container with ID starting with 4cffe9eeedeed0853d102a2d7309801437702b6da7b98a3f54cc05c45cebc427 not found: ID does not exist" Jun 11 21:21:46.353877 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:46.353856 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cd464788f-6csp8"] Jun 11 21:21:46.357420 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:46.357398 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6cd464788f-6csp8"] Jun 11 21:21:48.050698 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:21:48.050668 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91913017-a3ef-4324-9546-22e0fead3df1" path="/var/lib/kubelet/pods/91913017-a3ef-4324-9546-22e0fead3df1/volumes" Jun 11 21:25:16.185239 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.185207 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-74f5576b77-cplfp"] Jun 11 21:25:16.185737 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.185482 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82296dc1-4d45-4b57-bb06-50853f80ccf8" containerName="console" Jun 11 21:25:16.185737 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.185495 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="82296dc1-4d45-4b57-bb06-50853f80ccf8" containerName="console" Jun 11 21:25:16.185737 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.185504 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91913017-a3ef-4324-9546-22e0fead3df1" containerName="console" Jun 11 21:25:16.185737 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.185509 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="91913017-a3ef-4324-9546-22e0fead3df1" containerName="console" Jun 11 21:25:16.185737 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.185519 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6123c52f-479c-4c94-8427-407efd3d891f" containerName="console" Jun 11 21:25:16.185737 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.185525 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6123c52f-479c-4c94-8427-407efd3d891f" containerName="console" Jun 11 21:25:16.185737 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.185578 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="91913017-a3ef-4324-9546-22e0fead3df1" containerName="console" Jun 11 21:25:16.185737 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.185587 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="6123c52f-479c-4c94-8427-407efd3d891f" containerName="console" Jun 11 21:25:16.185737 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.185594 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="82296dc1-4d45-4b57-bb06-50853f80ccf8" containerName="console" Jun 11 21:25:16.187497 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.187477 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:16.189804 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.189781 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 11 21:25:16.189933 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.189912 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 11 21:25:16.190034 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.190020 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 11 21:25:16.190114 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.190100 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 11 21:25:16.190946 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.190919 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-z2vmg\"" Jun 11 21:25:16.191050 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.190944 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 11 21:25:16.191050 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.190973 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 11 21:25:16.191050 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.190982 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 11 21:25:16.195740 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.195721 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 11 21:25:16.200143 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.200124 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74f5576b77-cplfp"] Jun 11 21:25:16.220068 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.220044 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cea20d06-45c0-4aa9-811d-4e80e2285568-trusted-ca-bundle\") pod \"console-74f5576b77-cplfp\" (UID: \"cea20d06-45c0-4aa9-811d-4e80e2285568\") " pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:16.220068 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.220071 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cea20d06-45c0-4aa9-811d-4e80e2285568-service-ca\") pod \"console-74f5576b77-cplfp\" (UID: \"cea20d06-45c0-4aa9-811d-4e80e2285568\") " pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:16.220232 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.220096 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cea20d06-45c0-4aa9-811d-4e80e2285568-console-config\") pod \"console-74f5576b77-cplfp\" (UID: \"cea20d06-45c0-4aa9-811d-4e80e2285568\") " pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:16.220232 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.220142 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c65c7\" (UniqueName: \"kubernetes.io/projected/cea20d06-45c0-4aa9-811d-4e80e2285568-kube-api-access-c65c7\") pod \"console-74f5576b77-cplfp\" (UID: \"cea20d06-45c0-4aa9-811d-4e80e2285568\") " pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:16.220232 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.220195 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cea20d06-45c0-4aa9-811d-4e80e2285568-console-serving-cert\") pod \"console-74f5576b77-cplfp\" (UID: \"cea20d06-45c0-4aa9-811d-4e80e2285568\") " pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:16.220328 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.220245 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cea20d06-45c0-4aa9-811d-4e80e2285568-console-oauth-config\") pod \"console-74f5576b77-cplfp\" (UID: \"cea20d06-45c0-4aa9-811d-4e80e2285568\") " pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:16.220328 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.220281 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cea20d06-45c0-4aa9-811d-4e80e2285568-oauth-serving-cert\") pod \"console-74f5576b77-cplfp\" (UID: \"cea20d06-45c0-4aa9-811d-4e80e2285568\") " pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:16.321422 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.321383 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cea20d06-45c0-4aa9-811d-4e80e2285568-console-oauth-config\") pod \"console-74f5576b77-cplfp\" (UID: \"cea20d06-45c0-4aa9-811d-4e80e2285568\") " pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:16.321422 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.321425 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cea20d06-45c0-4aa9-811d-4e80e2285568-oauth-serving-cert\") pod \"console-74f5576b77-cplfp\" (UID: \"cea20d06-45c0-4aa9-811d-4e80e2285568\") " pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:16.321683 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.321457 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cea20d06-45c0-4aa9-811d-4e80e2285568-trusted-ca-bundle\") pod \"console-74f5576b77-cplfp\" (UID: \"cea20d06-45c0-4aa9-811d-4e80e2285568\") " pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:16.321683 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.321476 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cea20d06-45c0-4aa9-811d-4e80e2285568-service-ca\") pod \"console-74f5576b77-cplfp\" (UID: \"cea20d06-45c0-4aa9-811d-4e80e2285568\") " pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:16.321683 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.321608 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cea20d06-45c0-4aa9-811d-4e80e2285568-console-config\") pod \"console-74f5576b77-cplfp\" (UID: \"cea20d06-45c0-4aa9-811d-4e80e2285568\") " pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:16.321683 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.321638 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c65c7\" (UniqueName: \"kubernetes.io/projected/cea20d06-45c0-4aa9-811d-4e80e2285568-kube-api-access-c65c7\") pod \"console-74f5576b77-cplfp\" (UID: \"cea20d06-45c0-4aa9-811d-4e80e2285568\") " pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:16.321683 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.321673 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cea20d06-45c0-4aa9-811d-4e80e2285568-console-serving-cert\") pod \"console-74f5576b77-cplfp\" (UID: \"cea20d06-45c0-4aa9-811d-4e80e2285568\") " pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:16.322199 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.322170 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cea20d06-45c0-4aa9-811d-4e80e2285568-oauth-serving-cert\") pod \"console-74f5576b77-cplfp\" (UID: \"cea20d06-45c0-4aa9-811d-4e80e2285568\") " pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:16.322311 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.322213 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cea20d06-45c0-4aa9-811d-4e80e2285568-service-ca\") pod \"console-74f5576b77-cplfp\" (UID: \"cea20d06-45c0-4aa9-811d-4e80e2285568\") " pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:16.322311 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.322277 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cea20d06-45c0-4aa9-811d-4e80e2285568-console-config\") pod \"console-74f5576b77-cplfp\" (UID: \"cea20d06-45c0-4aa9-811d-4e80e2285568\") " pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:16.322470 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.322447 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cea20d06-45c0-4aa9-811d-4e80e2285568-trusted-ca-bundle\") pod \"console-74f5576b77-cplfp\" (UID: \"cea20d06-45c0-4aa9-811d-4e80e2285568\") " pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:16.323978 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.323954 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cea20d06-45c0-4aa9-811d-4e80e2285568-console-serving-cert\") pod \"console-74f5576b77-cplfp\" (UID: \"cea20d06-45c0-4aa9-811d-4e80e2285568\") " pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:16.324086 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.324065 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cea20d06-45c0-4aa9-811d-4e80e2285568-console-oauth-config\") pod \"console-74f5576b77-cplfp\" (UID: \"cea20d06-45c0-4aa9-811d-4e80e2285568\") " pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:16.329403 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.329383 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c65c7\" (UniqueName: \"kubernetes.io/projected/cea20d06-45c0-4aa9-811d-4e80e2285568-kube-api-access-c65c7\") pod \"console-74f5576b77-cplfp\" (UID: \"cea20d06-45c0-4aa9-811d-4e80e2285568\") " pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:16.497879 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.497777 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:16.621599 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:16.621543 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74f5576b77-cplfp"] Jun 11 21:25:16.624644 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:25:16.624610 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcea20d06_45c0_4aa9_811d_4e80e2285568.slice/crio-5e62da74de1802bed964b52c71f93ff78e042555faa459c91af97c32c42ba59d WatchSource:0}: Error finding container 5e62da74de1802bed964b52c71f93ff78e042555faa459c91af97c32c42ba59d: Status 404 returned error can't find the container with id 5e62da74de1802bed964b52c71f93ff78e042555faa459c91af97c32c42ba59d Jun 11 21:25:17.063394 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:17.063360 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74f5576b77-cplfp" event={"ID":"cea20d06-45c0-4aa9-811d-4e80e2285568","Type":"ContainerStarted","Data":"ee377186a47d73202964156051a74c96c7cebc7be019d9dccd3f0cad28d30ed0"} Jun 11 21:25:17.063711 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:17.063407 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74f5576b77-cplfp" event={"ID":"cea20d06-45c0-4aa9-811d-4e80e2285568","Type":"ContainerStarted","Data":"5e62da74de1802bed964b52c71f93ff78e042555faa459c91af97c32c42ba59d"} Jun 11 21:25:17.082401 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:17.082344 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-74f5576b77-cplfp" podStartSLOduration=1.082329365 podStartE2EDuration="1.082329365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:25:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:25:17.081456381 +0000 UTC m=+575.492779959" watchObservedRunningTime="2026-06-11 21:25:17.082329365 +0000 UTC m=+575.493652902" Jun 11 21:25:26.498765 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:26.498711 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:26.499276 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:26.498813 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:26.503657 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:26.503627 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:27.102677 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:27.102649 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-74f5576b77-cplfp" Jun 11 21:25:42.033686 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:42.033655 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/ovn-acl-logging/0.log" Jun 11 21:25:42.034660 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:25:42.034633 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/ovn-acl-logging/0.log" Jun 11 21:30:01.912365 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:01.912329 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-test-apply-q86qn"] Jun 11 21:30:01.914387 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:01.914371 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-q86qn" Jun 11 21:30:01.916838 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:01.916816 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-2r82d\"" Jun 11 21:30:01.916936 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:01.916816 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jun 11 21:30:01.917791 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:01.917772 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jun 11 21:30:01.917895 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:01.917804 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Jun 11 21:30:01.922333 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:01.922311 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-q86qn"] Jun 11 21:30:02.014822 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:02.014786 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkh9n\" (UniqueName: \"kubernetes.io/projected/91a4f040-0576-4093-b9ae-8b371ba48e19-kube-api-access-vkh9n\") pod \"feast-test-apply-q86qn\" (UID: \"91a4f040-0576-4093-b9ae-8b371ba48e19\") " pod="test-ns-feast/feast-test-apply-q86qn" Jun 11 21:30:02.115434 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:02.115392 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkh9n\" (UniqueName: \"kubernetes.io/projected/91a4f040-0576-4093-b9ae-8b371ba48e19-kube-api-access-vkh9n\") pod \"feast-test-apply-q86qn\" (UID: \"91a4f040-0576-4093-b9ae-8b371ba48e19\") " pod="test-ns-feast/feast-test-apply-q86qn" Jun 11 21:30:02.124189 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:02.124158 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkh9n\" (UniqueName: \"kubernetes.io/projected/91a4f040-0576-4093-b9ae-8b371ba48e19-kube-api-access-vkh9n\") pod \"feast-test-apply-q86qn\" (UID: \"91a4f040-0576-4093-b9ae-8b371ba48e19\") " pod="test-ns-feast/feast-test-apply-q86qn" Jun 11 21:30:02.223412 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:02.223328 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-q86qn" Jun 11 21:30:02.348196 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:02.348165 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-q86qn"] Jun 11 21:30:02.351469 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:30:02.351438 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod91a4f040_0576_4093_b9ae_8b371ba48e19.slice/crio-32c54434e350208048df0ebd4215818b817fe620c6dba8c0262fd0a0cda451be WatchSource:0}: Error finding container 32c54434e350208048df0ebd4215818b817fe620c6dba8c0262fd0a0cda451be: Status 404 returned error can't find the container with id 32c54434e350208048df0ebd4215818b817fe620c6dba8c0262fd0a0cda451be Jun 11 21:30:02.353330 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:02.353316 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 21:30:03.065454 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:03.065413 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-q86qn" event={"ID":"91a4f040-0576-4093-b9ae-8b371ba48e19","Type":"ContainerStarted","Data":"32c54434e350208048df0ebd4215818b817fe620c6dba8c0262fd0a0cda451be"} Jun 11 21:30:08.088148 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:08.088108 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-q86qn" event={"ID":"91a4f040-0576-4093-b9ae-8b371ba48e19","Type":"ContainerStarted","Data":"0c927d50a0ff92b1b9e053b10a5f3febb4e6bd4045a146cc9b496afde800a978"} Jun 11 21:30:15.115373 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:15.115341 2570 generic.go:356] "Generic (PLEG): container finished" podID="91a4f040-0576-4093-b9ae-8b371ba48e19" containerID="0c927d50a0ff92b1b9e053b10a5f3febb4e6bd4045a146cc9b496afde800a978" exitCode=0 Jun 11 21:30:15.115833 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:15.115387 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-q86qn" event={"ID":"91a4f040-0576-4093-b9ae-8b371ba48e19","Type":"ContainerDied","Data":"0c927d50a0ff92b1b9e053b10a5f3febb4e6bd4045a146cc9b496afde800a978"} Jun 11 21:30:16.120783 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:16.120751 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-q86qn" event={"ID":"91a4f040-0576-4093-b9ae-8b371ba48e19","Type":"ContainerStarted","Data":"7cb6ab090668b9d218bf7cc069d8e535b3d987993cd1c60b3f703fd6f37fdfb7"} Jun 11 21:30:16.137085 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:16.137013 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-test-apply-q86qn" podStartSLOduration=3.164443842 podStartE2EDuration="15.136996989s" totalImagesPullingTime="5.537486585s" totalInitContainerRuntime="6.435066562s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:30:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:30:16.136080003 +0000 UTC m=+874.547403541" watchObservedRunningTime="2026-06-11 21:30:16.136996989 +0000 UTC m=+874.548320526" Jun 11 21:30:25.154463 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:25.154429 2570 generic.go:356] "Generic (PLEG): container finished" podID="91a4f040-0576-4093-b9ae-8b371ba48e19" containerID="7cb6ab090668b9d218bf7cc069d8e535b3d987993cd1c60b3f703fd6f37fdfb7" exitCode=0 Jun 11 21:30:25.154879 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:25.154478 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-q86qn" event={"ID":"91a4f040-0576-4093-b9ae-8b371ba48e19","Type":"ContainerDied","Data":"7cb6ab090668b9d218bf7cc069d8e535b3d987993cd1c60b3f703fd6f37fdfb7"} Jun 11 21:30:26.293654 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:26.293629 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-q86qn" Jun 11 21:30:26.396425 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:26.396387 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkh9n\" (UniqueName: \"kubernetes.io/projected/91a4f040-0576-4093-b9ae-8b371ba48e19-kube-api-access-vkh9n\") pod \"91a4f040-0576-4093-b9ae-8b371ba48e19\" (UID: \"91a4f040-0576-4093-b9ae-8b371ba48e19\") " Jun 11 21:30:26.398627 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:26.398602 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91a4f040-0576-4093-b9ae-8b371ba48e19-kube-api-access-vkh9n" (OuterVolumeSpecName: "kube-api-access-vkh9n") pod "91a4f040-0576-4093-b9ae-8b371ba48e19" (UID: "91a4f040-0576-4093-b9ae-8b371ba48e19"). InnerVolumeSpecName "kube-api-access-vkh9n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 21:30:26.497483 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:26.497390 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkh9n\" (UniqueName: \"kubernetes.io/projected/91a4f040-0576-4093-b9ae-8b371ba48e19-kube-api-access-vkh9n\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:30:27.163549 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:27.163522 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-q86qn" Jun 11 21:30:27.163733 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:27.163516 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-q86qn" event={"ID":"91a4f040-0576-4093-b9ae-8b371ba48e19","Type":"ContainerDied","Data":"32c54434e350208048df0ebd4215818b817fe620c6dba8c0262fd0a0cda451be"} Jun 11 21:30:27.163733 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:27.163640 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="32c54434e350208048df0ebd4215818b817fe620c6dba8c0262fd0a0cda451be" Jun 11 21:30:27.343109 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:27.343054 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-q86qn_91a4f040-0576-4093-b9ae-8b371ba48e19/feast-0/0.log" Jun 11 21:30:27.349570 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:27.349547 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-q86qn_91a4f040-0576-4093-b9ae-8b371ba48e19/feast-1/0.log" Jun 11 21:30:42.055828 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:42.055753 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/ovn-acl-logging/0.log" Jun 11 21:30:42.057920 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:30:42.057900 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/ovn-acl-logging/0.log" Jun 11 21:31:05.535397 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:31:05.535362 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-test-apply-q86qn"] Jun 11 21:31:05.540904 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:31:05.540878 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-test-apply-q86qn"] Jun 11 21:31:06.050904 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:31:06.050864 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91a4f040-0576-4093-b9ae-8b371ba48e19" path="/var/lib/kubelet/pods/91a4f040-0576-4093-b9ae-8b371ba48e19/volumes" Jun 11 21:35:42.077313 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:35:42.077282 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/ovn-acl-logging/0.log" Jun 11 21:35:42.080300 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:35:42.080278 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/ovn-acl-logging/0.log" Jun 11 21:36:08.325203 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:08.325166 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6d54c7db9d-r49qx"] Jun 11 21:36:08.325626 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:08.325413 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91a4f040-0576-4093-b9ae-8b371ba48e19" containerName="feast-1" Jun 11 21:36:08.325626 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:08.325423 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="91a4f040-0576-4093-b9ae-8b371ba48e19" containerName="feast-1" Jun 11 21:36:08.325626 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:08.325432 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91a4f040-0576-4093-b9ae-8b371ba48e19" containerName="feast-0" Jun 11 21:36:08.325626 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:08.325438 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="91a4f040-0576-4093-b9ae-8b371ba48e19" containerName="feast-0" Jun 11 21:36:08.325626 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:08.325485 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="91a4f040-0576-4093-b9ae-8b371ba48e19" containerName="feast-1" Jun 11 21:36:08.327213 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:08.327198 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-r49qx" Jun 11 21:36:08.330765 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:08.330739 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Jun 11 21:36:08.331485 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:08.331453 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-725cf\"" Jun 11 21:36:08.331805 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:08.331786 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Jun 11 21:36:08.345800 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:08.345771 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rq9q\" (UniqueName: \"kubernetes.io/projected/9c3eb1b5-0528-4273-8331-9ef8676bfa1d-kube-api-access-8rq9q\") pod \"feast-operator-controller-manager-6d54c7db9d-r49qx\" (UID: \"9c3eb1b5-0528-4273-8331-9ef8676bfa1d\") " pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-r49qx" Jun 11 21:36:08.351174 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:08.351152 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6d54c7db9d-r49qx"] Jun 11 21:36:08.446963 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:08.446922 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rq9q\" (UniqueName: \"kubernetes.io/projected/9c3eb1b5-0528-4273-8331-9ef8676bfa1d-kube-api-access-8rq9q\") pod \"feast-operator-controller-manager-6d54c7db9d-r49qx\" (UID: \"9c3eb1b5-0528-4273-8331-9ef8676bfa1d\") " pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-r49qx" Jun 11 21:36:08.457790 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:08.457768 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rq9q\" (UniqueName: \"kubernetes.io/projected/9c3eb1b5-0528-4273-8331-9ef8676bfa1d-kube-api-access-8rq9q\") pod \"feast-operator-controller-manager-6d54c7db9d-r49qx\" (UID: \"9c3eb1b5-0528-4273-8331-9ef8676bfa1d\") " pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-r49qx" Jun 11 21:36:08.639653 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:08.639619 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-r49qx" Jun 11 21:36:08.765823 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:08.765781 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6d54c7db9d-r49qx"] Jun 11 21:36:08.768505 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:36:08.768471 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c3eb1b5_0528_4273_8331_9ef8676bfa1d.slice/crio-7326800cc18310e56ac4c70c8f732a4f559b72312c2d9c43bbef4f9d4cb5496c WatchSource:0}: Error finding container 7326800cc18310e56ac4c70c8f732a4f559b72312c2d9c43bbef4f9d4cb5496c: Status 404 returned error can't find the container with id 7326800cc18310e56ac4c70c8f732a4f559b72312c2d9c43bbef4f9d4cb5496c Jun 11 21:36:08.770461 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:08.770439 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 21:36:09.414930 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:09.414889 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-r49qx" event={"ID":"9c3eb1b5-0528-4273-8331-9ef8676bfa1d","Type":"ContainerStarted","Data":"7326800cc18310e56ac4c70c8f732a4f559b72312c2d9c43bbef4f9d4cb5496c"} Jun 11 21:36:12.426581 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:12.426535 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-r49qx" event={"ID":"9c3eb1b5-0528-4273-8331-9ef8676bfa1d","Type":"ContainerStarted","Data":"ad522e99855753e8a06cd6af69dea3c0330dbc78e45f5b3a36e90b1d790196c5"} Jun 11 21:36:12.427078 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:12.426616 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-r49qx" Jun 11 21:36:12.445389 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:12.445334 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-r49qx" podStartSLOduration=1.647946004 podStartE2EDuration="4.445317081s" totalImagesPullingTime="2.797371077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:36:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:36:12.444108718 +0000 UTC m=+1230.855432320" watchObservedRunningTime="2026-06-11 21:36:12.445317081 +0000 UTC m=+1230.856640619" Jun 11 21:36:23.432622 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:23.432593 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-r49qx" Jun 11 21:36:24.994256 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:24.994218 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8"] Jun 11 21:36:25.002037 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.002016 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:25.004558 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.004533 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Jun 11 21:36:25.004558 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.004537 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Jun 11 21:36:25.004776 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.004759 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Jun 11 21:36:25.004837 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.004819 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-z9vmp\"" Jun 11 21:36:25.004889 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.004840 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jun 11 21:36:25.004889 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.004865 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Jun 11 21:36:25.004889 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.004869 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jun 11 21:36:25.011925 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.011904 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8"] Jun 11 21:36:25.070764 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.070726 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-lpsw8\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:25.070925 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.070775 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-lpsw8\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:25.070925 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.070798 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-lpsw8\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:25.070925 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.070877 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-lpsw8\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:25.070925 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.070909 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-lpsw8\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:25.071061 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.070932 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8kz6\" (UniqueName: \"kubernetes.io/projected/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-kube-api-access-l8kz6\") pod \"feast-simple-feast-setup-75c64c888b-lpsw8\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:25.171828 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.171784 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-lpsw8\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:25.172000 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.171842 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-lpsw8\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:25.172000 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.171869 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-lpsw8\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:25.172000 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.171909 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-lpsw8\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:25.172000 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.171930 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-lpsw8\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:25.172000 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.171981 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8kz6\" (UniqueName: \"kubernetes.io/projected/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-kube-api-access-l8kz6\") pod \"feast-simple-feast-setup-75c64c888b-lpsw8\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:25.172239 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:36:25.172101 2570 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Jun 11 21:36:25.172239 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:36:25.172184 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-ui-tls podName:fe8e1bf9-2cff-4638-9862-9e88ff7323f0 nodeName:}" failed. No retries permitted until 2026-06-11 21:36:25.672161424 +0000 UTC m=+1244.083484949 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-ui-tls") pod "feast-simple-feast-setup-75c64c888b-lpsw8" (UID: "fe8e1bf9-2cff-4638-9862-9e88ff7323f0") : secret "feast-simple-feast-setup-ui-tls" not found Jun 11 21:36:25.172387 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.172365 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-lpsw8\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:25.174463 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.174439 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-lpsw8\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:25.174634 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.174616 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-lpsw8\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:25.174634 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.174627 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-lpsw8\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:25.181946 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.181923 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8kz6\" (UniqueName: \"kubernetes.io/projected/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-kube-api-access-l8kz6\") pod \"feast-simple-feast-setup-75c64c888b-lpsw8\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:25.676173 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.676137 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-lpsw8\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:25.678606 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.678574 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-lpsw8\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:25.912027 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:25.911995 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:26.076343 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:26.076320 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8"] Jun 11 21:36:26.077963 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:36:26.077939 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfe8e1bf9_2cff_4638_9862_9e88ff7323f0.slice/crio-9eb1cb3d20dd76731e1582a323cbc56ba26f43f505259828df715a840ad10992 WatchSource:0}: Error finding container 9eb1cb3d20dd76731e1582a323cbc56ba26f43f505259828df715a840ad10992: Status 404 returned error can't find the container with id 9eb1cb3d20dd76731e1582a323cbc56ba26f43f505259828df715a840ad10992 Jun 11 21:36:26.476610 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:26.476551 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" event={"ID":"fe8e1bf9-2cff-4638-9862-9e88ff7323f0","Type":"ContainerStarted","Data":"9eb1cb3d20dd76731e1582a323cbc56ba26f43f505259828df715a840ad10992"} Jun 11 21:36:41.548002 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:41.547959 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" event={"ID":"fe8e1bf9-2cff-4638-9862-9e88ff7323f0","Type":"ContainerStarted","Data":"73a00f40440a69c0968c0e493924aedd6b0c139e715b7daac1d439d76740a05e"} Jun 11 21:36:42.097039 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:42.097007 2570 scope.go:117] "RemoveContainer" containerID="7cb6ab090668b9d218bf7cc069d8e535b3d987993cd1c60b3f703fd6f37fdfb7" Jun 11 21:36:42.106118 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:42.106098 2570 scope.go:117] "RemoveContainer" containerID="0c927d50a0ff92b1b9e053b10a5f3febb4e6bd4045a146cc9b496afde800a978" Jun 11 21:36:46.568720 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:46.568683 2570 generic.go:356] "Generic (PLEG): container finished" podID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerID="73a00f40440a69c0968c0e493924aedd6b0c139e715b7daac1d439d76740a05e" exitCode=0 Jun 11 21:36:46.569148 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:46.568730 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" event={"ID":"fe8e1bf9-2cff-4638-9862-9e88ff7323f0","Type":"ContainerDied","Data":"73a00f40440a69c0968c0e493924aedd6b0c139e715b7daac1d439d76740a05e"} Jun 11 21:36:47.574506 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:47.574473 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" event={"ID":"fe8e1bf9-2cff-4638-9862-9e88ff7323f0","Type":"ContainerStarted","Data":"adaa69927a13cb96d86e3303e1d04b6c4df4510214f66f413dc582bc57987503"} Jun 11 21:36:52.596443 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:52.596405 2570 generic.go:356] "Generic (PLEG): container finished" podID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerID="adaa69927a13cb96d86e3303e1d04b6c4df4510214f66f413dc582bc57987503" exitCode=0 Jun 11 21:36:52.596846 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:52.596471 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" event={"ID":"fe8e1bf9-2cff-4638-9862-9e88ff7323f0","Type":"ContainerDied","Data":"adaa69927a13cb96d86e3303e1d04b6c4df4510214f66f413dc582bc57987503"} Jun 11 21:36:53.606940 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:53.606899 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" event={"ID":"fe8e1bf9-2cff-4638-9862-9e88ff7323f0","Type":"ContainerStarted","Data":"15de2821f78ae867858ee16117cc79785c5d0647fa04172ee68740048d9ee01a"} Jun 11 21:36:53.607405 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:53.606964 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" event={"ID":"fe8e1bf9-2cff-4638-9862-9e88ff7323f0","Type":"ContainerStarted","Data":"4286f2c141e500e206c20a1c02b7255757e9a4bb51cfe94f31c3068b79e62be1"} Jun 11 21:36:53.607405 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:53.606987 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" event={"ID":"fe8e1bf9-2cff-4638-9862-9e88ff7323f0","Type":"ContainerStarted","Data":"8089dc984e5bb60092fbedb570ee225b8ac352896d43f94b6ce5c9aacfc6eb21"} Jun 11 21:36:53.607405 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:53.607007 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" event={"ID":"fe8e1bf9-2cff-4638-9862-9e88ff7323f0","Type":"ContainerStarted","Data":"e023ac726f03795dc02b2a1ced9f4dfae491738257514d273669e5d82d12ea6d"} Jun 11 21:36:53.635909 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:53.635838 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" podStartSLOduration=14.4142957 podStartE2EDuration="29.635815116s" totalImagesPullingTime="15.221519416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:36:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:36:53.633710598 +0000 UTC m=+1272.045034162" watchObservedRunningTime="2026-06-11 21:36:53.635815116 +0000 UTC m=+1272.047138656" Jun 11 21:36:55.913237 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:55.913183 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:55.913750 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:55.913447 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:55.913750 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:55.913481 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:55.913750 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:55.913500 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:55.913750 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:55.913520 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:55.913750 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:55.913537 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:55.913750 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:55.913553 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:55.913750 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:55.913590 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:55.915659 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:55.915622 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.21:8443: connect: connection refused" Jun 11 21:36:55.915659 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:55.915636 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.21:6571: connect: connection refused" Jun 11 21:36:55.915836 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:55.915741 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": dial tcp 10.134.0.21:6567: connect: connection refused" Jun 11 21:36:55.915836 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:55.915753 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.21:8816: connect: connection refused" Jun 11 21:36:58.914225 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:58.914198 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:58.914714 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:58.914289 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:58.914714 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:58.914508 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:58.915024 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:58.915003 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:58.915146 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:58.915095 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:58.915354 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:58.915339 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:58.919013 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:58.918998 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:36:59.638182 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:36:59.638135 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:38:02.345234 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.345148 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8"] Jun 11 21:38:02.345765 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.345404 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="registry" containerID="cri-o://e023ac726f03795dc02b2a1ced9f4dfae491738257514d273669e5d82d12ea6d" gracePeriod=30 Jun 11 21:38:02.345765 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.345440 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="offline" containerID="cri-o://4286f2c141e500e206c20a1c02b7255757e9a4bb51cfe94f31c3068b79e62be1" gracePeriod=30 Jun 11 21:38:02.345765 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.345477 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="ui" containerID="cri-o://15de2821f78ae867858ee16117cc79785c5d0647fa04172ee68740048d9ee01a" gracePeriod=30 Jun 11 21:38:02.345765 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.345477 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="online" containerID="cri-o://8089dc984e5bb60092fbedb570ee225b8ac352896d43f94b6ce5c9aacfc6eb21" gracePeriod=30 Jun 11 21:38:02.732514 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.732428 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t"] Jun 11 21:38:02.736543 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.736524 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:02.739196 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.739165 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-tgk7m\"" Jun 11 21:38:02.746969 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.746933 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t"] Jun 11 21:38:02.866219 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.866189 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-h8m2t\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:02.866219 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.866222 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b624a0e1-25bf-41c5-bb23-a79cd9574b31-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-h8m2t\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:02.866441 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.866245 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-h8m2t\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:02.866441 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.866329 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-h8m2t\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:02.866441 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.866405 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-h8m2t\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:02.866650 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.866438 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sw4t\" (UniqueName: \"kubernetes.io/projected/b624a0e1-25bf-41c5-bb23-a79cd9574b31-kube-api-access-7sw4t\") pod \"feast-simple-feast-setup-75c64c888b-h8m2t\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:02.867598 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.867551 2570 generic.go:356] "Generic (PLEG): container finished" podID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerID="15de2821f78ae867858ee16117cc79785c5d0647fa04172ee68740048d9ee01a" exitCode=0 Jun 11 21:38:02.867724 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.867596 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" event={"ID":"fe8e1bf9-2cff-4638-9862-9e88ff7323f0","Type":"ContainerDied","Data":"15de2821f78ae867858ee16117cc79785c5d0647fa04172ee68740048d9ee01a"} Jun 11 21:38:02.967775 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.967737 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-h8m2t\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:02.967937 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.967794 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-h8m2t\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:02.967937 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.967826 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sw4t\" (UniqueName: \"kubernetes.io/projected/b624a0e1-25bf-41c5-bb23-a79cd9574b31-kube-api-access-7sw4t\") pod \"feast-simple-feast-setup-75c64c888b-h8m2t\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:02.967937 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.967890 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-h8m2t\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:02.968052 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:38:02.967966 2570 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Jun 11 21:38:02.968052 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.968014 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b624a0e1-25bf-41c5-bb23-a79cd9574b31-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-h8m2t\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:02.968052 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:38:02.968028 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-ui-tls podName:b624a0e1-25bf-41c5-bb23-a79cd9574b31 nodeName:}" failed. No retries permitted until 2026-06-11 21:38:03.468006052 +0000 UTC m=+1341.879329569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-ui-tls") pod "feast-simple-feast-setup-75c64c888b-h8m2t" (UID: "b624a0e1-25bf-41c5-bb23-a79cd9574b31") : secret "feast-simple-feast-setup-ui-tls" not found Jun 11 21:38:02.968210 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.968074 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-h8m2t\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:02.968386 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.968364 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b624a0e1-25bf-41c5-bb23-a79cd9574b31-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-h8m2t\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:02.970485 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.970458 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-h8m2t\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:02.970699 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.970683 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-h8m2t\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:02.970852 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.970833 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-h8m2t\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:02.977112 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:02.977087 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sw4t\" (UniqueName: \"kubernetes.io/projected/b624a0e1-25bf-41c5-bb23-a79cd9574b31-kube-api-access-7sw4t\") pod \"feast-simple-feast-setup-75c64c888b-h8m2t\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:03.473515 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:03.473472 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-h8m2t\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:03.475832 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:03.475808 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-h8m2t\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:03.647210 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:03.647176 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:03.778318 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:03.778293 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t"] Jun 11 21:38:03.779728 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:38:03.779701 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb624a0e1_25bf_41c5_bb23_a79cd9574b31.slice/crio-da81774d0065c89e2f55649fc99cf5a801936c28d7944e07731ff007ba0cc006 WatchSource:0}: Error finding container da81774d0065c89e2f55649fc99cf5a801936c28d7944e07731ff007ba0cc006: Status 404 returned error can't find the container with id da81774d0065c89e2f55649fc99cf5a801936c28d7944e07731ff007ba0cc006 Jun 11 21:38:03.875832 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:03.875798 2570 generic.go:356] "Generic (PLEG): container finished" podID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerID="8089dc984e5bb60092fbedb570ee225b8ac352896d43f94b6ce5c9aacfc6eb21" exitCode=0 Jun 11 21:38:03.876014 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:03.875856 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" event={"ID":"fe8e1bf9-2cff-4638-9862-9e88ff7323f0","Type":"ContainerDied","Data":"8089dc984e5bb60092fbedb570ee225b8ac352896d43f94b6ce5c9aacfc6eb21"} Jun 11 21:38:03.877425 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:03.877396 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" event={"ID":"b624a0e1-25bf-41c5-bb23-a79cd9574b31","Type":"ContainerStarted","Data":"c245182bdeeae27582178230c1092cef218f3f00df08aba9f6bdeed3c2d209fc"} Jun 11 21:38:03.877586 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:03.877437 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" event={"ID":"b624a0e1-25bf-41c5-bb23-a79cd9574b31","Type":"ContainerStarted","Data":"da81774d0065c89e2f55649fc99cf5a801936c28d7944e07731ff007ba0cc006"} Jun 11 21:38:08.900440 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:08.900404 2570 generic.go:356] "Generic (PLEG): container finished" podID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerID="c245182bdeeae27582178230c1092cef218f3f00df08aba9f6bdeed3c2d209fc" exitCode=0 Jun 11 21:38:08.900887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:08.900475 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" event={"ID":"b624a0e1-25bf-41c5-bb23-a79cd9574b31","Type":"ContainerDied","Data":"c245182bdeeae27582178230c1092cef218f3f00df08aba9f6bdeed3c2d209fc"} Jun 11 21:38:08.918199 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:08.918139 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.21:8443: connect: connection refused" Jun 11 21:38:09.635768 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:09.635725 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": dial tcp 10.134.0.21:6567: connect: connection refused" Jun 11 21:38:09.907002 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:09.906903 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" event={"ID":"b624a0e1-25bf-41c5-bb23-a79cd9574b31","Type":"ContainerStarted","Data":"8dc31f21738728d0236923217e65471fcb694f4a79ac30a9b4a0d95b361f5bcb"} Jun 11 21:38:14.927315 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:14.927279 2570 generic.go:356] "Generic (PLEG): container finished" podID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerID="8dc31f21738728d0236923217e65471fcb694f4a79ac30a9b4a0d95b361f5bcb" exitCode=0 Jun 11 21:38:14.927708 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:14.927352 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" event={"ID":"b624a0e1-25bf-41c5-bb23-a79cd9574b31","Type":"ContainerDied","Data":"8dc31f21738728d0236923217e65471fcb694f4a79ac30a9b4a0d95b361f5bcb"} Jun 11 21:38:15.938172 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:15.938131 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" event={"ID":"b624a0e1-25bf-41c5-bb23-a79cd9574b31","Type":"ContainerStarted","Data":"82f116f2571752dca6a43b3b610044f3315858ce6e2e4264b7e4358fc3c8fded"} Jun 11 21:38:15.938671 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:15.938196 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" event={"ID":"b624a0e1-25bf-41c5-bb23-a79cd9574b31","Type":"ContainerStarted","Data":"22af7d0ea9ad41e98d006ddcdef283eaf849939b6633749ee19e2ac3e4281407"} Jun 11 21:38:15.938671 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:15.938219 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" event={"ID":"b624a0e1-25bf-41c5-bb23-a79cd9574b31","Type":"ContainerStarted","Data":"5bb1f52e35b9a285a6a3d78740529d7e0d15825f0707a02bdf8fd0f0b7617bbd"} Jun 11 21:38:15.938671 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:15.938238 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" event={"ID":"b624a0e1-25bf-41c5-bb23-a79cd9574b31","Type":"ContainerStarted","Data":"5876c4f4b0ef9e6da53f6afd1ac37de99544677cb43c1bb32e39c7dd12a9e3f3"} Jun 11 21:38:15.972772 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:15.972706 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" podStartSLOduration=13.972685069 podStartE2EDuration="13.972685069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:38:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:38:15.970684647 +0000 UTC m=+1354.382008188" watchObservedRunningTime="2026-06-11 21:38:15.972685069 +0000 UTC m=+1354.384008607" Jun 11 21:38:18.648101 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:18.648036 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:18.648101 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:18.648101 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:18.648669 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:18.648121 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:18.648669 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:18.648139 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:18.650030 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:18.649994 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.22:8816: connect: connection refused" Jun 11 21:38:18.650257 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:18.650016 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.22:6571: connect: connection refused" Jun 11 21:38:18.650340 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:18.650033 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused" Jun 11 21:38:18.650340 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:18.650054 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.22:8443: connect: connection refused" Jun 11 21:38:18.916131 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:18.916029 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.21:8443: connect: connection refused" Jun 11 21:38:19.635780 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:19.635731 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": dial tcp 10.134.0.21:6567: connect: connection refused" Jun 11 21:38:21.648919 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:21.648886 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:21.649324 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:21.649053 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:21.649375 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:21.649347 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:21.649469 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:21.649448 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:21.649531 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:21.649495 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:21.649531 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:21.649516 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:21.649853 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:21.649837 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:21.649933 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:21.649920 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:21.650707 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:21.650687 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:21.653659 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:21.653625 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:21.972989 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:21.972910 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:21.976061 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:21.976041 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:38:28.915541 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:28.915494 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.21:8443: connect: connection refused" Jun 11 21:38:28.916056 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:28.915709 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:38:29.635783 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:29.635736 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="online" probeResult="failure" output="Get \"https://10.134.0.21:6567/health\": dial tcp 10.134.0.21:6567: connect: connection refused" Jun 11 21:38:29.635992 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:29.635872 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:38:33.000990 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.000962 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:38:33.017208 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.017179 2570 generic.go:356] "Generic (PLEG): container finished" podID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerID="4286f2c141e500e206c20a1c02b7255757e9a4bb51cfe94f31c3068b79e62be1" exitCode=137 Jun 11 21:38:33.017208 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.017202 2570 generic.go:356] "Generic (PLEG): container finished" podID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerID="e023ac726f03795dc02b2a1ced9f4dfae491738257514d273669e5d82d12ea6d" exitCode=137 Jun 11 21:38:33.017389 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.017236 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" Jun 11 21:38:33.017389 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.017262 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" event={"ID":"fe8e1bf9-2cff-4638-9862-9e88ff7323f0","Type":"ContainerDied","Data":"4286f2c141e500e206c20a1c02b7255757e9a4bb51cfe94f31c3068b79e62be1"} Jun 11 21:38:33.017389 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.017315 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" event={"ID":"fe8e1bf9-2cff-4638-9862-9e88ff7323f0","Type":"ContainerDied","Data":"e023ac726f03795dc02b2a1ced9f4dfae491738257514d273669e5d82d12ea6d"} Jun 11 21:38:33.017389 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.017330 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8" event={"ID":"fe8e1bf9-2cff-4638-9862-9e88ff7323f0","Type":"ContainerDied","Data":"9eb1cb3d20dd76731e1582a323cbc56ba26f43f505259828df715a840ad10992"} Jun 11 21:38:33.017389 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.017348 2570 scope.go:117] "RemoveContainer" containerID="15de2821f78ae867858ee16117cc79785c5d0647fa04172ee68740048d9ee01a" Jun 11 21:38:33.026491 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.026466 2570 scope.go:117] "RemoveContainer" containerID="4286f2c141e500e206c20a1c02b7255757e9a4bb51cfe94f31c3068b79e62be1" Jun 11 21:38:33.035518 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.035498 2570 scope.go:117] "RemoveContainer" containerID="8089dc984e5bb60092fbedb570ee225b8ac352896d43f94b6ce5c9aacfc6eb21" Jun 11 21:38:33.043292 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.043272 2570 scope.go:117] "RemoveContainer" containerID="e023ac726f03795dc02b2a1ced9f4dfae491738257514d273669e5d82d12ea6d" Jun 11 21:38:33.050831 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.050813 2570 scope.go:117] "RemoveContainer" containerID="adaa69927a13cb96d86e3303e1d04b6c4df4510214f66f413dc582bc57987503" Jun 11 21:38:33.058157 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.058136 2570 scope.go:117] "RemoveContainer" containerID="73a00f40440a69c0968c0e493924aedd6b0c139e715b7daac1d439d76740a05e" Jun 11 21:38:33.065237 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.065215 2570 scope.go:117] "RemoveContainer" containerID="15de2821f78ae867858ee16117cc79785c5d0647fa04172ee68740048d9ee01a" Jun 11 21:38:33.065488 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:38:33.065463 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15de2821f78ae867858ee16117cc79785c5d0647fa04172ee68740048d9ee01a\": container with ID starting with 15de2821f78ae867858ee16117cc79785c5d0647fa04172ee68740048d9ee01a not found: ID does not exist" containerID="15de2821f78ae867858ee16117cc79785c5d0647fa04172ee68740048d9ee01a" Jun 11 21:38:33.065531 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.065497 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15de2821f78ae867858ee16117cc79785c5d0647fa04172ee68740048d9ee01a"} err="failed to get container status \"15de2821f78ae867858ee16117cc79785c5d0647fa04172ee68740048d9ee01a\": rpc error: code = NotFound desc = could not find container \"15de2821f78ae867858ee16117cc79785c5d0647fa04172ee68740048d9ee01a\": container with ID starting with 15de2821f78ae867858ee16117cc79785c5d0647fa04172ee68740048d9ee01a not found: ID does not exist" Jun 11 21:38:33.065531 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.065515 2570 scope.go:117] "RemoveContainer" containerID="4286f2c141e500e206c20a1c02b7255757e9a4bb51cfe94f31c3068b79e62be1" Jun 11 21:38:33.065794 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:38:33.065776 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4286f2c141e500e206c20a1c02b7255757e9a4bb51cfe94f31c3068b79e62be1\": container with ID starting with 4286f2c141e500e206c20a1c02b7255757e9a4bb51cfe94f31c3068b79e62be1 not found: ID does not exist" containerID="4286f2c141e500e206c20a1c02b7255757e9a4bb51cfe94f31c3068b79e62be1" Jun 11 21:38:33.065838 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.065804 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4286f2c141e500e206c20a1c02b7255757e9a4bb51cfe94f31c3068b79e62be1"} err="failed to get container status \"4286f2c141e500e206c20a1c02b7255757e9a4bb51cfe94f31c3068b79e62be1\": rpc error: code = NotFound desc = could not find container \"4286f2c141e500e206c20a1c02b7255757e9a4bb51cfe94f31c3068b79e62be1\": container with ID starting with 4286f2c141e500e206c20a1c02b7255757e9a4bb51cfe94f31c3068b79e62be1 not found: ID does not exist" Jun 11 21:38:33.065918 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.065832 2570 scope.go:117] "RemoveContainer" containerID="8089dc984e5bb60092fbedb570ee225b8ac352896d43f94b6ce5c9aacfc6eb21" Jun 11 21:38:33.066096 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:38:33.066080 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8089dc984e5bb60092fbedb570ee225b8ac352896d43f94b6ce5c9aacfc6eb21\": container with ID starting with 8089dc984e5bb60092fbedb570ee225b8ac352896d43f94b6ce5c9aacfc6eb21 not found: ID does not exist" containerID="8089dc984e5bb60092fbedb570ee225b8ac352896d43f94b6ce5c9aacfc6eb21" Jun 11 21:38:33.066136 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.066101 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8089dc984e5bb60092fbedb570ee225b8ac352896d43f94b6ce5c9aacfc6eb21"} err="failed to get container status \"8089dc984e5bb60092fbedb570ee225b8ac352896d43f94b6ce5c9aacfc6eb21\": rpc error: code = NotFound desc = could not find container \"8089dc984e5bb60092fbedb570ee225b8ac352896d43f94b6ce5c9aacfc6eb21\": container with ID starting with 8089dc984e5bb60092fbedb570ee225b8ac352896d43f94b6ce5c9aacfc6eb21 not found: ID does not exist" Jun 11 21:38:33.066136 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.066115 2570 scope.go:117] "RemoveContainer" containerID="e023ac726f03795dc02b2a1ced9f4dfae491738257514d273669e5d82d12ea6d" Jun 11 21:38:33.066346 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:38:33.066330 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e023ac726f03795dc02b2a1ced9f4dfae491738257514d273669e5d82d12ea6d\": container with ID starting with e023ac726f03795dc02b2a1ced9f4dfae491738257514d273669e5d82d12ea6d not found: ID does not exist" containerID="e023ac726f03795dc02b2a1ced9f4dfae491738257514d273669e5d82d12ea6d" Jun 11 21:38:33.066385 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.066353 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e023ac726f03795dc02b2a1ced9f4dfae491738257514d273669e5d82d12ea6d"} err="failed to get container status \"e023ac726f03795dc02b2a1ced9f4dfae491738257514d273669e5d82d12ea6d\": rpc error: code = NotFound desc = could not find container \"e023ac726f03795dc02b2a1ced9f4dfae491738257514d273669e5d82d12ea6d\": container with ID starting with e023ac726f03795dc02b2a1ced9f4dfae491738257514d273669e5d82d12ea6d not found: ID does not exist" Jun 11 21:38:33.066385 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.066369 2570 scope.go:117] "RemoveContainer" containerID="adaa69927a13cb96d86e3303e1d04b6c4df4510214f66f413dc582bc57987503" Jun 11 21:38:33.066641 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:38:33.066611 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adaa69927a13cb96d86e3303e1d04b6c4df4510214f66f413dc582bc57987503\": container with ID starting with adaa69927a13cb96d86e3303e1d04b6c4df4510214f66f413dc582bc57987503 not found: ID does not exist" containerID="adaa69927a13cb96d86e3303e1d04b6c4df4510214f66f413dc582bc57987503" Jun 11 21:38:33.066707 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.066642 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adaa69927a13cb96d86e3303e1d04b6c4df4510214f66f413dc582bc57987503"} err="failed to get container status \"adaa69927a13cb96d86e3303e1d04b6c4df4510214f66f413dc582bc57987503\": rpc error: code = NotFound desc = could not find container \"adaa69927a13cb96d86e3303e1d04b6c4df4510214f66f413dc582bc57987503\": container with ID starting with adaa69927a13cb96d86e3303e1d04b6c4df4510214f66f413dc582bc57987503 not found: ID does not exist" Jun 11 21:38:33.066707 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.066673 2570 scope.go:117] "RemoveContainer" containerID="73a00f40440a69c0968c0e493924aedd6b0c139e715b7daac1d439d76740a05e" Jun 11 21:38:33.066928 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:38:33.066913 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73a00f40440a69c0968c0e493924aedd6b0c139e715b7daac1d439d76740a05e\": container with ID starting with 73a00f40440a69c0968c0e493924aedd6b0c139e715b7daac1d439d76740a05e not found: ID does not exist" containerID="73a00f40440a69c0968c0e493924aedd6b0c139e715b7daac1d439d76740a05e" Jun 11 21:38:33.066971 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.066933 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73a00f40440a69c0968c0e493924aedd6b0c139e715b7daac1d439d76740a05e"} err="failed to get container status \"73a00f40440a69c0968c0e493924aedd6b0c139e715b7daac1d439d76740a05e\": rpc error: code = NotFound desc = could not find container \"73a00f40440a69c0968c0e493924aedd6b0c139e715b7daac1d439d76740a05e\": container with ID starting with 73a00f40440a69c0968c0e493924aedd6b0c139e715b7daac1d439d76740a05e not found: ID does not exist" Jun 11 21:38:33.066971 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.066946 2570 scope.go:117] "RemoveContainer" containerID="15de2821f78ae867858ee16117cc79785c5d0647fa04172ee68740048d9ee01a" Jun 11 21:38:33.067182 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.067163 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15de2821f78ae867858ee16117cc79785c5d0647fa04172ee68740048d9ee01a"} err="failed to get container status \"15de2821f78ae867858ee16117cc79785c5d0647fa04172ee68740048d9ee01a\": rpc error: code = NotFound desc = could not find container \"15de2821f78ae867858ee16117cc79785c5d0647fa04172ee68740048d9ee01a\": container with ID starting with 15de2821f78ae867858ee16117cc79785c5d0647fa04172ee68740048d9ee01a not found: ID does not exist" Jun 11 21:38:33.067182 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.067181 2570 scope.go:117] "RemoveContainer" containerID="4286f2c141e500e206c20a1c02b7255757e9a4bb51cfe94f31c3068b79e62be1" Jun 11 21:38:33.067375 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.067359 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4286f2c141e500e206c20a1c02b7255757e9a4bb51cfe94f31c3068b79e62be1"} err="failed to get container status \"4286f2c141e500e206c20a1c02b7255757e9a4bb51cfe94f31c3068b79e62be1\": rpc error: code = NotFound desc = could not find container \"4286f2c141e500e206c20a1c02b7255757e9a4bb51cfe94f31c3068b79e62be1\": container with ID starting with 4286f2c141e500e206c20a1c02b7255757e9a4bb51cfe94f31c3068b79e62be1 not found: ID does not exist" Jun 11 21:38:33.067420 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.067375 2570 scope.go:117] "RemoveContainer" containerID="8089dc984e5bb60092fbedb570ee225b8ac352896d43f94b6ce5c9aacfc6eb21" Jun 11 21:38:33.067602 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.067584 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8089dc984e5bb60092fbedb570ee225b8ac352896d43f94b6ce5c9aacfc6eb21"} err="failed to get container status \"8089dc984e5bb60092fbedb570ee225b8ac352896d43f94b6ce5c9aacfc6eb21\": rpc error: code = NotFound desc = could not find container \"8089dc984e5bb60092fbedb570ee225b8ac352896d43f94b6ce5c9aacfc6eb21\": container with ID starting with 8089dc984e5bb60092fbedb570ee225b8ac352896d43f94b6ce5c9aacfc6eb21 not found: ID does not exist" Jun 11 21:38:33.067655 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.067603 2570 scope.go:117] "RemoveContainer" containerID="e023ac726f03795dc02b2a1ced9f4dfae491738257514d273669e5d82d12ea6d" Jun 11 21:38:33.067796 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.067780 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e023ac726f03795dc02b2a1ced9f4dfae491738257514d273669e5d82d12ea6d"} err="failed to get container status \"e023ac726f03795dc02b2a1ced9f4dfae491738257514d273669e5d82d12ea6d\": rpc error: code = NotFound desc = could not find container \"e023ac726f03795dc02b2a1ced9f4dfae491738257514d273669e5d82d12ea6d\": container with ID starting with e023ac726f03795dc02b2a1ced9f4dfae491738257514d273669e5d82d12ea6d not found: ID does not exist" Jun 11 21:38:33.067796 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.067795 2570 scope.go:117] "RemoveContainer" containerID="adaa69927a13cb96d86e3303e1d04b6c4df4510214f66f413dc582bc57987503" Jun 11 21:38:33.067975 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.067959 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adaa69927a13cb96d86e3303e1d04b6c4df4510214f66f413dc582bc57987503"} err="failed to get container status \"adaa69927a13cb96d86e3303e1d04b6c4df4510214f66f413dc582bc57987503\": rpc error: code = NotFound desc = could not find container \"adaa69927a13cb96d86e3303e1d04b6c4df4510214f66f413dc582bc57987503\": container with ID starting with adaa69927a13cb96d86e3303e1d04b6c4df4510214f66f413dc582bc57987503 not found: ID does not exist" Jun 11 21:38:33.068017 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.067975 2570 scope.go:117] "RemoveContainer" containerID="73a00f40440a69c0968c0e493924aedd6b0c139e715b7daac1d439d76740a05e" Jun 11 21:38:33.068144 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.068128 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73a00f40440a69c0968c0e493924aedd6b0c139e715b7daac1d439d76740a05e"} err="failed to get container status \"73a00f40440a69c0968c0e493924aedd6b0c139e715b7daac1d439d76740a05e\": rpc error: code = NotFound desc = could not find container \"73a00f40440a69c0968c0e493924aedd6b0c139e715b7daac1d439d76740a05e\": container with ID starting with 73a00f40440a69c0968c0e493924aedd6b0c139e715b7daac1d439d76740a05e not found: ID does not exist" Jun 11 21:38:33.138205 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.138120 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-ui-tls\") pod \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " Jun 11 21:38:33.138205 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.138160 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-feast-data\") pod \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " Jun 11 21:38:33.138205 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.138184 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8kz6\" (UniqueName: \"kubernetes.io/projected/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-kube-api-access-l8kz6\") pod \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " Jun 11 21:38:33.138205 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.138205 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-online-tls\") pod \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " Jun 11 21:38:33.138683 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.138232 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-offline-tls\") pod \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " Jun 11 21:38:33.138683 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.138262 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-registry-tls\") pod \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\" (UID: \"fe8e1bf9-2cff-4638-9862-9e88ff7323f0\") " Jun 11 21:38:33.138796 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.138767 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-feast-data" (OuterVolumeSpecName: "feast-data") pod "fe8e1bf9-2cff-4638-9862-9e88ff7323f0" (UID: "fe8e1bf9-2cff-4638-9862-9e88ff7323f0"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 21:38:33.140657 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.140629 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-kube-api-access-l8kz6" (OuterVolumeSpecName: "kube-api-access-l8kz6") pod "fe8e1bf9-2cff-4638-9862-9e88ff7323f0" (UID: "fe8e1bf9-2cff-4638-9862-9e88ff7323f0"). InnerVolumeSpecName "kube-api-access-l8kz6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 21:38:33.140789 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.140704 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "fe8e1bf9-2cff-4638-9862-9e88ff7323f0" (UID: "fe8e1bf9-2cff-4638-9862-9e88ff7323f0"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:38:33.140789 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.140729 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-online-tls" (OuterVolumeSpecName: "online-tls") pod "fe8e1bf9-2cff-4638-9862-9e88ff7323f0" (UID: "fe8e1bf9-2cff-4638-9862-9e88ff7323f0"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:38:33.140865 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.140807 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "fe8e1bf9-2cff-4638-9862-9e88ff7323f0" (UID: "fe8e1bf9-2cff-4638-9862-9e88ff7323f0"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:38:33.140996 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.140972 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "fe8e1bf9-2cff-4638-9862-9e88ff7323f0" (UID: "fe8e1bf9-2cff-4638-9862-9e88ff7323f0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:38:33.239556 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.239520 2570 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-ui-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:38:33.239556 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.239552 2570 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-feast-data\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:38:33.239783 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.239593 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8kz6\" (UniqueName: \"kubernetes.io/projected/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-kube-api-access-l8kz6\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:38:33.239783 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.239606 2570 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-online-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:38:33.239783 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.239619 2570 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-offline-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:38:33.239783 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.239629 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fe8e1bf9-2cff-4638-9862-9e88ff7323f0-registry-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:38:33.339172 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.339144 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8"] Jun 11 21:38:33.344492 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:33.344465 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-lpsw8"] Jun 11 21:38:34.050852 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:38:34.050817 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" path="/var/lib/kubelet/pods/fe8e1bf9-2cff-4638-9862-9e88ff7323f0/volumes" Jun 11 21:39:24.461678 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.461646 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s"] Jun 11 21:39:24.462139 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.461912 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="registry" Jun 11 21:39:24.462139 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.461922 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="registry" Jun 11 21:39:24.462139 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.461931 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="ui" Jun 11 21:39:24.462139 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.461938 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="ui" Jun 11 21:39:24.462139 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.461948 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="feast-init" Jun 11 21:39:24.462139 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.461955 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="feast-init" Jun 11 21:39:24.462139 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.461965 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="offline" Jun 11 21:39:24.462139 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.461970 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="offline" Jun 11 21:39:24.462139 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.461977 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="feast-apply" Jun 11 21:39:24.462139 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.461982 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="feast-apply" Jun 11 21:39:24.462139 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.461989 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="online" Jun 11 21:39:24.462139 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.461994 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="online" Jun 11 21:39:24.462139 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.462051 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="ui" Jun 11 21:39:24.462139 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.462060 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="online" Jun 11 21:39:24.462139 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.462067 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="offline" Jun 11 21:39:24.462139 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.462072 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe8e1bf9-2cff-4638-9862-9e88ff7323f0" containerName="registry" Jun 11 21:39:24.465380 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.465361 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:24.467879 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.467856 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Jun 11 21:39:24.467879 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.467866 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-tz4wf\"" Jun 11 21:39:24.467879 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.467891 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Jun 11 21:39:24.468204 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.468188 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Jun 11 21:39:24.468302 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.468285 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Jun 11 21:39:24.468811 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.468798 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Jun 11 21:39:24.468889 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.468802 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Jun 11 21:39:24.476851 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.476830 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s"] Jun 11 21:39:24.522887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.522853 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn5n2\" (UniqueName: \"kubernetes.io/projected/d8af6452-beb7-4123-bf5a-e352416910d4-kube-api-access-vn5n2\") pod \"feast-simple-feast-remote-setup-669bcf45fc-jq97s\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:24.523059 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.522929 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d8af6452-beb7-4123-bf5a-e352416910d4-feast-data\") pod \"feast-simple-feast-remote-setup-669bcf45fc-jq97s\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:24.523059 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.522959 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d8af6452-beb7-4123-bf5a-e352416910d4-registry-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-jq97s\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:24.523059 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.522983 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d8af6452-beb7-4123-bf5a-e352416910d4-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-jq97s\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:24.523059 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.523045 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d8af6452-beb7-4123-bf5a-e352416910d4-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-jq97s\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:24.523227 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.523069 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d8af6452-beb7-4123-bf5a-e352416910d4-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-jq97s\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:24.624239 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.624204 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn5n2\" (UniqueName: \"kubernetes.io/projected/d8af6452-beb7-4123-bf5a-e352416910d4-kube-api-access-vn5n2\") pod \"feast-simple-feast-remote-setup-669bcf45fc-jq97s\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:24.624444 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.624263 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d8af6452-beb7-4123-bf5a-e352416910d4-feast-data\") pod \"feast-simple-feast-remote-setup-669bcf45fc-jq97s\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:24.624444 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.624375 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d8af6452-beb7-4123-bf5a-e352416910d4-registry-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-jq97s\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:24.624444 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.624409 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d8af6452-beb7-4123-bf5a-e352416910d4-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-jq97s\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:24.624444 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.624438 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d8af6452-beb7-4123-bf5a-e352416910d4-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-jq97s\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:24.624677 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.624461 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d8af6452-beb7-4123-bf5a-e352416910d4-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-jq97s\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:24.624677 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:39:24.624611 2570 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Jun 11 21:39:24.624677 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.624653 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d8af6452-beb7-4123-bf5a-e352416910d4-feast-data\") pod \"feast-simple-feast-remote-setup-669bcf45fc-jq97s\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:24.624792 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:39:24.624686 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8af6452-beb7-4123-bf5a-e352416910d4-ui-tls podName:d8af6452-beb7-4123-bf5a-e352416910d4 nodeName:}" failed. No retries permitted until 2026-06-11 21:39:25.124667242 +0000 UTC m=+1423.535990764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/d8af6452-beb7-4123-bf5a-e352416910d4-ui-tls") pod "feast-simple-feast-remote-setup-669bcf45fc-jq97s" (UID: "d8af6452-beb7-4123-bf5a-e352416910d4") : secret "feast-simple-feast-remote-setup-ui-tls" not found Jun 11 21:39:24.625043 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.625016 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d8af6452-beb7-4123-bf5a-e352416910d4-registry-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-jq97s\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:24.626942 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.626805 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d8af6452-beb7-4123-bf5a-e352416910d4-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-jq97s\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:24.627143 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.627119 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d8af6452-beb7-4123-bf5a-e352416910d4-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-jq97s\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:24.633160 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:24.633023 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn5n2\" (UniqueName: \"kubernetes.io/projected/d8af6452-beb7-4123-bf5a-e352416910d4-kube-api-access-vn5n2\") pod \"feast-simple-feast-remote-setup-669bcf45fc-jq97s\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:25.129495 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:25.129456 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d8af6452-beb7-4123-bf5a-e352416910d4-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-jq97s\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:25.131703 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:25.131678 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d8af6452-beb7-4123-bf5a-e352416910d4-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-jq97s\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:25.374734 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:25.374696 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:25.508690 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:25.508662 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s"] Jun 11 21:39:25.510227 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:39:25.510203 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd8af6452_beb7_4123_bf5a_e352416910d4.slice/crio-80c06027306a7a62c38b7567a8880f2e2a9ea1b2242a8ca2cdd59044d3b2a238 WatchSource:0}: Error finding container 80c06027306a7a62c38b7567a8880f2e2a9ea1b2242a8ca2cdd59044d3b2a238: Status 404 returned error can't find the container with id 80c06027306a7a62c38b7567a8880f2e2a9ea1b2242a8ca2cdd59044d3b2a238 Jun 11 21:39:26.220390 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:26.220358 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" event={"ID":"d8af6452-beb7-4123-bf5a-e352416910d4","Type":"ContainerStarted","Data":"5436fe339c58072754cf5e4efd0531eea93a544547a949632f5e451252db9376"} Jun 11 21:39:26.220390 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:26.220397 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" event={"ID":"d8af6452-beb7-4123-bf5a-e352416910d4","Type":"ContainerStarted","Data":"80c06027306a7a62c38b7567a8880f2e2a9ea1b2242a8ca2cdd59044d3b2a238"} Jun 11 21:39:30.239517 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:30.239486 2570 generic.go:356] "Generic (PLEG): container finished" podID="d8af6452-beb7-4123-bf5a-e352416910d4" containerID="5436fe339c58072754cf5e4efd0531eea93a544547a949632f5e451252db9376" exitCode=0 Jun 11 21:39:30.239926 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:30.239552 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" event={"ID":"d8af6452-beb7-4123-bf5a-e352416910d4","Type":"ContainerDied","Data":"5436fe339c58072754cf5e4efd0531eea93a544547a949632f5e451252db9376"} Jun 11 21:39:31.245658 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:31.245622 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" event={"ID":"d8af6452-beb7-4123-bf5a-e352416910d4","Type":"ContainerStarted","Data":"9a0cf98aaf8a32afe5b6e40656c3204492f4ec383bd6b7bfde92c54aa9109c28"} Jun 11 21:39:36.268079 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:36.267996 2570 generic.go:356] "Generic (PLEG): container finished" podID="d8af6452-beb7-4123-bf5a-e352416910d4" containerID="9a0cf98aaf8a32afe5b6e40656c3204492f4ec383bd6b7bfde92c54aa9109c28" exitCode=0 Jun 11 21:39:36.268079 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:36.268060 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" event={"ID":"d8af6452-beb7-4123-bf5a-e352416910d4","Type":"ContainerDied","Data":"9a0cf98aaf8a32afe5b6e40656c3204492f4ec383bd6b7bfde92c54aa9109c28"} Jun 11 21:39:37.276826 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:37.276785 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" event={"ID":"d8af6452-beb7-4123-bf5a-e352416910d4","Type":"ContainerStarted","Data":"67c45a7f61310f2003bd9cab2e189af623fbdfff1346180384dcf0aa179df68f"} Jun 11 21:39:37.277198 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:37.276850 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" event={"ID":"d8af6452-beb7-4123-bf5a-e352416910d4","Type":"ContainerStarted","Data":"3e651e4600335aebdafd8a39220d38c7fe0ec5c62df360a1f15ff35c694c7076"} Jun 11 21:39:37.277198 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:37.276871 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" event={"ID":"d8af6452-beb7-4123-bf5a-e352416910d4","Type":"ContainerStarted","Data":"ae52b970902e7f0a9ae4a8c6d9e829d54f5bde58b0d0533902eee48c8d5af26a"} Jun 11 21:39:37.300254 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:37.300195 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" podStartSLOduration=13.300174687 podStartE2EDuration="13.300174687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:39:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:39:37.297674446 +0000 UTC m=+1435.708997985" watchObservedRunningTime="2026-06-11 21:39:37.300174687 +0000 UTC m=+1435.711498227" Jun 11 21:39:37.375802 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:37.375743 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:37.376001 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:37.375831 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:37.376001 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:37.375854 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:37.377929 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:37.377891 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.23:8816: connect: connection refused" Jun 11 21:39:37.378058 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:37.377933 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused" Jun 11 21:39:37.378058 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:37.377947 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Jun 11 21:39:40.376041 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:40.375996 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused" Jun 11 21:39:40.376579 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:40.375996 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.23:8816: connect: connection refused" Jun 11 21:39:40.376579 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:40.376000 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Jun 11 21:39:43.376255 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:43.376220 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:43.376723 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:43.376384 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:43.376837 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:43.376813 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:43.376903 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:43.376857 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:43.377020 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:43.377006 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:43.377076 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:43.377060 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:43.380330 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:43.380309 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:44.310862 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:44.310833 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:39:44.313829 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:39:44.313805 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:40:42.101180 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:40:42.101148 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/ovn-acl-logging/0.log" Jun 11 21:40:42.104703 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:40:42.104682 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/ovn-acl-logging/0.log" Jun 11 21:40:45.703785 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:40:45.703751 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s"] Jun 11 21:40:45.704221 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:40:45.704032 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="online" containerID="cri-o://ae52b970902e7f0a9ae4a8c6d9e829d54f5bde58b0d0533902eee48c8d5af26a" gracePeriod=30 Jun 11 21:40:45.704221 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:40:45.704111 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="ui" containerID="cri-o://67c45a7f61310f2003bd9cab2e189af623fbdfff1346180384dcf0aa179df68f" gracePeriod=30 Jun 11 21:40:45.704423 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:40:45.704109 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="offline" containerID="cri-o://3e651e4600335aebdafd8a39220d38c7fe0ec5c62df360a1f15ff35c694c7076" gracePeriod=30 Jun 11 21:40:45.857379 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:40:45.857343 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t"] Jun 11 21:40:45.857633 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:40:45.857605 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="registry" containerID="cri-o://5876c4f4b0ef9e6da53f6afd1ac37de99544677cb43c1bb32e39c7dd12a9e3f3" gracePeriod=30 Jun 11 21:40:45.857722 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:40:45.857653 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="online" containerID="cri-o://5bb1f52e35b9a285a6a3d78740529d7e0d15825f0707a02bdf8fd0f0b7617bbd" gracePeriod=30 Jun 11 21:40:45.857722 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:40:45.857651 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="ui" containerID="cri-o://82f116f2571752dca6a43b3b610044f3315858ce6e2e4264b7e4358fc3c8fded" gracePeriod=30 Jun 11 21:40:45.857826 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:40:45.857666 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="offline" containerID="cri-o://22af7d0ea9ad41e98d006ddcdef283eaf849939b6633749ee19e2ac3e4281407" gracePeriod=30 Jun 11 21:40:46.558195 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:40:46.558163 2570 generic.go:356] "Generic (PLEG): container finished" podID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerID="82f116f2571752dca6a43b3b610044f3315858ce6e2e4264b7e4358fc3c8fded" exitCode=0 Jun 11 21:40:46.558363 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:40:46.558235 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" event={"ID":"b624a0e1-25bf-41c5-bb23-a79cd9574b31","Type":"ContainerDied","Data":"82f116f2571752dca6a43b3b610044f3315858ce6e2e4264b7e4358fc3c8fded"} Jun 11 21:40:46.561047 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:40:46.561024 2570 generic.go:356] "Generic (PLEG): container finished" podID="d8af6452-beb7-4123-bf5a-e352416910d4" containerID="67c45a7f61310f2003bd9cab2e189af623fbdfff1346180384dcf0aa179df68f" exitCode=0 Jun 11 21:40:46.561047 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:40:46.561042 2570 generic.go:356] "Generic (PLEG): container finished" podID="d8af6452-beb7-4123-bf5a-e352416910d4" containerID="ae52b970902e7f0a9ae4a8c6d9e829d54f5bde58b0d0533902eee48c8d5af26a" exitCode=0 Jun 11 21:40:46.561217 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:40:46.561099 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" event={"ID":"d8af6452-beb7-4123-bf5a-e352416910d4","Type":"ContainerDied","Data":"67c45a7f61310f2003bd9cab2e189af623fbdfff1346180384dcf0aa179df68f"} Jun 11 21:40:46.561217 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:40:46.561148 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" event={"ID":"d8af6452-beb7-4123-bf5a-e352416910d4","Type":"ContainerDied","Data":"ae52b970902e7f0a9ae4a8c6d9e829d54f5bde58b0d0533902eee48c8d5af26a"} Jun 11 21:40:47.568142 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:40:47.568108 2570 generic.go:356] "Generic (PLEG): container finished" podID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerID="5bb1f52e35b9a285a6a3d78740529d7e0d15825f0707a02bdf8fd0f0b7617bbd" exitCode=0 Jun 11 21:40:47.568529 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:40:47.568183 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" event={"ID":"b624a0e1-25bf-41c5-bb23-a79cd9574b31","Type":"ContainerDied","Data":"5bb1f52e35b9a285a6a3d78740529d7e0d15825f0707a02bdf8fd0f0b7617bbd"} Jun 11 21:40:51.649922 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:40:51.649875 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.22:8443: connect: connection refused" Jun 11 21:40:51.973799 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:40:51.973701 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused" Jun 11 21:40:53.377632 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:40:53.377589 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused" Jun 11 21:40:54.311056 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:40:54.311011 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Jun 11 21:41:01.650720 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:01.650472 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.22:8443: connect: connection refused" Jun 11 21:41:01.973784 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:01.973697 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused" Jun 11 21:41:03.377256 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:03.377205 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused" Jun 11 21:41:04.311390 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:04.311348 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Jun 11 21:41:11.650167 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:11.650113 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.22:8443: connect: connection refused" Jun 11 21:41:11.650809 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:11.650258 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:41:11.973390 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:11.973300 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused" Jun 11 21:41:11.973532 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:11.973422 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:41:13.377783 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:13.377737 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused" Jun 11 21:41:13.378203 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:13.377870 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:41:14.311238 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:14.311199 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Jun 11 21:41:14.311407 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:14.311301 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:41:16.352714 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.352690 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:41:16.450074 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.450043 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d8af6452-beb7-4123-bf5a-e352416910d4-offline-tls\") pod \"d8af6452-beb7-4123-bf5a-e352416910d4\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " Jun 11 21:41:16.450212 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.450086 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d8af6452-beb7-4123-bf5a-e352416910d4-online-tls\") pod \"d8af6452-beb7-4123-bf5a-e352416910d4\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " Jun 11 21:41:16.450252 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.450214 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d8af6452-beb7-4123-bf5a-e352416910d4-ui-tls\") pod \"d8af6452-beb7-4123-bf5a-e352416910d4\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " Jun 11 21:41:16.450294 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.450279 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vn5n2\" (UniqueName: \"kubernetes.io/projected/d8af6452-beb7-4123-bf5a-e352416910d4-kube-api-access-vn5n2\") pod \"d8af6452-beb7-4123-bf5a-e352416910d4\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " Jun 11 21:41:16.450330 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.450315 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d8af6452-beb7-4123-bf5a-e352416910d4-feast-data\") pod \"d8af6452-beb7-4123-bf5a-e352416910d4\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " Jun 11 21:41:16.450784 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.450364 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d8af6452-beb7-4123-bf5a-e352416910d4-registry-tls\") pod \"d8af6452-beb7-4123-bf5a-e352416910d4\" (UID: \"d8af6452-beb7-4123-bf5a-e352416910d4\") " Jun 11 21:41:16.450784 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.450730 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8af6452-beb7-4123-bf5a-e352416910d4-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d8af6452-beb7-4123-bf5a-e352416910d4" (UID: "d8af6452-beb7-4123-bf5a-e352416910d4"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 21:41:16.450937 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.450874 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8af6452-beb7-4123-bf5a-e352416910d4-feast-data" (OuterVolumeSpecName: "feast-data") pod "d8af6452-beb7-4123-bf5a-e352416910d4" (UID: "d8af6452-beb7-4123-bf5a-e352416910d4"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 21:41:16.452381 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.452362 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8af6452-beb7-4123-bf5a-e352416910d4-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "d8af6452-beb7-4123-bf5a-e352416910d4" (UID: "d8af6452-beb7-4123-bf5a-e352416910d4"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:41:16.452442 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.452396 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8af6452-beb7-4123-bf5a-e352416910d4-kube-api-access-vn5n2" (OuterVolumeSpecName: "kube-api-access-vn5n2") pod "d8af6452-beb7-4123-bf5a-e352416910d4" (UID: "d8af6452-beb7-4123-bf5a-e352416910d4"). InnerVolumeSpecName "kube-api-access-vn5n2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 21:41:16.452602 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.452553 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8af6452-beb7-4123-bf5a-e352416910d4-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "d8af6452-beb7-4123-bf5a-e352416910d4" (UID: "d8af6452-beb7-4123-bf5a-e352416910d4"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:41:16.452698 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.452599 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8af6452-beb7-4123-bf5a-e352416910d4-online-tls" (OuterVolumeSpecName: "online-tls") pod "d8af6452-beb7-4123-bf5a-e352416910d4" (UID: "d8af6452-beb7-4123-bf5a-e352416910d4"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:41:16.501212 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.501184 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:41:16.551781 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.551747 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b624a0e1-25bf-41c5-bb23-a79cd9574b31-feast-data\") pod \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " Jun 11 21:41:16.551781 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.551785 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-registry-tls\") pod \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " Jun 11 21:41:16.552019 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.551809 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-online-tls\") pod \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " Jun 11 21:41:16.552019 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.551847 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-offline-tls\") pod \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " Jun 11 21:41:16.552019 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.551982 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-ui-tls\") pod \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " Jun 11 21:41:16.552019 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.552013 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7sw4t\" (UniqueName: \"kubernetes.io/projected/b624a0e1-25bf-41c5-bb23-a79cd9574b31-kube-api-access-7sw4t\") pod \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\" (UID: \"b624a0e1-25bf-41c5-bb23-a79cd9574b31\") " Jun 11 21:41:16.552274 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.552255 2570 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d8af6452-beb7-4123-bf5a-e352416910d4-offline-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:41:16.552334 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.552280 2570 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d8af6452-beb7-4123-bf5a-e352416910d4-online-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:41:16.552334 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.552294 2570 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d8af6452-beb7-4123-bf5a-e352416910d4-ui-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:41:16.552334 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.552309 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vn5n2\" (UniqueName: \"kubernetes.io/projected/d8af6452-beb7-4123-bf5a-e352416910d4-kube-api-access-vn5n2\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:41:16.552334 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.552313 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b624a0e1-25bf-41c5-bb23-a79cd9574b31-feast-data" (OuterVolumeSpecName: "feast-data") pod "b624a0e1-25bf-41c5-bb23-a79cd9574b31" (UID: "b624a0e1-25bf-41c5-bb23-a79cd9574b31"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 21:41:16.552528 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.552323 2570 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d8af6452-beb7-4123-bf5a-e352416910d4-feast-data\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:41:16.552528 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.552362 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d8af6452-beb7-4123-bf5a-e352416910d4-registry-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:41:16.554012 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.553983 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-online-tls" (OuterVolumeSpecName: "online-tls") pod "b624a0e1-25bf-41c5-bb23-a79cd9574b31" (UID: "b624a0e1-25bf-41c5-bb23-a79cd9574b31"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:41:16.554138 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.554053 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b624a0e1-25bf-41c5-bb23-a79cd9574b31" (UID: "b624a0e1-25bf-41c5-bb23-a79cd9574b31"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:41:16.554138 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.554074 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b624a0e1-25bf-41c5-bb23-a79cd9574b31-kube-api-access-7sw4t" (OuterVolumeSpecName: "kube-api-access-7sw4t") pod "b624a0e1-25bf-41c5-bb23-a79cd9574b31" (UID: "b624a0e1-25bf-41c5-bb23-a79cd9574b31"). InnerVolumeSpecName "kube-api-access-7sw4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 21:41:16.554346 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.554324 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "b624a0e1-25bf-41c5-bb23-a79cd9574b31" (UID: "b624a0e1-25bf-41c5-bb23-a79cd9574b31"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:41:16.554346 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.554337 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "b624a0e1-25bf-41c5-bb23-a79cd9574b31" (UID: "b624a0e1-25bf-41c5-bb23-a79cd9574b31"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:41:16.653540 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.653510 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7sw4t\" (UniqueName: \"kubernetes.io/projected/b624a0e1-25bf-41c5-bb23-a79cd9574b31-kube-api-access-7sw4t\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:41:16.653540 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.653538 2570 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b624a0e1-25bf-41c5-bb23-a79cd9574b31-feast-data\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:41:16.653540 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.653546 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-registry-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:41:16.653768 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.653555 2570 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-online-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:41:16.653768 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.653580 2570 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-offline-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:41:16.653768 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.653589 2570 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b624a0e1-25bf-41c5-bb23-a79cd9574b31-ui-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:41:16.677318 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.677281 2570 generic.go:356] "Generic (PLEG): container finished" podID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerID="22af7d0ea9ad41e98d006ddcdef283eaf849939b6633749ee19e2ac3e4281407" exitCode=137 Jun 11 21:41:16.677318 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.677308 2570 generic.go:356] "Generic (PLEG): container finished" podID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerID="5876c4f4b0ef9e6da53f6afd1ac37de99544677cb43c1bb32e39c7dd12a9e3f3" exitCode=137 Jun 11 21:41:16.677527 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.677343 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" Jun 11 21:41:16.677527 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.677347 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" event={"ID":"b624a0e1-25bf-41c5-bb23-a79cd9574b31","Type":"ContainerDied","Data":"22af7d0ea9ad41e98d006ddcdef283eaf849939b6633749ee19e2ac3e4281407"} Jun 11 21:41:16.677527 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.677392 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" event={"ID":"b624a0e1-25bf-41c5-bb23-a79cd9574b31","Type":"ContainerDied","Data":"5876c4f4b0ef9e6da53f6afd1ac37de99544677cb43c1bb32e39c7dd12a9e3f3"} Jun 11 21:41:16.677527 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.677407 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t" event={"ID":"b624a0e1-25bf-41c5-bb23-a79cd9574b31","Type":"ContainerDied","Data":"da81774d0065c89e2f55649fc99cf5a801936c28d7944e07731ff007ba0cc006"} Jun 11 21:41:16.677527 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.677426 2570 scope.go:117] "RemoveContainer" containerID="82f116f2571752dca6a43b3b610044f3315858ce6e2e4264b7e4358fc3c8fded" Jun 11 21:41:16.680093 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.680072 2570 generic.go:356] "Generic (PLEG): container finished" podID="d8af6452-beb7-4123-bf5a-e352416910d4" containerID="3e651e4600335aebdafd8a39220d38c7fe0ec5c62df360a1f15ff35c694c7076" exitCode=137 Jun 11 21:41:16.680208 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.680123 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" Jun 11 21:41:16.680208 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.680153 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" event={"ID":"d8af6452-beb7-4123-bf5a-e352416910d4","Type":"ContainerDied","Data":"3e651e4600335aebdafd8a39220d38c7fe0ec5c62df360a1f15ff35c694c7076"} Jun 11 21:41:16.680208 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.680201 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s" event={"ID":"d8af6452-beb7-4123-bf5a-e352416910d4","Type":"ContainerDied","Data":"80c06027306a7a62c38b7567a8880f2e2a9ea1b2242a8ca2cdd59044d3b2a238"} Jun 11 21:41:16.687135 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.687113 2570 scope.go:117] "RemoveContainer" containerID="22af7d0ea9ad41e98d006ddcdef283eaf849939b6633749ee19e2ac3e4281407" Jun 11 21:41:16.694842 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.694825 2570 scope.go:117] "RemoveContainer" containerID="5bb1f52e35b9a285a6a3d78740529d7e0d15825f0707a02bdf8fd0f0b7617bbd" Jun 11 21:41:16.702048 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.702023 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s"] Jun 11 21:41:16.702337 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.702316 2570 scope.go:117] "RemoveContainer" containerID="5876c4f4b0ef9e6da53f6afd1ac37de99544677cb43c1bb32e39c7dd12a9e3f3" Jun 11 21:41:16.705522 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.705497 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-jq97s"] Jun 11 21:41:16.711542 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.711404 2570 scope.go:117] "RemoveContainer" containerID="8dc31f21738728d0236923217e65471fcb694f4a79ac30a9b4a0d95b361f5bcb" Jun 11 21:41:16.715584 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.715543 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t"] Jun 11 21:41:16.719576 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.719541 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-h8m2t"] Jun 11 21:41:16.726836 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.726818 2570 scope.go:117] "RemoveContainer" containerID="c245182bdeeae27582178230c1092cef218f3f00df08aba9f6bdeed3c2d209fc" Jun 11 21:41:16.733941 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.733925 2570 scope.go:117] "RemoveContainer" containerID="82f116f2571752dca6a43b3b610044f3315858ce6e2e4264b7e4358fc3c8fded" Jun 11 21:41:16.734190 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:41:16.734172 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82f116f2571752dca6a43b3b610044f3315858ce6e2e4264b7e4358fc3c8fded\": container with ID starting with 82f116f2571752dca6a43b3b610044f3315858ce6e2e4264b7e4358fc3c8fded not found: ID does not exist" containerID="82f116f2571752dca6a43b3b610044f3315858ce6e2e4264b7e4358fc3c8fded" Jun 11 21:41:16.734251 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.734203 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82f116f2571752dca6a43b3b610044f3315858ce6e2e4264b7e4358fc3c8fded"} err="failed to get container status \"82f116f2571752dca6a43b3b610044f3315858ce6e2e4264b7e4358fc3c8fded\": rpc error: code = NotFound desc = could not find container \"82f116f2571752dca6a43b3b610044f3315858ce6e2e4264b7e4358fc3c8fded\": container with ID starting with 82f116f2571752dca6a43b3b610044f3315858ce6e2e4264b7e4358fc3c8fded not found: ID does not exist" Jun 11 21:41:16.734251 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.734223 2570 scope.go:117] "RemoveContainer" containerID="22af7d0ea9ad41e98d006ddcdef283eaf849939b6633749ee19e2ac3e4281407" Jun 11 21:41:16.734488 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:41:16.734469 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22af7d0ea9ad41e98d006ddcdef283eaf849939b6633749ee19e2ac3e4281407\": container with ID starting with 22af7d0ea9ad41e98d006ddcdef283eaf849939b6633749ee19e2ac3e4281407 not found: ID does not exist" containerID="22af7d0ea9ad41e98d006ddcdef283eaf849939b6633749ee19e2ac3e4281407" Jun 11 21:41:16.734531 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.734496 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22af7d0ea9ad41e98d006ddcdef283eaf849939b6633749ee19e2ac3e4281407"} err="failed to get container status \"22af7d0ea9ad41e98d006ddcdef283eaf849939b6633749ee19e2ac3e4281407\": rpc error: code = NotFound desc = could not find container \"22af7d0ea9ad41e98d006ddcdef283eaf849939b6633749ee19e2ac3e4281407\": container with ID starting with 22af7d0ea9ad41e98d006ddcdef283eaf849939b6633749ee19e2ac3e4281407 not found: ID does not exist" Jun 11 21:41:16.734531 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.734510 2570 scope.go:117] "RemoveContainer" containerID="5bb1f52e35b9a285a6a3d78740529d7e0d15825f0707a02bdf8fd0f0b7617bbd" Jun 11 21:41:16.734766 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:41:16.734744 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bb1f52e35b9a285a6a3d78740529d7e0d15825f0707a02bdf8fd0f0b7617bbd\": container with ID starting with 5bb1f52e35b9a285a6a3d78740529d7e0d15825f0707a02bdf8fd0f0b7617bbd not found: ID does not exist" containerID="5bb1f52e35b9a285a6a3d78740529d7e0d15825f0707a02bdf8fd0f0b7617bbd" Jun 11 21:41:16.734852 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.734773 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bb1f52e35b9a285a6a3d78740529d7e0d15825f0707a02bdf8fd0f0b7617bbd"} err="failed to get container status \"5bb1f52e35b9a285a6a3d78740529d7e0d15825f0707a02bdf8fd0f0b7617bbd\": rpc error: code = NotFound desc = could not find container \"5bb1f52e35b9a285a6a3d78740529d7e0d15825f0707a02bdf8fd0f0b7617bbd\": container with ID starting with 5bb1f52e35b9a285a6a3d78740529d7e0d15825f0707a02bdf8fd0f0b7617bbd not found: ID does not exist" Jun 11 21:41:16.734852 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.734788 2570 scope.go:117] "RemoveContainer" containerID="5876c4f4b0ef9e6da53f6afd1ac37de99544677cb43c1bb32e39c7dd12a9e3f3" Jun 11 21:41:16.734964 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:41:16.734949 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5876c4f4b0ef9e6da53f6afd1ac37de99544677cb43c1bb32e39c7dd12a9e3f3\": container with ID starting with 5876c4f4b0ef9e6da53f6afd1ac37de99544677cb43c1bb32e39c7dd12a9e3f3 not found: ID does not exist" containerID="5876c4f4b0ef9e6da53f6afd1ac37de99544677cb43c1bb32e39c7dd12a9e3f3" Jun 11 21:41:16.735015 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.734968 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5876c4f4b0ef9e6da53f6afd1ac37de99544677cb43c1bb32e39c7dd12a9e3f3"} err="failed to get container status \"5876c4f4b0ef9e6da53f6afd1ac37de99544677cb43c1bb32e39c7dd12a9e3f3\": rpc error: code = NotFound desc = could not find container \"5876c4f4b0ef9e6da53f6afd1ac37de99544677cb43c1bb32e39c7dd12a9e3f3\": container with ID starting with 5876c4f4b0ef9e6da53f6afd1ac37de99544677cb43c1bb32e39c7dd12a9e3f3 not found: ID does not exist" Jun 11 21:41:16.735015 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.734986 2570 scope.go:117] "RemoveContainer" containerID="8dc31f21738728d0236923217e65471fcb694f4a79ac30a9b4a0d95b361f5bcb" Jun 11 21:41:16.735223 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:41:16.735206 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dc31f21738728d0236923217e65471fcb694f4a79ac30a9b4a0d95b361f5bcb\": container with ID starting with 8dc31f21738728d0236923217e65471fcb694f4a79ac30a9b4a0d95b361f5bcb not found: ID does not exist" containerID="8dc31f21738728d0236923217e65471fcb694f4a79ac30a9b4a0d95b361f5bcb" Jun 11 21:41:16.735291 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.735233 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc31f21738728d0236923217e65471fcb694f4a79ac30a9b4a0d95b361f5bcb"} err="failed to get container status \"8dc31f21738728d0236923217e65471fcb694f4a79ac30a9b4a0d95b361f5bcb\": rpc error: code = NotFound desc = could not find container \"8dc31f21738728d0236923217e65471fcb694f4a79ac30a9b4a0d95b361f5bcb\": container with ID starting with 8dc31f21738728d0236923217e65471fcb694f4a79ac30a9b4a0d95b361f5bcb not found: ID does not exist" Jun 11 21:41:16.735291 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.735252 2570 scope.go:117] "RemoveContainer" containerID="c245182bdeeae27582178230c1092cef218f3f00df08aba9f6bdeed3c2d209fc" Jun 11 21:41:16.735450 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:41:16.735435 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c245182bdeeae27582178230c1092cef218f3f00df08aba9f6bdeed3c2d209fc\": container with ID starting with c245182bdeeae27582178230c1092cef218f3f00df08aba9f6bdeed3c2d209fc not found: ID does not exist" containerID="c245182bdeeae27582178230c1092cef218f3f00df08aba9f6bdeed3c2d209fc" Jun 11 21:41:16.735491 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.735455 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c245182bdeeae27582178230c1092cef218f3f00df08aba9f6bdeed3c2d209fc"} err="failed to get container status \"c245182bdeeae27582178230c1092cef218f3f00df08aba9f6bdeed3c2d209fc\": rpc error: code = NotFound desc = could not find container \"c245182bdeeae27582178230c1092cef218f3f00df08aba9f6bdeed3c2d209fc\": container with ID starting with c245182bdeeae27582178230c1092cef218f3f00df08aba9f6bdeed3c2d209fc not found: ID does not exist" Jun 11 21:41:16.735491 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.735468 2570 scope.go:117] "RemoveContainer" containerID="82f116f2571752dca6a43b3b610044f3315858ce6e2e4264b7e4358fc3c8fded" Jun 11 21:41:16.735699 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.735673 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82f116f2571752dca6a43b3b610044f3315858ce6e2e4264b7e4358fc3c8fded"} err="failed to get container status \"82f116f2571752dca6a43b3b610044f3315858ce6e2e4264b7e4358fc3c8fded\": rpc error: code = NotFound desc = could not find container \"82f116f2571752dca6a43b3b610044f3315858ce6e2e4264b7e4358fc3c8fded\": container with ID starting with 82f116f2571752dca6a43b3b610044f3315858ce6e2e4264b7e4358fc3c8fded not found: ID does not exist" Jun 11 21:41:16.735779 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.735701 2570 scope.go:117] "RemoveContainer" containerID="22af7d0ea9ad41e98d006ddcdef283eaf849939b6633749ee19e2ac3e4281407" Jun 11 21:41:16.735942 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.735925 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22af7d0ea9ad41e98d006ddcdef283eaf849939b6633749ee19e2ac3e4281407"} err="failed to get container status \"22af7d0ea9ad41e98d006ddcdef283eaf849939b6633749ee19e2ac3e4281407\": rpc error: code = NotFound desc = could not find container \"22af7d0ea9ad41e98d006ddcdef283eaf849939b6633749ee19e2ac3e4281407\": container with ID starting with 22af7d0ea9ad41e98d006ddcdef283eaf849939b6633749ee19e2ac3e4281407 not found: ID does not exist" Jun 11 21:41:16.735991 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.735943 2570 scope.go:117] "RemoveContainer" containerID="5bb1f52e35b9a285a6a3d78740529d7e0d15825f0707a02bdf8fd0f0b7617bbd" Jun 11 21:41:16.736138 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.736120 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bb1f52e35b9a285a6a3d78740529d7e0d15825f0707a02bdf8fd0f0b7617bbd"} err="failed to get container status \"5bb1f52e35b9a285a6a3d78740529d7e0d15825f0707a02bdf8fd0f0b7617bbd\": rpc error: code = NotFound desc = could not find container \"5bb1f52e35b9a285a6a3d78740529d7e0d15825f0707a02bdf8fd0f0b7617bbd\": container with ID starting with 5bb1f52e35b9a285a6a3d78740529d7e0d15825f0707a02bdf8fd0f0b7617bbd not found: ID does not exist" Jun 11 21:41:16.736177 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.736138 2570 scope.go:117] "RemoveContainer" containerID="5876c4f4b0ef9e6da53f6afd1ac37de99544677cb43c1bb32e39c7dd12a9e3f3" Jun 11 21:41:16.736332 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.736317 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5876c4f4b0ef9e6da53f6afd1ac37de99544677cb43c1bb32e39c7dd12a9e3f3"} err="failed to get container status \"5876c4f4b0ef9e6da53f6afd1ac37de99544677cb43c1bb32e39c7dd12a9e3f3\": rpc error: code = NotFound desc = could not find container \"5876c4f4b0ef9e6da53f6afd1ac37de99544677cb43c1bb32e39c7dd12a9e3f3\": container with ID starting with 5876c4f4b0ef9e6da53f6afd1ac37de99544677cb43c1bb32e39c7dd12a9e3f3 not found: ID does not exist" Jun 11 21:41:16.736373 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.736334 2570 scope.go:117] "RemoveContainer" containerID="8dc31f21738728d0236923217e65471fcb694f4a79ac30a9b4a0d95b361f5bcb" Jun 11 21:41:16.736534 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.736518 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc31f21738728d0236923217e65471fcb694f4a79ac30a9b4a0d95b361f5bcb"} err="failed to get container status \"8dc31f21738728d0236923217e65471fcb694f4a79ac30a9b4a0d95b361f5bcb\": rpc error: code = NotFound desc = could not find container \"8dc31f21738728d0236923217e65471fcb694f4a79ac30a9b4a0d95b361f5bcb\": container with ID starting with 8dc31f21738728d0236923217e65471fcb694f4a79ac30a9b4a0d95b361f5bcb not found: ID does not exist" Jun 11 21:41:16.736534 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.736533 2570 scope.go:117] "RemoveContainer" containerID="c245182bdeeae27582178230c1092cef218f3f00df08aba9f6bdeed3c2d209fc" Jun 11 21:41:16.736778 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.736762 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c245182bdeeae27582178230c1092cef218f3f00df08aba9f6bdeed3c2d209fc"} err="failed to get container status \"c245182bdeeae27582178230c1092cef218f3f00df08aba9f6bdeed3c2d209fc\": rpc error: code = NotFound desc = could not find container \"c245182bdeeae27582178230c1092cef218f3f00df08aba9f6bdeed3c2d209fc\": container with ID starting with c245182bdeeae27582178230c1092cef218f3f00df08aba9f6bdeed3c2d209fc not found: ID does not exist" Jun 11 21:41:16.736823 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.736779 2570 scope.go:117] "RemoveContainer" containerID="67c45a7f61310f2003bd9cab2e189af623fbdfff1346180384dcf0aa179df68f" Jun 11 21:41:16.744006 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.743992 2570 scope.go:117] "RemoveContainer" containerID="3e651e4600335aebdafd8a39220d38c7fe0ec5c62df360a1f15ff35c694c7076" Jun 11 21:41:16.750885 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.750870 2570 scope.go:117] "RemoveContainer" containerID="ae52b970902e7f0a9ae4a8c6d9e829d54f5bde58b0d0533902eee48c8d5af26a" Jun 11 21:41:16.757825 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.757809 2570 scope.go:117] "RemoveContainer" containerID="9a0cf98aaf8a32afe5b6e40656c3204492f4ec383bd6b7bfde92c54aa9109c28" Jun 11 21:41:16.764467 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.764452 2570 scope.go:117] "RemoveContainer" containerID="5436fe339c58072754cf5e4efd0531eea93a544547a949632f5e451252db9376" Jun 11 21:41:16.771001 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.770985 2570 scope.go:117] "RemoveContainer" containerID="67c45a7f61310f2003bd9cab2e189af623fbdfff1346180384dcf0aa179df68f" Jun 11 21:41:16.771257 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:41:16.771239 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67c45a7f61310f2003bd9cab2e189af623fbdfff1346180384dcf0aa179df68f\": container with ID starting with 67c45a7f61310f2003bd9cab2e189af623fbdfff1346180384dcf0aa179df68f not found: ID does not exist" containerID="67c45a7f61310f2003bd9cab2e189af623fbdfff1346180384dcf0aa179df68f" Jun 11 21:41:16.771309 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.771264 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67c45a7f61310f2003bd9cab2e189af623fbdfff1346180384dcf0aa179df68f"} err="failed to get container status \"67c45a7f61310f2003bd9cab2e189af623fbdfff1346180384dcf0aa179df68f\": rpc error: code = NotFound desc = could not find container \"67c45a7f61310f2003bd9cab2e189af623fbdfff1346180384dcf0aa179df68f\": container with ID starting with 67c45a7f61310f2003bd9cab2e189af623fbdfff1346180384dcf0aa179df68f not found: ID does not exist" Jun 11 21:41:16.771309 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.771280 2570 scope.go:117] "RemoveContainer" containerID="3e651e4600335aebdafd8a39220d38c7fe0ec5c62df360a1f15ff35c694c7076" Jun 11 21:41:16.771502 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:41:16.771485 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e651e4600335aebdafd8a39220d38c7fe0ec5c62df360a1f15ff35c694c7076\": container with ID starting with 3e651e4600335aebdafd8a39220d38c7fe0ec5c62df360a1f15ff35c694c7076 not found: ID does not exist" containerID="3e651e4600335aebdafd8a39220d38c7fe0ec5c62df360a1f15ff35c694c7076" Jun 11 21:41:16.771546 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.771507 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e651e4600335aebdafd8a39220d38c7fe0ec5c62df360a1f15ff35c694c7076"} err="failed to get container status \"3e651e4600335aebdafd8a39220d38c7fe0ec5c62df360a1f15ff35c694c7076\": rpc error: code = NotFound desc = could not find container \"3e651e4600335aebdafd8a39220d38c7fe0ec5c62df360a1f15ff35c694c7076\": container with ID starting with 3e651e4600335aebdafd8a39220d38c7fe0ec5c62df360a1f15ff35c694c7076 not found: ID does not exist" Jun 11 21:41:16.771546 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.771521 2570 scope.go:117] "RemoveContainer" containerID="ae52b970902e7f0a9ae4a8c6d9e829d54f5bde58b0d0533902eee48c8d5af26a" Jun 11 21:41:16.771768 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:41:16.771753 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae52b970902e7f0a9ae4a8c6d9e829d54f5bde58b0d0533902eee48c8d5af26a\": container with ID starting with ae52b970902e7f0a9ae4a8c6d9e829d54f5bde58b0d0533902eee48c8d5af26a not found: ID does not exist" containerID="ae52b970902e7f0a9ae4a8c6d9e829d54f5bde58b0d0533902eee48c8d5af26a" Jun 11 21:41:16.771807 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.771772 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae52b970902e7f0a9ae4a8c6d9e829d54f5bde58b0d0533902eee48c8d5af26a"} err="failed to get container status \"ae52b970902e7f0a9ae4a8c6d9e829d54f5bde58b0d0533902eee48c8d5af26a\": rpc error: code = NotFound desc = could not find container \"ae52b970902e7f0a9ae4a8c6d9e829d54f5bde58b0d0533902eee48c8d5af26a\": container with ID starting with ae52b970902e7f0a9ae4a8c6d9e829d54f5bde58b0d0533902eee48c8d5af26a not found: ID does not exist" Jun 11 21:41:16.771807 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.771785 2570 scope.go:117] "RemoveContainer" containerID="9a0cf98aaf8a32afe5b6e40656c3204492f4ec383bd6b7bfde92c54aa9109c28" Jun 11 21:41:16.771992 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:41:16.771977 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a0cf98aaf8a32afe5b6e40656c3204492f4ec383bd6b7bfde92c54aa9109c28\": container with ID starting with 9a0cf98aaf8a32afe5b6e40656c3204492f4ec383bd6b7bfde92c54aa9109c28 not found: ID does not exist" containerID="9a0cf98aaf8a32afe5b6e40656c3204492f4ec383bd6b7bfde92c54aa9109c28" Jun 11 21:41:16.772037 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.771996 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a0cf98aaf8a32afe5b6e40656c3204492f4ec383bd6b7bfde92c54aa9109c28"} err="failed to get container status \"9a0cf98aaf8a32afe5b6e40656c3204492f4ec383bd6b7bfde92c54aa9109c28\": rpc error: code = NotFound desc = could not find container \"9a0cf98aaf8a32afe5b6e40656c3204492f4ec383bd6b7bfde92c54aa9109c28\": container with ID starting with 9a0cf98aaf8a32afe5b6e40656c3204492f4ec383bd6b7bfde92c54aa9109c28 not found: ID does not exist" Jun 11 21:41:16.772037 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.772009 2570 scope.go:117] "RemoveContainer" containerID="5436fe339c58072754cf5e4efd0531eea93a544547a949632f5e451252db9376" Jun 11 21:41:16.772228 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:41:16.772211 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5436fe339c58072754cf5e4efd0531eea93a544547a949632f5e451252db9376\": container with ID starting with 5436fe339c58072754cf5e4efd0531eea93a544547a949632f5e451252db9376 not found: ID does not exist" containerID="5436fe339c58072754cf5e4efd0531eea93a544547a949632f5e451252db9376" Jun 11 21:41:16.772273 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:16.772234 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5436fe339c58072754cf5e4efd0531eea93a544547a949632f5e451252db9376"} err="failed to get container status \"5436fe339c58072754cf5e4efd0531eea93a544547a949632f5e451252db9376\": rpc error: code = NotFound desc = could not find container \"5436fe339c58072754cf5e4efd0531eea93a544547a949632f5e451252db9376\": container with ID starting with 5436fe339c58072754cf5e4efd0531eea93a544547a949632f5e451252db9376 not found: ID does not exist" Jun 11 21:41:18.051107 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:18.051072 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" path="/var/lib/kubelet/pods/b624a0e1-25bf-41c5-bb23-a79cd9574b31/volumes" Jun 11 21:41:18.051711 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:18.051697 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" path="/var/lib/kubelet/pods/d8af6452-beb7-4123-bf5a-e352416910d4/volumes" Jun 11 21:41:34.475773 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:34.475736 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6d54c7db9d-r49qx"] Jun 11 21:41:34.478370 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:34.475930 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-r49qx" podUID="9c3eb1b5-0528-4273-8331-9ef8676bfa1d" containerName="manager" containerID="cri-o://ad522e99855753e8a06cd6af69dea3c0330dbc78e45f5b3a36e90b1d790196c5" gracePeriod=10 Jun 11 21:41:34.721164 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:34.721141 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-r49qx" Jun 11 21:41:34.750077 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:34.749991 2570 generic.go:356] "Generic (PLEG): container finished" podID="9c3eb1b5-0528-4273-8331-9ef8676bfa1d" containerID="ad522e99855753e8a06cd6af69dea3c0330dbc78e45f5b3a36e90b1d790196c5" exitCode=0 Jun 11 21:41:34.750077 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:34.750033 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-r49qx" Jun 11 21:41:34.750077 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:34.750036 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-r49qx" event={"ID":"9c3eb1b5-0528-4273-8331-9ef8676bfa1d","Type":"ContainerDied","Data":"ad522e99855753e8a06cd6af69dea3c0330dbc78e45f5b3a36e90b1d790196c5"} Jun 11 21:41:34.750293 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:34.750087 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-r49qx" event={"ID":"9c3eb1b5-0528-4273-8331-9ef8676bfa1d","Type":"ContainerDied","Data":"7326800cc18310e56ac4c70c8f732a4f559b72312c2d9c43bbef4f9d4cb5496c"} Jun 11 21:41:34.750293 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:34.750121 2570 scope.go:117] "RemoveContainer" containerID="ad522e99855753e8a06cd6af69dea3c0330dbc78e45f5b3a36e90b1d790196c5" Jun 11 21:41:34.758720 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:34.758698 2570 scope.go:117] "RemoveContainer" containerID="ad522e99855753e8a06cd6af69dea3c0330dbc78e45f5b3a36e90b1d790196c5" Jun 11 21:41:34.759041 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:41:34.759022 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad522e99855753e8a06cd6af69dea3c0330dbc78e45f5b3a36e90b1d790196c5\": container with ID starting with ad522e99855753e8a06cd6af69dea3c0330dbc78e45f5b3a36e90b1d790196c5 not found: ID does not exist" containerID="ad522e99855753e8a06cd6af69dea3c0330dbc78e45f5b3a36e90b1d790196c5" Jun 11 21:41:34.759102 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:34.759050 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad522e99855753e8a06cd6af69dea3c0330dbc78e45f5b3a36e90b1d790196c5"} err="failed to get container status \"ad522e99855753e8a06cd6af69dea3c0330dbc78e45f5b3a36e90b1d790196c5\": rpc error: code = NotFound desc = could not find container \"ad522e99855753e8a06cd6af69dea3c0330dbc78e45f5b3a36e90b1d790196c5\": container with ID starting with ad522e99855753e8a06cd6af69dea3c0330dbc78e45f5b3a36e90b1d790196c5 not found: ID does not exist" Jun 11 21:41:34.795354 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:34.795322 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rq9q\" (UniqueName: \"kubernetes.io/projected/9c3eb1b5-0528-4273-8331-9ef8676bfa1d-kube-api-access-8rq9q\") pod \"9c3eb1b5-0528-4273-8331-9ef8676bfa1d\" (UID: \"9c3eb1b5-0528-4273-8331-9ef8676bfa1d\") " Jun 11 21:41:34.797452 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:34.797426 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c3eb1b5-0528-4273-8331-9ef8676bfa1d-kube-api-access-8rq9q" (OuterVolumeSpecName: "kube-api-access-8rq9q") pod "9c3eb1b5-0528-4273-8331-9ef8676bfa1d" (UID: "9c3eb1b5-0528-4273-8331-9ef8676bfa1d"). InnerVolumeSpecName "kube-api-access-8rq9q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 21:41:34.896614 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:34.896539 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rq9q\" (UniqueName: \"kubernetes.io/projected/9c3eb1b5-0528-4273-8331-9ef8676bfa1d-kube-api-access-8rq9q\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:41:35.071771 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:35.071738 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6d54c7db9d-r49qx"] Jun 11 21:41:35.076777 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:35.076750 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6d54c7db9d-r49qx"] Jun 11 21:41:36.050929 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:36.050888 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c3eb1b5-0528-4273-8331-9ef8676bfa1d" path="/var/lib/kubelet/pods/9c3eb1b5-0528-4273-8331-9ef8676bfa1d/volumes" Jun 11 21:41:38.844659 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.844623 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6d54c7db9d-8c4kq"] Jun 11 21:41:38.845129 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.844987 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="registry" Jun 11 21:41:38.845129 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845004 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="registry" Jun 11 21:41:38.845129 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845016 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="ui" Jun 11 21:41:38.845129 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845024 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="ui" Jun 11 21:41:38.845129 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845035 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="feast-apply" Jun 11 21:41:38.845129 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845044 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="feast-apply" Jun 11 21:41:38.845129 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845058 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="offline" Jun 11 21:41:38.845129 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845066 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="offline" Jun 11 21:41:38.845129 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845082 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="feast-init" Jun 11 21:41:38.845129 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845090 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="feast-init" Jun 11 21:41:38.845129 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845098 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c3eb1b5-0528-4273-8331-9ef8676bfa1d" containerName="manager" Jun 11 21:41:38.845129 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845106 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c3eb1b5-0528-4273-8331-9ef8676bfa1d" containerName="manager" Jun 11 21:41:38.845129 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845114 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="offline" Jun 11 21:41:38.845129 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845121 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="offline" Jun 11 21:41:38.845129 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845132 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="feast-apply" Jun 11 21:41:38.845841 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845141 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="feast-apply" Jun 11 21:41:38.845841 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845152 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="online" Jun 11 21:41:38.845841 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845160 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="online" Jun 11 21:41:38.845841 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845170 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="online" Jun 11 21:41:38.845841 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845178 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="online" Jun 11 21:41:38.845841 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845191 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="feast-init" Jun 11 21:41:38.845841 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845199 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="feast-init" Jun 11 21:41:38.845841 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845211 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="ui" Jun 11 21:41:38.845841 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845219 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="ui" Jun 11 21:41:38.845841 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845284 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c3eb1b5-0528-4273-8331-9ef8676bfa1d" containerName="manager" Jun 11 21:41:38.845841 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845295 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="online" Jun 11 21:41:38.845841 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845304 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="online" Jun 11 21:41:38.845841 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845315 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="ui" Jun 11 21:41:38.845841 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845324 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="ui" Jun 11 21:41:38.845841 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845333 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8af6452-beb7-4123-bf5a-e352416910d4" containerName="offline" Jun 11 21:41:38.845841 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845343 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="offline" Jun 11 21:41:38.845841 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.845351 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b624a0e1-25bf-41c5-bb23-a79cd9574b31" containerName="registry" Jun 11 21:41:38.848486 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.848465 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-8c4kq" Jun 11 21:41:38.851153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.851110 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Jun 11 21:41:38.851275 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.851154 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Jun 11 21:41:38.852040 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.852026 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-725cf\"" Jun 11 21:41:38.855674 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.855651 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6d54c7db9d-8c4kq"] Jun 11 21:41:38.928063 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:38.928024 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlbp2\" (UniqueName: \"kubernetes.io/projected/03484b2b-9889-4220-b313-a710f5e12ae3-kube-api-access-hlbp2\") pod \"feast-operator-controller-manager-6d54c7db9d-8c4kq\" (UID: \"03484b2b-9889-4220-b313-a710f5e12ae3\") " pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-8c4kq" Jun 11 21:41:39.028589 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:39.028527 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlbp2\" (UniqueName: \"kubernetes.io/projected/03484b2b-9889-4220-b313-a710f5e12ae3-kube-api-access-hlbp2\") pod \"feast-operator-controller-manager-6d54c7db9d-8c4kq\" (UID: \"03484b2b-9889-4220-b313-a710f5e12ae3\") " pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-8c4kq" Jun 11 21:41:39.036782 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:39.036757 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlbp2\" (UniqueName: \"kubernetes.io/projected/03484b2b-9889-4220-b313-a710f5e12ae3-kube-api-access-hlbp2\") pod \"feast-operator-controller-manager-6d54c7db9d-8c4kq\" (UID: \"03484b2b-9889-4220-b313-a710f5e12ae3\") " pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-8c4kq" Jun 11 21:41:39.158685 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:39.158588 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-8c4kq" Jun 11 21:41:39.303303 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:39.303270 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6d54c7db9d-8c4kq"] Jun 11 21:41:39.306303 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:41:39.306274 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03484b2b_9889_4220_b313_a710f5e12ae3.slice/crio-87d2bda7cb2274686a15aae744e98d12964570d18fc2098a5fde4bc64ccb8f46 WatchSource:0}: Error finding container 87d2bda7cb2274686a15aae744e98d12964570d18fc2098a5fde4bc64ccb8f46: Status 404 returned error can't find the container with id 87d2bda7cb2274686a15aae744e98d12964570d18fc2098a5fde4bc64ccb8f46 Jun 11 21:41:39.769531 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:39.769489 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-8c4kq" event={"ID":"03484b2b-9889-4220-b313-a710f5e12ae3","Type":"ContainerStarted","Data":"643b0c7d4e3ce3a730ddc11195c467cc4ac0d378a91063f071849d71861e8294"} Jun 11 21:41:39.769779 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:39.769553 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-8c4kq" event={"ID":"03484b2b-9889-4220-b313-a710f5e12ae3","Type":"ContainerStarted","Data":"87d2bda7cb2274686a15aae744e98d12964570d18fc2098a5fde4bc64ccb8f46"} Jun 11 21:41:39.769779 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:39.769606 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-8c4kq" Jun 11 21:41:39.784147 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:39.784089 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-8c4kq" podStartSLOduration=1.784070673 podStartE2EDuration="1.784070673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:41:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:41:39.783307706 +0000 UTC m=+1558.194631256" watchObservedRunningTime="2026-06-11 21:41:39.784070673 +0000 UTC m=+1558.195394211" Jun 11 21:41:50.776209 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:50.776178 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-8c4kq" Jun 11 21:41:58.159624 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.159586 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs"] Jun 11 21:41:58.164541 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.164519 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:41:58.167939 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.166981 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Jun 11 21:41:58.167939 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.167342 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Jun 11 21:41:58.167939 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.167416 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Jun 11 21:41:58.167939 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.167742 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jun 11 21:41:58.167939 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.167808 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-75x8r\"" Jun 11 21:41:58.167939 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.167742 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jun 11 21:41:58.168277 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.168031 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Jun 11 21:41:58.176684 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.176657 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs"] Jun 11 21:41:58.286416 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.286375 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-2g4zs\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:41:58.286416 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.286417 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-2g4zs\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:41:58.286668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.286452 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8gdf\" (UniqueName: \"kubernetes.io/projected/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-kube-api-access-d8gdf\") pod \"feast-simple-feast-setup-75c64c888b-2g4zs\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:41:58.286668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.286532 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-2g4zs\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:41:58.286668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.286594 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-2g4zs\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:41:58.286668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.286662 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-2g4zs\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:41:58.387458 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.387420 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-2g4zs\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:41:58.387458 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.387458 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-2g4zs\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:41:58.387732 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.387497 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-2g4zs\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:41:58.387732 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.387516 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-2g4zs\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:41:58.387732 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.387531 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-2g4zs\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:41:58.387732 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.387554 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8gdf\" (UniqueName: \"kubernetes.io/projected/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-kube-api-access-d8gdf\") pod \"feast-simple-feast-setup-75c64c888b-2g4zs\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:41:58.387732 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:41:58.387665 2570 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Jun 11 21:41:58.387989 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:41:58.387750 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-online-tls podName:c54277cb-e73f-4c05-96eb-a03dcea6b1c2 nodeName:}" failed. No retries permitted until 2026-06-11 21:41:58.88772964 +0000 UTC m=+1577.299053159 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-online-tls") pod "feast-simple-feast-setup-75c64c888b-2g4zs" (UID: "c54277cb-e73f-4c05-96eb-a03dcea6b1c2") : secret "feast-simple-feast-setup-online-tls" not found Jun 11 21:41:58.387989 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.387895 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-2g4zs\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:41:58.390153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.390121 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-2g4zs\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:41:58.390153 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.390131 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-2g4zs\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:41:58.390341 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.390221 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-2g4zs\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:41:58.398948 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.398922 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8gdf\" (UniqueName: \"kubernetes.io/projected/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-kube-api-access-d8gdf\") pod \"feast-simple-feast-setup-75c64c888b-2g4zs\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:41:58.892770 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.892731 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-2g4zs\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:41:58.895154 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:58.895129 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-2g4zs\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:41:59.076134 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:59.076094 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:41:59.207705 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:59.207674 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs"] Jun 11 21:41:59.208928 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:41:59.208902 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc54277cb_e73f_4c05_96eb_a03dcea6b1c2.slice/crio-4bbab0c6327644b7e3bbf24419baadf33766e93998e7ebe7fc48476908849bfd WatchSource:0}: Error finding container 4bbab0c6327644b7e3bbf24419baadf33766e93998e7ebe7fc48476908849bfd: Status 404 returned error can't find the container with id 4bbab0c6327644b7e3bbf24419baadf33766e93998e7ebe7fc48476908849bfd Jun 11 21:41:59.842781 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:59.842751 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" event={"ID":"c54277cb-e73f-4c05-96eb-a03dcea6b1c2","Type":"ContainerStarted","Data":"fd26354197d6d813593ff254af21b98c5ac7c6dcffd47e6787d9fdff5d204f66"} Jun 11 21:41:59.842944 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:41:59.842793 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" event={"ID":"c54277cb-e73f-4c05-96eb-a03dcea6b1c2","Type":"ContainerStarted","Data":"4bbab0c6327644b7e3bbf24419baadf33766e93998e7ebe7fc48476908849bfd"} Jun 11 21:42:04.867505 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:04.867466 2570 generic.go:356] "Generic (PLEG): container finished" podID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerID="fd26354197d6d813593ff254af21b98c5ac7c6dcffd47e6787d9fdff5d204f66" exitCode=0 Jun 11 21:42:04.868022 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:04.867546 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" event={"ID":"c54277cb-e73f-4c05-96eb-a03dcea6b1c2","Type":"ContainerDied","Data":"fd26354197d6d813593ff254af21b98c5ac7c6dcffd47e6787d9fdff5d204f66"} Jun 11 21:42:05.874014 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:05.873978 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" event={"ID":"c54277cb-e73f-4c05-96eb-a03dcea6b1c2","Type":"ContainerStarted","Data":"bd82ca36e405c6d7efc302e0f933b2ca8f6e70b448b7a40b425c9dede30a781a"} Jun 11 21:42:10.895509 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:10.895470 2570 generic.go:356] "Generic (PLEG): container finished" podID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerID="bd82ca36e405c6d7efc302e0f933b2ca8f6e70b448b7a40b425c9dede30a781a" exitCode=0 Jun 11 21:42:10.895509 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:10.895510 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" event={"ID":"c54277cb-e73f-4c05-96eb-a03dcea6b1c2","Type":"ContainerDied","Data":"bd82ca36e405c6d7efc302e0f933b2ca8f6e70b448b7a40b425c9dede30a781a"} Jun 11 21:42:11.905958 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:11.905912 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" event={"ID":"c54277cb-e73f-4c05-96eb-a03dcea6b1c2","Type":"ContainerStarted","Data":"81082fd60a1765436b2f30cf027b0d3ad785b2d456896d9fb41165b5ff3170ec"} Jun 11 21:42:11.906585 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:11.905981 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" event={"ID":"c54277cb-e73f-4c05-96eb-a03dcea6b1c2","Type":"ContainerStarted","Data":"5b1a60d7a0731f6ae1724501ecc09979f2a1630fff25dbf605b65439121946a5"} Jun 11 21:42:11.906585 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:11.906004 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" event={"ID":"c54277cb-e73f-4c05-96eb-a03dcea6b1c2","Type":"ContainerStarted","Data":"559d9bd8615bc09d7d409753831684ba3693d2fcfc9f43a6a42225e8d7da618c"} Jun 11 21:42:11.906585 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:11.906023 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" event={"ID":"c54277cb-e73f-4c05-96eb-a03dcea6b1c2","Type":"ContainerStarted","Data":"040804a5006df9efc49d870091566266b55c18f8963c4857fee6f7155cd1d7a2"} Jun 11 21:42:11.928829 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:11.928758 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" podStartSLOduration=13.928733696 podStartE2EDuration="13.928733696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:41:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:42:11.92735803 +0000 UTC m=+1590.338681573" watchObservedRunningTime="2026-06-11 21:42:11.928733696 +0000 UTC m=+1590.340057235" Jun 11 21:42:14.077100 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:14.077058 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:42:14.077750 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:14.077721 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:42:14.077750 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:14.077755 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:42:14.077945 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:14.077775 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:42:14.079235 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:14.079197 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Jun 11 21:42:14.079348 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:14.079239 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Jun 11 21:42:14.079348 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:14.079323 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.25:8816: connect: connection refused" Jun 11 21:42:14.079464 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:14.079392 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.25:6571: connect: connection refused" Jun 11 21:42:17.078095 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:17.078064 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:42:17.078553 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:17.078208 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:42:17.078709 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:17.078679 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:42:17.078832 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:17.078724 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:42:17.078832 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:17.078744 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:42:17.078832 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:17.078762 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:42:17.078980 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:17.078848 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:42:17.078980 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:17.078942 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:42:17.079183 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:17.079166 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:42:17.082682 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:17.082661 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:42:17.936215 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:17.936175 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:42:17.939303 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:42:17.939283 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:43:19.125038 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:19.125000 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs"] Jun 11 21:43:19.128888 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:19.128831 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="registry" containerID="cri-o://040804a5006df9efc49d870091566266b55c18f8963c4857fee6f7155cd1d7a2" gracePeriod=30 Jun 11 21:43:19.129049 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:19.128875 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="ui" containerID="cri-o://81082fd60a1765436b2f30cf027b0d3ad785b2d456896d9fb41165b5ff3170ec" gracePeriod=30 Jun 11 21:43:19.129193 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:19.128959 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="offline" containerID="cri-o://5b1a60d7a0731f6ae1724501ecc09979f2a1630fff25dbf605b65439121946a5" gracePeriod=30 Jun 11 21:43:19.129255 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:19.129224 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="online" containerID="cri-o://559d9bd8615bc09d7d409753831684ba3693d2fcfc9f43a6a42225e8d7da618c" gracePeriod=30 Jun 11 21:43:19.404403 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:19.404323 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6"] Jun 11 21:43:19.408367 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:19.408345 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:19.410730 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:19.410697 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-94rql\"" Jun 11 21:43:19.420324 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:19.420297 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6"] Jun 11 21:43:19.451392 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:19.451353 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/efb16dee-4971-458a-8cf4-6f95d4506c87-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-d94v6\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:19.451392 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:19.451393 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqv6m\" (UniqueName: \"kubernetes.io/projected/efb16dee-4971-458a-8cf4-6f95d4506c87-kube-api-access-pqv6m\") pod \"feast-simple-feast-setup-75c64c888b-d94v6\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:19.451668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:19.451423 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-d94v6\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:19.451668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:19.451537 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-d94v6\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:19.451668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:19.451609 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-d94v6\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:19.451766 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:19.451711 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-d94v6\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:19.552706 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:19.552659 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-d94v6\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:19.552908 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:19.552723 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-d94v6\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:19.552908 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:43:19.552847 2570 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Jun 11 21:43:19.552908 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:43:19.552890 2570 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Jun 11 21:43:19.553038 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:43:19.552932 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-registry-tls podName:efb16dee-4971-458a-8cf4-6f95d4506c87 nodeName:}" failed. No retries permitted until 2026-06-11 21:43:20.052912091 +0000 UTC m=+1658.464235624 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-registry-tls") pod "feast-simple-feast-setup-75c64c888b-d94v6" (UID: "efb16dee-4971-458a-8cf4-6f95d4506c87") : secret "feast-simple-feast-setup-registry-tls" not found Jun 11 21:43:19.553038 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:43:19.552951 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-ui-tls podName:efb16dee-4971-458a-8cf4-6f95d4506c87 nodeName:}" failed. No retries permitted until 2026-06-11 21:43:20.052943528 +0000 UTC m=+1658.464267062 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-ui-tls") pod "feast-simple-feast-setup-75c64c888b-d94v6" (UID: "efb16dee-4971-458a-8cf4-6f95d4506c87") : secret "feast-simple-feast-setup-ui-tls" not found Jun 11 21:43:19.553038 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:19.552973 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-d94v6\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:19.553196 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:19.553050 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-d94v6\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:19.553196 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:19.553095 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/efb16dee-4971-458a-8cf4-6f95d4506c87-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-d94v6\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:19.553196 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:19.553124 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqv6m\" (UniqueName: \"kubernetes.io/projected/efb16dee-4971-458a-8cf4-6f95d4506c87-kube-api-access-pqv6m\") pod \"feast-simple-feast-setup-75c64c888b-d94v6\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:19.553196 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:43:19.553168 2570 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Jun 11 21:43:19.553380 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:43:19.553230 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-online-tls podName:efb16dee-4971-458a-8cf4-6f95d4506c87 nodeName:}" failed. No retries permitted until 2026-06-11 21:43:20.053215796 +0000 UTC m=+1658.464539311 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-online-tls") pod "feast-simple-feast-setup-75c64c888b-d94v6" (UID: "efb16dee-4971-458a-8cf4-6f95d4506c87") : secret "feast-simple-feast-setup-online-tls" not found Jun 11 21:43:19.553523 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:19.553493 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/efb16dee-4971-458a-8cf4-6f95d4506c87-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-d94v6\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:19.555668 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:19.555638 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-d94v6\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:19.561287 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:19.561240 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqv6m\" (UniqueName: \"kubernetes.io/projected/efb16dee-4971-458a-8cf4-6f95d4506c87-kube-api-access-pqv6m\") pod \"feast-simple-feast-setup-75c64c888b-d94v6\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:20.056933 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:20.056852 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-d94v6\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:20.056933 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:20.056906 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-d94v6\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:20.056933 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:20.056932 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-d94v6\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:20.059305 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:20.059282 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-d94v6\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:20.059442 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:20.059402 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-d94v6\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:20.059503 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:20.059458 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-d94v6\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:20.167509 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:20.167477 2570 generic.go:356] "Generic (PLEG): container finished" podID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerID="81082fd60a1765436b2f30cf027b0d3ad785b2d456896d9fb41165b5ff3170ec" exitCode=0 Jun 11 21:43:20.167509 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:20.167502 2570 generic.go:356] "Generic (PLEG): container finished" podID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerID="559d9bd8615bc09d7d409753831684ba3693d2fcfc9f43a6a42225e8d7da618c" exitCode=0 Jun 11 21:43:20.167921 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:20.167548 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" event={"ID":"c54277cb-e73f-4c05-96eb-a03dcea6b1c2","Type":"ContainerDied","Data":"81082fd60a1765436b2f30cf027b0d3ad785b2d456896d9fb41165b5ff3170ec"} Jun 11 21:43:20.167921 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:20.167607 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" event={"ID":"c54277cb-e73f-4c05-96eb-a03dcea6b1c2","Type":"ContainerDied","Data":"559d9bd8615bc09d7d409753831684ba3693d2fcfc9f43a6a42225e8d7da618c"} Jun 11 21:43:20.317902 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:20.317810 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:20.449596 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:20.449554 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6"] Jun 11 21:43:20.450608 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:43:20.450585 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podefb16dee_4971_458a_8cf4_6f95d4506c87.slice/crio-8489db9870c538aa25a1bcb8aeb7a473e20fbe579878c3996db4480ab61bdcd2 WatchSource:0}: Error finding container 8489db9870c538aa25a1bcb8aeb7a473e20fbe579878c3996db4480ab61bdcd2: Status 404 returned error can't find the container with id 8489db9870c538aa25a1bcb8aeb7a473e20fbe579878c3996db4480ab61bdcd2 Jun 11 21:43:21.172884 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:21.172846 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" event={"ID":"efb16dee-4971-458a-8cf4-6f95d4506c87","Type":"ContainerStarted","Data":"30394d319ceb5d175fa4a2562774cb70c76842b14c9ea22f60bd7a6154bcbab5"} Jun 11 21:43:21.173339 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:21.172896 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" event={"ID":"efb16dee-4971-458a-8cf4-6f95d4506c87","Type":"ContainerStarted","Data":"8489db9870c538aa25a1bcb8aeb7a473e20fbe579878c3996db4480ab61bdcd2"} Jun 11 21:43:25.189514 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:25.189471 2570 generic.go:356] "Generic (PLEG): container finished" podID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerID="30394d319ceb5d175fa4a2562774cb70c76842b14c9ea22f60bd7a6154bcbab5" exitCode=0 Jun 11 21:43:25.189952 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:25.189541 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" event={"ID":"efb16dee-4971-458a-8cf4-6f95d4506c87","Type":"ContainerDied","Data":"30394d319ceb5d175fa4a2562774cb70c76842b14c9ea22f60bd7a6154bcbab5"} Jun 11 21:43:26.194876 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:26.194840 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" event={"ID":"efb16dee-4971-458a-8cf4-6f95d4506c87","Type":"ContainerStarted","Data":"0b0424feffada72a74d60b69f80fce26083cb23031a3a7edee28f4b30bd63543"} Jun 11 21:43:27.079403 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:27.079364 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Jun 11 21:43:27.937305 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:27.937255 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Jun 11 21:43:31.215530 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:31.215490 2570 generic.go:356] "Generic (PLEG): container finished" podID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerID="0b0424feffada72a74d60b69f80fce26083cb23031a3a7edee28f4b30bd63543" exitCode=0 Jun 11 21:43:31.215932 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:31.215583 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" event={"ID":"efb16dee-4971-458a-8cf4-6f95d4506c87","Type":"ContainerDied","Data":"0b0424feffada72a74d60b69f80fce26083cb23031a3a7edee28f4b30bd63543"} Jun 11 21:43:32.225737 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:32.225258 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" event={"ID":"efb16dee-4971-458a-8cf4-6f95d4506c87","Type":"ContainerStarted","Data":"5bb05d8974f2bd7773f71f977d818b1cf6fda3f22d1d7ce52428c11bfefe7ce7"} Jun 11 21:43:32.225737 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:32.225317 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" event={"ID":"efb16dee-4971-458a-8cf4-6f95d4506c87","Type":"ContainerStarted","Data":"ecac715d4bc140391e59df8efc4639b63901f25d364376b5bc9c87ba113057b3"} Jun 11 21:43:32.225737 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:32.225337 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" event={"ID":"efb16dee-4971-458a-8cf4-6f95d4506c87","Type":"ContainerStarted","Data":"cc531893fc178d3d447e09c3128799a43768748c562673e7655fc6ecc3abaf76"} Jun 11 21:43:32.225737 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:32.225355 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" event={"ID":"efb16dee-4971-458a-8cf4-6f95d4506c87","Type":"ContainerStarted","Data":"26d3fb7ee42b324f244e35d2929e944f7c620a3c78e8da76b3e1cbba7a19b6e8"} Jun 11 21:43:32.248885 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:32.248805 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" podStartSLOduration=13.2487849 podStartE2EDuration="13.2487849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:43:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:43:32.247746881 +0000 UTC m=+1670.659070445" watchObservedRunningTime="2026-06-11 21:43:32.2487849 +0000 UTC m=+1670.660108439" Jun 11 21:43:32.318723 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:32.318545 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:32.318723 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:32.318637 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:32.318723 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:32.318662 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:32.318723 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:32.318683 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:32.320913 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:32.320865 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.26:8816: connect: connection refused" Jun 11 21:43:32.321057 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:32.320914 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Jun 11 21:43:32.321057 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:32.320957 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.26:6571: connect: connection refused" Jun 11 21:43:32.321057 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:32.320914 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Jun 11 21:43:35.319169 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:35.319120 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.26:8816: connect: connection refused" Jun 11 21:43:35.319788 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:35.319134 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.26:6571: connect: connection refused" Jun 11 21:43:35.319921 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:35.319151 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Jun 11 21:43:35.319921 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:35.319197 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Jun 11 21:43:37.079774 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:37.079723 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Jun 11 21:43:37.937008 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:37.936972 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Jun 11 21:43:38.319912 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:38.319821 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:38.320360 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:38.320000 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:38.320360 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:38.320122 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:38.320546 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:38.320514 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:38.320702 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:38.320581 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:38.320702 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:38.320606 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:38.320894 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:38.320872 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:38.321001 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:38.320935 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:38.321057 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:38.321019 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:38.324319 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:38.324301 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:39.259785 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:39.259693 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:39.262859 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:39.262829 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:43:47.079469 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:47.079420 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Jun 11 21:43:47.079949 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:47.079598 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:43:47.937026 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:47.936989 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Jun 11 21:43:47.937209 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:47.937141 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:43:49.302049 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:49.302014 2570 generic.go:356] "Generic (PLEG): container finished" podID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerID="5b1a60d7a0731f6ae1724501ecc09979f2a1630fff25dbf605b65439121946a5" exitCode=137 Jun 11 21:43:49.302049 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:49.302040 2570 generic.go:356] "Generic (PLEG): container finished" podID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerID="040804a5006df9efc49d870091566266b55c18f8963c4857fee6f7155cd1d7a2" exitCode=137 Jun 11 21:43:49.302443 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:49.302080 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" event={"ID":"c54277cb-e73f-4c05-96eb-a03dcea6b1c2","Type":"ContainerDied","Data":"5b1a60d7a0731f6ae1724501ecc09979f2a1630fff25dbf605b65439121946a5"} Jun 11 21:43:49.302443 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:49.302120 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" event={"ID":"c54277cb-e73f-4c05-96eb-a03dcea6b1c2","Type":"ContainerDied","Data":"040804a5006df9efc49d870091566266b55c18f8963c4857fee6f7155cd1d7a2"} Jun 11 21:43:49.779340 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:49.779314 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:43:49.814473 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:49.814444 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-online-tls\") pod \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " Jun 11 21:43:49.814674 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:49.814481 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8gdf\" (UniqueName: \"kubernetes.io/projected/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-kube-api-access-d8gdf\") pod \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " Jun 11 21:43:49.814674 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:49.814534 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-ui-tls\") pod \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " Jun 11 21:43:49.814674 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:49.814581 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-feast-data\") pod \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " Jun 11 21:43:49.814674 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:49.814629 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-offline-tls\") pod \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " Jun 11 21:43:49.814674 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:49.814649 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-registry-tls\") pod \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\" (UID: \"c54277cb-e73f-4c05-96eb-a03dcea6b1c2\") " Jun 11 21:43:49.815222 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:49.815068 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-feast-data" (OuterVolumeSpecName: "feast-data") pod "c54277cb-e73f-4c05-96eb-a03dcea6b1c2" (UID: "c54277cb-e73f-4c05-96eb-a03dcea6b1c2"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 21:43:49.816961 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:49.816935 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c54277cb-e73f-4c05-96eb-a03dcea6b1c2" (UID: "c54277cb-e73f-4c05-96eb-a03dcea6b1c2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:43:49.817053 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:49.817006 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-kube-api-access-d8gdf" (OuterVolumeSpecName: "kube-api-access-d8gdf") pod "c54277cb-e73f-4c05-96eb-a03dcea6b1c2" (UID: "c54277cb-e73f-4c05-96eb-a03dcea6b1c2"). InnerVolumeSpecName "kube-api-access-d8gdf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 21:43:49.817185 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:49.817158 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "c54277cb-e73f-4c05-96eb-a03dcea6b1c2" (UID: "c54277cb-e73f-4c05-96eb-a03dcea6b1c2"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:43:49.817270 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:49.817219 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-online-tls" (OuterVolumeSpecName: "online-tls") pod "c54277cb-e73f-4c05-96eb-a03dcea6b1c2" (UID: "c54277cb-e73f-4c05-96eb-a03dcea6b1c2"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:43:49.817270 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:49.817243 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "c54277cb-e73f-4c05-96eb-a03dcea6b1c2" (UID: "c54277cb-e73f-4c05-96eb-a03dcea6b1c2"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:43:49.915480 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:49.915387 2570 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-ui-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:43:49.915480 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:49.915418 2570 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-feast-data\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:43:49.915480 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:49.915431 2570 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-offline-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:43:49.915480 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:49.915441 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-registry-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:43:49.915480 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:49.915453 2570 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-online-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:43:49.915480 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:49.915463 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8gdf\" (UniqueName: \"kubernetes.io/projected/c54277cb-e73f-4c05-96eb-a03dcea6b1c2-kube-api-access-d8gdf\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:43:50.308531 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:50.308445 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" event={"ID":"c54277cb-e73f-4c05-96eb-a03dcea6b1c2","Type":"ContainerDied","Data":"4bbab0c6327644b7e3bbf24419baadf33766e93998e7ebe7fc48476908849bfd"} Jun 11 21:43:50.308531 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:50.308510 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs" Jun 11 21:43:50.308969 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:50.308513 2570 scope.go:117] "RemoveContainer" containerID="81082fd60a1765436b2f30cf027b0d3ad785b2d456896d9fb41165b5ff3170ec" Jun 11 21:43:50.317043 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:50.317019 2570 scope.go:117] "RemoveContainer" containerID="5b1a60d7a0731f6ae1724501ecc09979f2a1630fff25dbf605b65439121946a5" Jun 11 21:43:50.324527 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:50.324508 2570 scope.go:117] "RemoveContainer" containerID="559d9bd8615bc09d7d409753831684ba3693d2fcfc9f43a6a42225e8d7da618c" Jun 11 21:43:50.327261 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:50.327234 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs"] Jun 11 21:43:50.331897 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:50.331873 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-2g4zs"] Jun 11 21:43:50.332464 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:50.332451 2570 scope.go:117] "RemoveContainer" containerID="040804a5006df9efc49d870091566266b55c18f8963c4857fee6f7155cd1d7a2" Jun 11 21:43:50.340209 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:50.340187 2570 scope.go:117] "RemoveContainer" containerID="bd82ca36e405c6d7efc302e0f933b2ca8f6e70b448b7a40b425c9dede30a781a" Jun 11 21:43:50.347306 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:50.347288 2570 scope.go:117] "RemoveContainer" containerID="fd26354197d6d813593ff254af21b98c5ac7c6dcffd47e6787d9fdff5d204f66" Jun 11 21:43:52.051455 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:43:52.051422 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" path="/var/lib/kubelet/pods/c54277cb-e73f-4c05-96eb-a03dcea6b1c2/volumes" Jun 11 21:44:40.878464 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.878429 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6"] Jun 11 21:44:40.878909 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.878781 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="registry" Jun 11 21:44:40.878909 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.878798 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="registry" Jun 11 21:44:40.878909 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.878808 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="online" Jun 11 21:44:40.878909 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.878814 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="online" Jun 11 21:44:40.878909 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.878821 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="ui" Jun 11 21:44:40.878909 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.878827 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="ui" Jun 11 21:44:40.878909 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.878834 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="feast-apply" Jun 11 21:44:40.878909 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.878840 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="feast-apply" Jun 11 21:44:40.878909 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.878846 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="offline" Jun 11 21:44:40.878909 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.878851 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="offline" Jun 11 21:44:40.878909 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.878862 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="feast-init" Jun 11 21:44:40.878909 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.878871 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="feast-init" Jun 11 21:44:40.879270 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.878924 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="registry" Jun 11 21:44:40.879270 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.878932 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="ui" Jun 11 21:44:40.879270 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.878942 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="offline" Jun 11 21:44:40.879270 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.878952 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c54277cb-e73f-4c05-96eb-a03dcea6b1c2" containerName="online" Jun 11 21:44:40.882069 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.882050 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:40.884651 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.884622 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Jun 11 21:44:40.884651 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.884644 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Jun 11 21:44:40.884651 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.884651 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Jun 11 21:44:40.884858 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.884633 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-926lb\"" Jun 11 21:44:40.885557 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.885496 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Jun 11 21:44:40.885557 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.885536 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Jun 11 21:44:40.885805 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.885789 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Jun 11 21:44:40.893316 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.893293 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6"] Jun 11 21:44:40.939754 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.939716 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-feast-data\") pod \"feast-simple-feast-remote-setup-669bcf45fc-gs9j6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:40.939936 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.939786 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnpwg\" (UniqueName: \"kubernetes.io/projected/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-kube-api-access-qnpwg\") pod \"feast-simple-feast-remote-setup-669bcf45fc-gs9j6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:40.939936 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.939866 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-registry-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-gs9j6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:40.939936 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.939904 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-gs9j6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:40.939936 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.939925 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-gs9j6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:40.940092 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:40.939942 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-gs9j6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:41.041290 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:41.041251 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-gs9j6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:41.041290 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:41.041289 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-gs9j6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:41.041500 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:41.041315 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-gs9j6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:41.041500 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:41.041353 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-feast-data\") pod \"feast-simple-feast-remote-setup-669bcf45fc-gs9j6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:41.041500 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:44:41.041409 2570 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found Jun 11 21:44:41.041500 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:44:41.041489 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-offline-tls podName:bbc23dc3-98c4-4934-8b17-c01e0a91ada6 nodeName:}" failed. No retries permitted until 2026-06-11 21:44:41.541472491 +0000 UTC m=+1739.952796026 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-offline-tls") pod "feast-simple-feast-remote-setup-669bcf45fc-gs9j6" (UID: "bbc23dc3-98c4-4934-8b17-c01e0a91ada6") : secret "feast-simple-feast-remote-setup-offline-tls" not found Jun 11 21:44:41.041756 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:41.041417 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnpwg\" (UniqueName: \"kubernetes.io/projected/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-kube-api-access-qnpwg\") pod \"feast-simple-feast-remote-setup-669bcf45fc-gs9j6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:41.041756 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:41.041616 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-registry-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-gs9j6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:41.041905 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:41.041885 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-feast-data\") pod \"feast-simple-feast-remote-setup-669bcf45fc-gs9j6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:41.042186 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:41.042168 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-registry-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-gs9j6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:41.043776 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:41.043754 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-gs9j6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:41.043881 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:41.043821 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-gs9j6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:41.052059 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:41.052035 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnpwg\" (UniqueName: \"kubernetes.io/projected/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-kube-api-access-qnpwg\") pod \"feast-simple-feast-remote-setup-669bcf45fc-gs9j6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:41.546031 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:41.545994 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-gs9j6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:41.548382 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:41.548355 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-gs9j6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:41.792426 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:41.792380 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:41.928151 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:41.928114 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6"] Jun 11 21:44:41.930853 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:44:41.930823 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbbc23dc3_98c4_4934_8b17_c01e0a91ada6.slice/crio-9229cbd0ea879ad14c3a4fca4c90aa738dc07de7b54518ba5d7cb832e7574c8e WatchSource:0}: Error finding container 9229cbd0ea879ad14c3a4fca4c90aa738dc07de7b54518ba5d7cb832e7574c8e: Status 404 returned error can't find the container with id 9229cbd0ea879ad14c3a4fca4c90aa738dc07de7b54518ba5d7cb832e7574c8e Jun 11 21:44:42.505943 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:42.505852 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" event={"ID":"bbc23dc3-98c4-4934-8b17-c01e0a91ada6","Type":"ContainerStarted","Data":"872af5d64aab4c1da01b637e8f7c00e6d4eb41a2e643f3d39910fbec0a47ac59"} Jun 11 21:44:42.505943 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:42.505907 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" event={"ID":"bbc23dc3-98c4-4934-8b17-c01e0a91ada6","Type":"ContainerStarted","Data":"9229cbd0ea879ad14c3a4fca4c90aa738dc07de7b54518ba5d7cb832e7574c8e"} Jun 11 21:44:47.528142 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:47.528109 2570 generic.go:356] "Generic (PLEG): container finished" podID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerID="872af5d64aab4c1da01b637e8f7c00e6d4eb41a2e643f3d39910fbec0a47ac59" exitCode=0 Jun 11 21:44:47.528603 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:47.528190 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" event={"ID":"bbc23dc3-98c4-4934-8b17-c01e0a91ada6","Type":"ContainerDied","Data":"872af5d64aab4c1da01b637e8f7c00e6d4eb41a2e643f3d39910fbec0a47ac59"} Jun 11 21:44:48.534613 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:48.534558 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" event={"ID":"bbc23dc3-98c4-4934-8b17-c01e0a91ada6","Type":"ContainerStarted","Data":"6d35eb864448f28685196a8b2b3f2782e56c4f555b19545d0c450355ec5def7c"} Jun 11 21:44:53.557324 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:53.557241 2570 generic.go:356] "Generic (PLEG): container finished" podID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerID="6d35eb864448f28685196a8b2b3f2782e56c4f555b19545d0c450355ec5def7c" exitCode=0 Jun 11 21:44:53.557324 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:53.557281 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" event={"ID":"bbc23dc3-98c4-4934-8b17-c01e0a91ada6","Type":"ContainerDied","Data":"6d35eb864448f28685196a8b2b3f2782e56c4f555b19545d0c450355ec5def7c"} Jun 11 21:44:54.566594 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:54.566541 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" event={"ID":"bbc23dc3-98c4-4934-8b17-c01e0a91ada6","Type":"ContainerStarted","Data":"5130edb175eccc73a097947c84cc33e0396387d30209cfaa114cb52da5046cf1"} Jun 11 21:44:54.566978 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:54.566619 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" event={"ID":"bbc23dc3-98c4-4934-8b17-c01e0a91ada6","Type":"ContainerStarted","Data":"79e3aecda2780e6b84ef775501c5a82cd69e8bc02bc3564fe4b305c3663ebc1c"} Jun 11 21:44:54.566978 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:54.566640 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" event={"ID":"bbc23dc3-98c4-4934-8b17-c01e0a91ada6","Type":"ContainerStarted","Data":"90f99b14915148cf1afc639f139bacd50ed00d23441857d0085e7d42bf90395a"} Jun 11 21:44:54.587737 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:54.587667 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" podStartSLOduration=14.587646965 podStartE2EDuration="14.587646965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 21:44:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:44:54.586284292 +0000 UTC m=+1752.997607830" watchObservedRunningTime="2026-06-11 21:44:54.587646965 +0000 UTC m=+1752.998970504" Jun 11 21:44:56.793168 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:56.793129 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:56.793734 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:56.793710 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:56.793734 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:56.793739 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:56.794875 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:56.794846 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused" Jun 11 21:44:56.794989 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:56.794878 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.27:8816: connect: connection refused" Jun 11 21:44:56.794989 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:56.794885 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Jun 11 21:44:59.794278 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:59.794245 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:59.794681 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:59.794462 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:59.794789 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:59.794764 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:59.794833 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:59.794812 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:59.795227 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:59.795206 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:59.795336 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:59.795292 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:44:59.798183 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:44:59.798167 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:45:00.594816 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:45:00.594786 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:45:00.597697 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:45:00.597674 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:45:42.126907 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:45:42.126875 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/ovn-acl-logging/0.log" Jun 11 21:45:42.130983 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:45:42.130963 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/ovn-acl-logging/0.log" Jun 11 21:46:01.877842 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:01.877811 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6"] Jun 11 21:46:01.878348 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:01.878114 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="online" containerID="cri-o://90f99b14915148cf1afc639f139bacd50ed00d23441857d0085e7d42bf90395a" gracePeriod=30 Jun 11 21:46:01.878348 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:01.878171 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="ui" containerID="cri-o://5130edb175eccc73a097947c84cc33e0396387d30209cfaa114cb52da5046cf1" gracePeriod=30 Jun 11 21:46:01.878348 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:01.878279 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="offline" containerID="cri-o://79e3aecda2780e6b84ef775501c5a82cd69e8bc02bc3564fe4b305c3663ebc1c" gracePeriod=30 Jun 11 21:46:02.042244 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:02.042212 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6"] Jun 11 21:46:02.042519 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:02.042477 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="registry" containerID="cri-o://26d3fb7ee42b324f244e35d2929e944f7c620a3c78e8da76b3e1cbba7a19b6e8" gracePeriod=30 Jun 11 21:46:02.042647 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:02.042598 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="ui" containerID="cri-o://5bb05d8974f2bd7773f71f977d818b1cf6fda3f22d1d7ce52428c11bfefe7ce7" gracePeriod=30 Jun 11 21:46:02.042647 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:02.042609 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="offline" containerID="cri-o://ecac715d4bc140391e59df8efc4639b63901f25d364376b5bc9c87ba113057b3" gracePeriod=30 Jun 11 21:46:02.042752 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:02.042599 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="online" containerID="cri-o://cc531893fc178d3d447e09c3128799a43768748c562673e7655fc6ecc3abaf76" gracePeriod=30 Jun 11 21:46:02.847316 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:02.847280 2570 generic.go:356] "Generic (PLEG): container finished" podID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerID="5bb05d8974f2bd7773f71f977d818b1cf6fda3f22d1d7ce52428c11bfefe7ce7" exitCode=0 Jun 11 21:46:02.847494 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:02.847344 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" event={"ID":"efb16dee-4971-458a-8cf4-6f95d4506c87","Type":"ContainerDied","Data":"5bb05d8974f2bd7773f71f977d818b1cf6fda3f22d1d7ce52428c11bfefe7ce7"} Jun 11 21:46:02.850211 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:02.850185 2570 generic.go:356] "Generic (PLEG): container finished" podID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerID="5130edb175eccc73a097947c84cc33e0396387d30209cfaa114cb52da5046cf1" exitCode=0 Jun 11 21:46:02.850211 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:02.850207 2570 generic.go:356] "Generic (PLEG): container finished" podID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerID="90f99b14915148cf1afc639f139bacd50ed00d23441857d0085e7d42bf90395a" exitCode=0 Jun 11 21:46:02.850367 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:02.850250 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" event={"ID":"bbc23dc3-98c4-4934-8b17-c01e0a91ada6","Type":"ContainerDied","Data":"5130edb175eccc73a097947c84cc33e0396387d30209cfaa114cb52da5046cf1"} Jun 11 21:46:02.850367 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:02.850282 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" event={"ID":"bbc23dc3-98c4-4934-8b17-c01e0a91ada6","Type":"ContainerDied","Data":"90f99b14915148cf1afc639f139bacd50ed00d23441857d0085e7d42bf90395a"} Jun 11 21:46:03.856868 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:03.856832 2570 generic.go:356] "Generic (PLEG): container finished" podID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerID="cc531893fc178d3d447e09c3128799a43768748c562673e7655fc6ecc3abaf76" exitCode=0 Jun 11 21:46:03.857227 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:03.856902 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" event={"ID":"efb16dee-4971-458a-8cf4-6f95d4506c87","Type":"ContainerDied","Data":"cc531893fc178d3d447e09c3128799a43768748c562673e7655fc6ecc3abaf76"} Jun 11 21:46:08.321350 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:08.321299 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Jun 11 21:46:09.260261 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:09.260214 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Jun 11 21:46:09.795139 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:09.795090 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused" Jun 11 21:46:10.595448 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:10.595405 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Jun 11 21:46:18.320873 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:18.320827 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Jun 11 21:46:19.260513 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:19.260474 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Jun 11 21:46:19.795041 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:19.794991 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused" Jun 11 21:46:20.595199 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:20.595141 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Jun 11 21:46:28.320807 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:28.320757 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Jun 11 21:46:28.321240 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:28.320905 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:46:29.260672 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:29.260633 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Jun 11 21:46:29.260859 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:29.260749 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:46:29.795446 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:29.795400 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused" Jun 11 21:46:29.795890 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:29.795546 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:46:30.595860 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:30.595814 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused" Jun 11 21:46:30.596071 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:30.595947 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:46:31.970550 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:31.970520 2570 generic.go:356] "Generic (PLEG): container finished" podID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerID="79e3aecda2780e6b84ef775501c5a82cd69e8bc02bc3564fe4b305c3663ebc1c" exitCode=137 Jun 11 21:46:31.971044 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:31.970606 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" event={"ID":"bbc23dc3-98c4-4934-8b17-c01e0a91ada6","Type":"ContainerDied","Data":"79e3aecda2780e6b84ef775501c5a82cd69e8bc02bc3564fe4b305c3663ebc1c"} Jun 11 21:46:32.076962 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.076942 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:46:32.166472 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.166374 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-feast-data\") pod \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " Jun 11 21:46:32.166472 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.166423 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-registry-tls\") pod \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " Jun 11 21:46:32.166472 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.166455 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-ui-tls\") pod \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " Jun 11 21:46:32.166783 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.166488 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-online-tls\") pod \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " Jun 11 21:46:32.166783 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.166507 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnpwg\" (UniqueName: \"kubernetes.io/projected/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-kube-api-access-qnpwg\") pod \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " Jun 11 21:46:32.166783 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.166525 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-offline-tls\") pod \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\" (UID: \"bbc23dc3-98c4-4934-8b17-c01e0a91ada6\") " Jun 11 21:46:32.166991 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.166857 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bbc23dc3-98c4-4934-8b17-c01e0a91ada6" (UID: "bbc23dc3-98c4-4934-8b17-c01e0a91ada6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 21:46:32.167037 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.166981 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-feast-data" (OuterVolumeSpecName: "feast-data") pod "bbc23dc3-98c4-4934-8b17-c01e0a91ada6" (UID: "bbc23dc3-98c4-4934-8b17-c01e0a91ada6"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 21:46:32.168777 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.168753 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "bbc23dc3-98c4-4934-8b17-c01e0a91ada6" (UID: "bbc23dc3-98c4-4934-8b17-c01e0a91ada6"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:46:32.168887 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.168789 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-online-tls" (OuterVolumeSpecName: "online-tls") pod "bbc23dc3-98c4-4934-8b17-c01e0a91ada6" (UID: "bbc23dc3-98c4-4934-8b17-c01e0a91ada6"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:46:32.169074 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.169057 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "bbc23dc3-98c4-4934-8b17-c01e0a91ada6" (UID: "bbc23dc3-98c4-4934-8b17-c01e0a91ada6"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:46:32.169074 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.169058 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-kube-api-access-qnpwg" (OuterVolumeSpecName: "kube-api-access-qnpwg") pod "bbc23dc3-98c4-4934-8b17-c01e0a91ada6" (UID: "bbc23dc3-98c4-4934-8b17-c01e0a91ada6"). InnerVolumeSpecName "kube-api-access-qnpwg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 21:46:32.267224 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.267190 2570 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-ui-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:46:32.267224 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.267223 2570 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-online-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:46:32.267419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.267239 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnpwg\" (UniqueName: \"kubernetes.io/projected/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-kube-api-access-qnpwg\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:46:32.267419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.267252 2570 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-offline-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:46:32.267419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.267261 2570 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-feast-data\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:46:32.267419 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.267269 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/bbc23dc3-98c4-4934-8b17-c01e0a91ada6-registry-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:46:32.685354 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.685324 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:46:32.770750 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.770719 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-ui-tls\") pod \"efb16dee-4971-458a-8cf4-6f95d4506c87\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " Jun 11 21:46:32.770920 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.770774 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-offline-tls\") pod \"efb16dee-4971-458a-8cf4-6f95d4506c87\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " Jun 11 21:46:32.770920 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.770794 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/efb16dee-4971-458a-8cf4-6f95d4506c87-feast-data\") pod \"efb16dee-4971-458a-8cf4-6f95d4506c87\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " Jun 11 21:46:32.770920 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.770817 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-online-tls\") pod \"efb16dee-4971-458a-8cf4-6f95d4506c87\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " Jun 11 21:46:32.770920 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.770846 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqv6m\" (UniqueName: \"kubernetes.io/projected/efb16dee-4971-458a-8cf4-6f95d4506c87-kube-api-access-pqv6m\") pod \"efb16dee-4971-458a-8cf4-6f95d4506c87\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " Jun 11 21:46:32.770920 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.770879 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-registry-tls\") pod \"efb16dee-4971-458a-8cf4-6f95d4506c87\" (UID: \"efb16dee-4971-458a-8cf4-6f95d4506c87\") " Jun 11 21:46:32.771391 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.771362 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efb16dee-4971-458a-8cf4-6f95d4506c87-feast-data" (OuterVolumeSpecName: "feast-data") pod "efb16dee-4971-458a-8cf4-6f95d4506c87" (UID: "efb16dee-4971-458a-8cf4-6f95d4506c87"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 21:46:32.773104 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.773073 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efb16dee-4971-458a-8cf4-6f95d4506c87-kube-api-access-pqv6m" (OuterVolumeSpecName: "kube-api-access-pqv6m") pod "efb16dee-4971-458a-8cf4-6f95d4506c87" (UID: "efb16dee-4971-458a-8cf4-6f95d4506c87"). InnerVolumeSpecName "kube-api-access-pqv6m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 21:46:32.773366 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.773351 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "efb16dee-4971-458a-8cf4-6f95d4506c87" (UID: "efb16dee-4971-458a-8cf4-6f95d4506c87"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:46:32.773431 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.773375 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-online-tls" (OuterVolumeSpecName: "online-tls") pod "efb16dee-4971-458a-8cf4-6f95d4506c87" (UID: "efb16dee-4971-458a-8cf4-6f95d4506c87"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:46:32.773484 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.773447 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "efb16dee-4971-458a-8cf4-6f95d4506c87" (UID: "efb16dee-4971-458a-8cf4-6f95d4506c87"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:46:32.773484 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.773464 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "efb16dee-4971-458a-8cf4-6f95d4506c87" (UID: "efb16dee-4971-458a-8cf4-6f95d4506c87"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 21:46:32.872026 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.871992 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-registry-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:46:32.872026 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.872023 2570 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-ui-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:46:32.872026 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.872031 2570 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-offline-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:46:32.872299 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.872040 2570 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/efb16dee-4971-458a-8cf4-6f95d4506c87-feast-data\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:46:32.872299 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.872051 2570 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/efb16dee-4971-458a-8cf4-6f95d4506c87-online-tls\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:46:32.872299 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.872060 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqv6m\" (UniqueName: \"kubernetes.io/projected/efb16dee-4971-458a-8cf4-6f95d4506c87-kube-api-access-pqv6m\") on node \"ip-10-0-143-6.ec2.internal\" DevicePath \"\"" Jun 11 21:46:32.977064 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.977029 2570 generic.go:356] "Generic (PLEG): container finished" podID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerID="ecac715d4bc140391e59df8efc4639b63901f25d364376b5bc9c87ba113057b3" exitCode=137 Jun 11 21:46:32.977064 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.977060 2570 generic.go:356] "Generic (PLEG): container finished" podID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerID="26d3fb7ee42b324f244e35d2929e944f7c620a3c78e8da76b3e1cbba7a19b6e8" exitCode=137 Jun 11 21:46:32.977518 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.977108 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" Jun 11 21:46:32.977518 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.977120 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" event={"ID":"efb16dee-4971-458a-8cf4-6f95d4506c87","Type":"ContainerDied","Data":"ecac715d4bc140391e59df8efc4639b63901f25d364376b5bc9c87ba113057b3"} Jun 11 21:46:32.977518 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.977166 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" event={"ID":"efb16dee-4971-458a-8cf4-6f95d4506c87","Type":"ContainerDied","Data":"26d3fb7ee42b324f244e35d2929e944f7c620a3c78e8da76b3e1cbba7a19b6e8"} Jun 11 21:46:32.977518 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.977180 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6" event={"ID":"efb16dee-4971-458a-8cf4-6f95d4506c87","Type":"ContainerDied","Data":"8489db9870c538aa25a1bcb8aeb7a473e20fbe579878c3996db4480ab61bdcd2"} Jun 11 21:46:32.977518 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.977200 2570 scope.go:117] "RemoveContainer" containerID="5bb05d8974f2bd7773f71f977d818b1cf6fda3f22d1d7ce52428c11bfefe7ce7" Jun 11 21:46:32.979876 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.979856 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" event={"ID":"bbc23dc3-98c4-4934-8b17-c01e0a91ada6","Type":"ContainerDied","Data":"9229cbd0ea879ad14c3a4fca4c90aa738dc07de7b54518ba5d7cb832e7574c8e"} Jun 11 21:46:32.979973 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.979927 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6" Jun 11 21:46:32.989726 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.989472 2570 scope.go:117] "RemoveContainer" containerID="ecac715d4bc140391e59df8efc4639b63901f25d364376b5bc9c87ba113057b3" Jun 11 21:46:32.999963 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:32.999945 2570 scope.go:117] "RemoveContainer" containerID="cc531893fc178d3d447e09c3128799a43768748c562673e7655fc6ecc3abaf76" Jun 11 21:46:33.003829 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.003807 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6"] Jun 11 21:46:33.006555 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.006532 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-d94v6"] Jun 11 21:46:33.008168 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.008152 2570 scope.go:117] "RemoveContainer" containerID="26d3fb7ee42b324f244e35d2929e944f7c620a3c78e8da76b3e1cbba7a19b6e8" Jun 11 21:46:33.015400 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.015380 2570 scope.go:117] "RemoveContainer" containerID="0b0424feffada72a74d60b69f80fce26083cb23031a3a7edee28f4b30bd63543" Jun 11 21:46:33.017665 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.017645 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6"] Jun 11 21:46:33.021305 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.021285 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-gs9j6"] Jun 11 21:46:33.023335 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.023316 2570 scope.go:117] "RemoveContainer" containerID="30394d319ceb5d175fa4a2562774cb70c76842b14c9ea22f60bd7a6154bcbab5" Jun 11 21:46:33.030821 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.030806 2570 scope.go:117] "RemoveContainer" containerID="5bb05d8974f2bd7773f71f977d818b1cf6fda3f22d1d7ce52428c11bfefe7ce7" Jun 11 21:46:33.031069 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:46:33.031050 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bb05d8974f2bd7773f71f977d818b1cf6fda3f22d1d7ce52428c11bfefe7ce7\": container with ID starting with 5bb05d8974f2bd7773f71f977d818b1cf6fda3f22d1d7ce52428c11bfefe7ce7 not found: ID does not exist" containerID="5bb05d8974f2bd7773f71f977d818b1cf6fda3f22d1d7ce52428c11bfefe7ce7" Jun 11 21:46:33.031123 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.031079 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bb05d8974f2bd7773f71f977d818b1cf6fda3f22d1d7ce52428c11bfefe7ce7"} err="failed to get container status \"5bb05d8974f2bd7773f71f977d818b1cf6fda3f22d1d7ce52428c11bfefe7ce7\": rpc error: code = NotFound desc = could not find container \"5bb05d8974f2bd7773f71f977d818b1cf6fda3f22d1d7ce52428c11bfefe7ce7\": container with ID starting with 5bb05d8974f2bd7773f71f977d818b1cf6fda3f22d1d7ce52428c11bfefe7ce7 not found: ID does not exist" Jun 11 21:46:33.031123 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.031097 2570 scope.go:117] "RemoveContainer" containerID="ecac715d4bc140391e59df8efc4639b63901f25d364376b5bc9c87ba113057b3" Jun 11 21:46:33.031353 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:46:33.031333 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecac715d4bc140391e59df8efc4639b63901f25d364376b5bc9c87ba113057b3\": container with ID starting with ecac715d4bc140391e59df8efc4639b63901f25d364376b5bc9c87ba113057b3 not found: ID does not exist" containerID="ecac715d4bc140391e59df8efc4639b63901f25d364376b5bc9c87ba113057b3" Jun 11 21:46:33.031396 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.031361 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecac715d4bc140391e59df8efc4639b63901f25d364376b5bc9c87ba113057b3"} err="failed to get container status \"ecac715d4bc140391e59df8efc4639b63901f25d364376b5bc9c87ba113057b3\": rpc error: code = NotFound desc = could not find container \"ecac715d4bc140391e59df8efc4639b63901f25d364376b5bc9c87ba113057b3\": container with ID starting with ecac715d4bc140391e59df8efc4639b63901f25d364376b5bc9c87ba113057b3 not found: ID does not exist" Jun 11 21:46:33.031396 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.031378 2570 scope.go:117] "RemoveContainer" containerID="cc531893fc178d3d447e09c3128799a43768748c562673e7655fc6ecc3abaf76" Jun 11 21:46:33.031598 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:46:33.031576 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc531893fc178d3d447e09c3128799a43768748c562673e7655fc6ecc3abaf76\": container with ID starting with cc531893fc178d3d447e09c3128799a43768748c562673e7655fc6ecc3abaf76 not found: ID does not exist" containerID="cc531893fc178d3d447e09c3128799a43768748c562673e7655fc6ecc3abaf76" Jun 11 21:46:33.031679 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.031605 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc531893fc178d3d447e09c3128799a43768748c562673e7655fc6ecc3abaf76"} err="failed to get container status \"cc531893fc178d3d447e09c3128799a43768748c562673e7655fc6ecc3abaf76\": rpc error: code = NotFound desc = could not find container \"cc531893fc178d3d447e09c3128799a43768748c562673e7655fc6ecc3abaf76\": container with ID starting with cc531893fc178d3d447e09c3128799a43768748c562673e7655fc6ecc3abaf76 not found: ID does not exist" Jun 11 21:46:33.031679 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.031626 2570 scope.go:117] "RemoveContainer" containerID="26d3fb7ee42b324f244e35d2929e944f7c620a3c78e8da76b3e1cbba7a19b6e8" Jun 11 21:46:33.031856 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:46:33.031840 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26d3fb7ee42b324f244e35d2929e944f7c620a3c78e8da76b3e1cbba7a19b6e8\": container with ID starting with 26d3fb7ee42b324f244e35d2929e944f7c620a3c78e8da76b3e1cbba7a19b6e8 not found: ID does not exist" containerID="26d3fb7ee42b324f244e35d2929e944f7c620a3c78e8da76b3e1cbba7a19b6e8" Jun 11 21:46:33.031894 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.031861 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26d3fb7ee42b324f244e35d2929e944f7c620a3c78e8da76b3e1cbba7a19b6e8"} err="failed to get container status \"26d3fb7ee42b324f244e35d2929e944f7c620a3c78e8da76b3e1cbba7a19b6e8\": rpc error: code = NotFound desc = could not find container \"26d3fb7ee42b324f244e35d2929e944f7c620a3c78e8da76b3e1cbba7a19b6e8\": container with ID starting with 26d3fb7ee42b324f244e35d2929e944f7c620a3c78e8da76b3e1cbba7a19b6e8 not found: ID does not exist" Jun 11 21:46:33.031894 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.031875 2570 scope.go:117] "RemoveContainer" containerID="0b0424feffada72a74d60b69f80fce26083cb23031a3a7edee28f4b30bd63543" Jun 11 21:46:33.032096 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:46:33.032079 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b0424feffada72a74d60b69f80fce26083cb23031a3a7edee28f4b30bd63543\": container with ID starting with 0b0424feffada72a74d60b69f80fce26083cb23031a3a7edee28f4b30bd63543 not found: ID does not exist" containerID="0b0424feffada72a74d60b69f80fce26083cb23031a3a7edee28f4b30bd63543" Jun 11 21:46:33.032163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.032103 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b0424feffada72a74d60b69f80fce26083cb23031a3a7edee28f4b30bd63543"} err="failed to get container status \"0b0424feffada72a74d60b69f80fce26083cb23031a3a7edee28f4b30bd63543\": rpc error: code = NotFound desc = could not find container \"0b0424feffada72a74d60b69f80fce26083cb23031a3a7edee28f4b30bd63543\": container with ID starting with 0b0424feffada72a74d60b69f80fce26083cb23031a3a7edee28f4b30bd63543 not found: ID does not exist" Jun 11 21:46:33.032163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.032123 2570 scope.go:117] "RemoveContainer" containerID="30394d319ceb5d175fa4a2562774cb70c76842b14c9ea22f60bd7a6154bcbab5" Jun 11 21:46:33.032344 ip-10-0-143-6 kubenswrapper[2570]: E0611 21:46:33.032328 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30394d319ceb5d175fa4a2562774cb70c76842b14c9ea22f60bd7a6154bcbab5\": container with ID starting with 30394d319ceb5d175fa4a2562774cb70c76842b14c9ea22f60bd7a6154bcbab5 not found: ID does not exist" containerID="30394d319ceb5d175fa4a2562774cb70c76842b14c9ea22f60bd7a6154bcbab5" Jun 11 21:46:33.032382 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.032348 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30394d319ceb5d175fa4a2562774cb70c76842b14c9ea22f60bd7a6154bcbab5"} err="failed to get container status \"30394d319ceb5d175fa4a2562774cb70c76842b14c9ea22f60bd7a6154bcbab5\": rpc error: code = NotFound desc = could not find container \"30394d319ceb5d175fa4a2562774cb70c76842b14c9ea22f60bd7a6154bcbab5\": container with ID starting with 30394d319ceb5d175fa4a2562774cb70c76842b14c9ea22f60bd7a6154bcbab5 not found: ID does not exist" Jun 11 21:46:33.032382 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.032361 2570 scope.go:117] "RemoveContainer" containerID="5bb05d8974f2bd7773f71f977d818b1cf6fda3f22d1d7ce52428c11bfefe7ce7" Jun 11 21:46:33.032546 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.032531 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bb05d8974f2bd7773f71f977d818b1cf6fda3f22d1d7ce52428c11bfefe7ce7"} err="failed to get container status \"5bb05d8974f2bd7773f71f977d818b1cf6fda3f22d1d7ce52428c11bfefe7ce7\": rpc error: code = NotFound desc = could not find container \"5bb05d8974f2bd7773f71f977d818b1cf6fda3f22d1d7ce52428c11bfefe7ce7\": container with ID starting with 5bb05d8974f2bd7773f71f977d818b1cf6fda3f22d1d7ce52428c11bfefe7ce7 not found: ID does not exist" Jun 11 21:46:33.032620 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.032545 2570 scope.go:117] "RemoveContainer" containerID="ecac715d4bc140391e59df8efc4639b63901f25d364376b5bc9c87ba113057b3" Jun 11 21:46:33.032786 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.032767 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecac715d4bc140391e59df8efc4639b63901f25d364376b5bc9c87ba113057b3"} err="failed to get container status \"ecac715d4bc140391e59df8efc4639b63901f25d364376b5bc9c87ba113057b3\": rpc error: code = NotFound desc = could not find container \"ecac715d4bc140391e59df8efc4639b63901f25d364376b5bc9c87ba113057b3\": container with ID starting with ecac715d4bc140391e59df8efc4639b63901f25d364376b5bc9c87ba113057b3 not found: ID does not exist" Jun 11 21:46:33.032829 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.032787 2570 scope.go:117] "RemoveContainer" containerID="cc531893fc178d3d447e09c3128799a43768748c562673e7655fc6ecc3abaf76" Jun 11 21:46:33.032960 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.032943 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc531893fc178d3d447e09c3128799a43768748c562673e7655fc6ecc3abaf76"} err="failed to get container status \"cc531893fc178d3d447e09c3128799a43768748c562673e7655fc6ecc3abaf76\": rpc error: code = NotFound desc = could not find container \"cc531893fc178d3d447e09c3128799a43768748c562673e7655fc6ecc3abaf76\": container with ID starting with cc531893fc178d3d447e09c3128799a43768748c562673e7655fc6ecc3abaf76 not found: ID does not exist" Jun 11 21:46:33.033009 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.032961 2570 scope.go:117] "RemoveContainer" containerID="26d3fb7ee42b324f244e35d2929e944f7c620a3c78e8da76b3e1cbba7a19b6e8" Jun 11 21:46:33.033111 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.033096 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26d3fb7ee42b324f244e35d2929e944f7c620a3c78e8da76b3e1cbba7a19b6e8"} err="failed to get container status \"26d3fb7ee42b324f244e35d2929e944f7c620a3c78e8da76b3e1cbba7a19b6e8\": rpc error: code = NotFound desc = could not find container \"26d3fb7ee42b324f244e35d2929e944f7c620a3c78e8da76b3e1cbba7a19b6e8\": container with ID starting with 26d3fb7ee42b324f244e35d2929e944f7c620a3c78e8da76b3e1cbba7a19b6e8 not found: ID does not exist" Jun 11 21:46:33.033152 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.033110 2570 scope.go:117] "RemoveContainer" containerID="0b0424feffada72a74d60b69f80fce26083cb23031a3a7edee28f4b30bd63543" Jun 11 21:46:33.033303 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.033286 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b0424feffada72a74d60b69f80fce26083cb23031a3a7edee28f4b30bd63543"} err="failed to get container status \"0b0424feffada72a74d60b69f80fce26083cb23031a3a7edee28f4b30bd63543\": rpc error: code = NotFound desc = could not find container \"0b0424feffada72a74d60b69f80fce26083cb23031a3a7edee28f4b30bd63543\": container with ID starting with 0b0424feffada72a74d60b69f80fce26083cb23031a3a7edee28f4b30bd63543 not found: ID does not exist" Jun 11 21:46:33.033345 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.033302 2570 scope.go:117] "RemoveContainer" containerID="30394d319ceb5d175fa4a2562774cb70c76842b14c9ea22f60bd7a6154bcbab5" Jun 11 21:46:33.033465 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.033450 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30394d319ceb5d175fa4a2562774cb70c76842b14c9ea22f60bd7a6154bcbab5"} err="failed to get container status \"30394d319ceb5d175fa4a2562774cb70c76842b14c9ea22f60bd7a6154bcbab5\": rpc error: code = NotFound desc = could not find container \"30394d319ceb5d175fa4a2562774cb70c76842b14c9ea22f60bd7a6154bcbab5\": container with ID starting with 30394d319ceb5d175fa4a2562774cb70c76842b14c9ea22f60bd7a6154bcbab5 not found: ID does not exist" Jun 11 21:46:33.033505 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.033465 2570 scope.go:117] "RemoveContainer" containerID="5130edb175eccc73a097947c84cc33e0396387d30209cfaa114cb52da5046cf1" Jun 11 21:46:33.040688 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.040670 2570 scope.go:117] "RemoveContainer" containerID="79e3aecda2780e6b84ef775501c5a82cd69e8bc02bc3564fe4b305c3663ebc1c" Jun 11 21:46:33.048146 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.048131 2570 scope.go:117] "RemoveContainer" containerID="90f99b14915148cf1afc639f139bacd50ed00d23441857d0085e7d42bf90395a" Jun 11 21:46:33.054942 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.054911 2570 scope.go:117] "RemoveContainer" containerID="6d35eb864448f28685196a8b2b3f2782e56c4f555b19545d0c450355ec5def7c" Jun 11 21:46:33.061854 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:33.061829 2570 scope.go:117] "RemoveContainer" containerID="872af5d64aab4c1da01b637e8f7c00e6d4eb41a2e643f3d39910fbec0a47ac59" Jun 11 21:46:34.051449 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:34.051416 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" path="/var/lib/kubelet/pods/bbc23dc3-98c4-4934-8b17-c01e0a91ada6/volumes" Jun 11 21:46:34.051988 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:46:34.051973 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" path="/var/lib/kubelet/pods/efb16dee-4971-458a-8cf4-6f95d4506c87/volumes" Jun 11 21:47:11.935050 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:11.934975 2570 ???:1] "http: TLS handshake error from 10.0.142.82:39764: EOF" Jun 11 21:47:11.938937 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:11.938913 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4c247_6f4a1fd8-505b-4061-a432-620cb3c68386/global-pull-secret-syncer/0.log" Jun 11 21:47:12.154271 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:12.154242 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-x65dp_b641e893-d298-4942-8ccb-6145ef84bf81/konnectivity-agent/0.log" Jun 11 21:47:12.222187 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:12.222103 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-6.ec2.internal_f9fdf41d985ef036011199dc3759ac96/haproxy/0.log" Jun 11 21:47:15.930521 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:15.930495 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a22dba60-3268-4a0a-a94d-d03c33d5b38a/alertmanager/0.log" Jun 11 21:47:15.968327 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:15.968299 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a22dba60-3268-4a0a-a94d-d03c33d5b38a/config-reloader/0.log" Jun 11 21:47:16.002436 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:16.002412 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a22dba60-3268-4a0a-a94d-d03c33d5b38a/kube-rbac-proxy-web/0.log" Jun 11 21:47:16.035461 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:16.035433 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a22dba60-3268-4a0a-a94d-d03c33d5b38a/kube-rbac-proxy/0.log" Jun 11 21:47:16.064536 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:16.064511 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a22dba60-3268-4a0a-a94d-d03c33d5b38a/kube-rbac-proxy-metric/0.log" Jun 11 21:47:16.097445 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:16.097419 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a22dba60-3268-4a0a-a94d-d03c33d5b38a/prom-label-proxy/0.log" Jun 11 21:47:16.127282 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:16.127258 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a22dba60-3268-4a0a-a94d-d03c33d5b38a/init-config-reloader/0.log" Jun 11 21:47:16.275217 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:16.275136 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7d868dd696-hhxbl_61bce198-7787-4f9d-83a4-75133afaca31/metrics-server/0.log" Jun 11 21:47:16.307034 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:16.307006 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-589576dfc7-bplr4_e630b187-62f5-4152-9b03-887853c93376/monitoring-plugin/0.log" Jun 11 21:47:16.517023 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:16.516999 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mpqmj_6aac3ed7-198f-44c0-934f-f0606ffb7955/node-exporter/0.log" Jun 11 21:47:16.545510 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:16.545441 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mpqmj_6aac3ed7-198f-44c0-934f-f0606ffb7955/kube-rbac-proxy/0.log" Jun 11 21:47:16.574044 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:16.574023 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mpqmj_6aac3ed7-198f-44c0-934f-f0606ffb7955/init-textfile/0.log" Jun 11 21:47:16.918670 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:16.918640 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5cf667749c-7gxwt_9d605903-7c76-48d9-9a1a-439b29742598/prometheus-operator-admission-webhook/0.log" Jun 11 21:47:19.282112 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.282079 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-74f5576b77-cplfp_cea20d06-45c0-4aa9-811d-4e80e2285568/console/0.log" Jun 11 21:47:19.569769 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.569694 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8"] Jun 11 21:47:19.569964 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.569952 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="feast-apply" Jun 11 21:47:19.570004 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.569966 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="feast-apply" Jun 11 21:47:19.570004 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.569978 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="ui" Jun 11 21:47:19.570004 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.569983 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="ui" Jun 11 21:47:19.570004 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.569996 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="offline" Jun 11 21:47:19.570004 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.570003 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="offline" Jun 11 21:47:19.570163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.570011 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="registry" Jun 11 21:47:19.570163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.570017 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="registry" Jun 11 21:47:19.570163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.570022 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="online" Jun 11 21:47:19.570163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.570028 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="online" Jun 11 21:47:19.570163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.570034 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="ui" Jun 11 21:47:19.570163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.570038 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="ui" Jun 11 21:47:19.570163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.570044 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="online" Jun 11 21:47:19.570163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.570050 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="online" Jun 11 21:47:19.570163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.570055 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="feast-init" Jun 11 21:47:19.570163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.570060 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="feast-init" Jun 11 21:47:19.570163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.570066 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="feast-apply" Jun 11 21:47:19.570163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.570071 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="feast-apply" Jun 11 21:47:19.570163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.570080 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="offline" Jun 11 21:47:19.570163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.570085 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="offline" Jun 11 21:47:19.570163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.570093 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="feast-init" Jun 11 21:47:19.570163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.570098 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="feast-init" Jun 11 21:47:19.570163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.570134 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="offline" Jun 11 21:47:19.570163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.570141 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="online" Jun 11 21:47:19.570163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.570146 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="offline" Jun 11 21:47:19.570163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.570152 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="registry" Jun 11 21:47:19.570163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.570157 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="online" Jun 11 21:47:19.570163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.570165 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="efb16dee-4971-458a-8cf4-6f95d4506c87" containerName="ui" Jun 11 21:47:19.570163 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.570172 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbc23dc3-98c4-4934-8b17-c01e0a91ada6" containerName="ui" Jun 11 21:47:19.573182 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.573166 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8" Jun 11 21:47:19.575855 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.575827 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-knwbb\"/\"default-dockercfg-ws4gd\"" Jun 11 21:47:19.575855 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.575827 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-knwbb\"/\"openshift-service-ca.crt\"" Jun 11 21:47:19.576754 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.576733 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-knwbb\"/\"kube-root-ca.crt\"" Jun 11 21:47:19.580915 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.580895 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8"] Jun 11 21:47:19.740808 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.740771 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/95042e7b-1959-4a00-a571-98dc27f36254-podres\") pod \"perf-node-gather-daemonset-4n8x8\" (UID: \"95042e7b-1959-4a00-a571-98dc27f36254\") " pod="openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8" Jun 11 21:47:19.741002 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.740845 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjhnp\" (UniqueName: \"kubernetes.io/projected/95042e7b-1959-4a00-a571-98dc27f36254-kube-api-access-pjhnp\") pod \"perf-node-gather-daemonset-4n8x8\" (UID: \"95042e7b-1959-4a00-a571-98dc27f36254\") " pod="openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8" Jun 11 21:47:19.741002 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.740879 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/95042e7b-1959-4a00-a571-98dc27f36254-proc\") pod \"perf-node-gather-daemonset-4n8x8\" (UID: \"95042e7b-1959-4a00-a571-98dc27f36254\") " pod="openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8" Jun 11 21:47:19.741002 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.740914 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95042e7b-1959-4a00-a571-98dc27f36254-lib-modules\") pod \"perf-node-gather-daemonset-4n8x8\" (UID: \"95042e7b-1959-4a00-a571-98dc27f36254\") " pod="openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8" Jun 11 21:47:19.741002 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.740945 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95042e7b-1959-4a00-a571-98dc27f36254-sys\") pod \"perf-node-gather-daemonset-4n8x8\" (UID: \"95042e7b-1959-4a00-a571-98dc27f36254\") " pod="openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8" Jun 11 21:47:19.842054 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.842024 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95042e7b-1959-4a00-a571-98dc27f36254-lib-modules\") pod \"perf-node-gather-daemonset-4n8x8\" (UID: \"95042e7b-1959-4a00-a571-98dc27f36254\") " pod="openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8" Jun 11 21:47:19.842221 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.842066 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95042e7b-1959-4a00-a571-98dc27f36254-sys\") pod \"perf-node-gather-daemonset-4n8x8\" (UID: \"95042e7b-1959-4a00-a571-98dc27f36254\") " pod="openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8" Jun 11 21:47:19.842221 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.842088 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/95042e7b-1959-4a00-a571-98dc27f36254-podres\") pod \"perf-node-gather-daemonset-4n8x8\" (UID: \"95042e7b-1959-4a00-a571-98dc27f36254\") " pod="openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8" Jun 11 21:47:19.842221 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.842123 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjhnp\" (UniqueName: \"kubernetes.io/projected/95042e7b-1959-4a00-a571-98dc27f36254-kube-api-access-pjhnp\") pod \"perf-node-gather-daemonset-4n8x8\" (UID: \"95042e7b-1959-4a00-a571-98dc27f36254\") " pod="openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8" Jun 11 21:47:19.842221 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.842146 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/95042e7b-1959-4a00-a571-98dc27f36254-proc\") pod \"perf-node-gather-daemonset-4n8x8\" (UID: \"95042e7b-1959-4a00-a571-98dc27f36254\") " pod="openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8" Jun 11 21:47:19.842221 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.842163 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95042e7b-1959-4a00-a571-98dc27f36254-sys\") pod \"perf-node-gather-daemonset-4n8x8\" (UID: \"95042e7b-1959-4a00-a571-98dc27f36254\") " pod="openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8" Jun 11 21:47:19.842221 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.842211 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95042e7b-1959-4a00-a571-98dc27f36254-lib-modules\") pod \"perf-node-gather-daemonset-4n8x8\" (UID: \"95042e7b-1959-4a00-a571-98dc27f36254\") " pod="openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8" Jun 11 21:47:19.842429 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.842217 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/95042e7b-1959-4a00-a571-98dc27f36254-proc\") pod \"perf-node-gather-daemonset-4n8x8\" (UID: \"95042e7b-1959-4a00-a571-98dc27f36254\") " pod="openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8" Jun 11 21:47:19.842429 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.842277 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/95042e7b-1959-4a00-a571-98dc27f36254-podres\") pod \"perf-node-gather-daemonset-4n8x8\" (UID: \"95042e7b-1959-4a00-a571-98dc27f36254\") " pod="openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8" Jun 11 21:47:19.850873 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.850853 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjhnp\" (UniqueName: \"kubernetes.io/projected/95042e7b-1959-4a00-a571-98dc27f36254-kube-api-access-pjhnp\") pod \"perf-node-gather-daemonset-4n8x8\" (UID: \"95042e7b-1959-4a00-a571-98dc27f36254\") " pod="openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8" Jun 11 21:47:19.883855 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:19.883825 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8" Jun 11 21:47:20.010701 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:20.010674 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8"] Jun 11 21:47:20.014054 ip-10-0-143-6 kubenswrapper[2570]: W0611 21:47:20.014018 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod95042e7b_1959_4a00_a571_98dc27f36254.slice/crio-e833851ca9f81d90b11ea635ee6a21aeef570ac170d0c8196761669919dc1535 WatchSource:0}: Error finding container e833851ca9f81d90b11ea635ee6a21aeef570ac170d0c8196761669919dc1535: Status 404 returned error can't find the container with id e833851ca9f81d90b11ea635ee6a21aeef570ac170d0c8196761669919dc1535 Jun 11 21:47:20.147791 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:20.147766 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8" event={"ID":"95042e7b-1959-4a00-a571-98dc27f36254","Type":"ContainerStarted","Data":"e833851ca9f81d90b11ea635ee6a21aeef570ac170d0c8196761669919dc1535"} Jun 11 21:47:20.751425 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:20.751393 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zpqjq_2c0c15cb-361e-40de-83bf-06ccac34cc83/dns/0.log" Jun 11 21:47:20.777052 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:20.777030 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zpqjq_2c0c15cb-361e-40de-83bf-06ccac34cc83/kube-rbac-proxy/0.log" Jun 11 21:47:20.854811 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:20.854781 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s8b5p_bdfaf7d3-ae13-48e9-b9ea-040ae02d886a/dns-node-resolver/0.log" Jun 11 21:47:21.152979 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:21.152943 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8" event={"ID":"95042e7b-1959-4a00-a571-98dc27f36254","Type":"ContainerStarted","Data":"a348b2da529049174f52cd9472e213c1e7206af048653ba42d6b4f75c2e4e7c3"} Jun 11 21:47:21.153180 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:21.153042 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8" Jun 11 21:47:21.169290 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:21.169239 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8" podStartSLOduration=2.169224484 podStartE2EDuration="2.169224484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 21:47:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 21:47:21.167720755 +0000 UTC m=+1899.579044285" watchObservedRunningTime="2026-06-11 21:47:21.169224484 +0000 UTC m=+1899.580548021" Jun 11 21:47:21.298856 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:21.298799 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-thgw8_7b88bd69-b4df-4345-a24c-e51adeba9023/node-ca/0.log" Jun 11 21:47:22.366354 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:22.366305 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-f5djm_5317cca9-f317-4c21-9dad-a2f1ac629264/serve-healthcheck-canary/0.log" Jun 11 21:47:22.807732 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:22.807636 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lbtkn_3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1/kube-rbac-proxy/0.log" Jun 11 21:47:22.827829 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:22.827794 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lbtkn_3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1/exporter/0.log" Jun 11 21:47:22.849220 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:22.849194 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lbtkn_3efb29bd-d2e2-4d55-bcd3-039c4fbf83d1/extractor/0.log" Jun 11 21:47:27.166620 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:27.166595 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-knwbb/perf-node-gather-daemonset-4n8x8" Jun 11 21:47:28.564088 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:28.564053 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w26g9_5678560c-39b0-4892-a879-3613ac253097/kube-multus-additional-cni-plugins/0.log" Jun 11 21:47:28.587891 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:28.587865 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w26g9_5678560c-39b0-4892-a879-3613ac253097/egress-router-binary-copy/0.log" Jun 11 21:47:28.610440 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:28.610416 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w26g9_5678560c-39b0-4892-a879-3613ac253097/cni-plugins/0.log" Jun 11 21:47:28.631643 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:28.631555 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w26g9_5678560c-39b0-4892-a879-3613ac253097/bond-cni-plugin/0.log" Jun 11 21:47:28.656508 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:28.656437 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w26g9_5678560c-39b0-4892-a879-3613ac253097/routeoverride-cni/0.log" Jun 11 21:47:28.679894 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:28.679870 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w26g9_5678560c-39b0-4892-a879-3613ac253097/whereabouts-cni-bincopy/0.log" Jun 11 21:47:28.703893 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:28.703866 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w26g9_5678560c-39b0-4892-a879-3613ac253097/whereabouts-cni/0.log" Jun 11 21:47:28.734189 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:28.734154 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-bl9g7_9094ab8c-8937-44bb-9f4e-ed8445c3c55f/kube-multus/0.log" Jun 11 21:47:28.758462 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:28.758439 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2bmhl_329e0949-4197-4a29-afd8-1300e76b75cf/network-metrics-daemon/0.log" Jun 11 21:47:28.784795 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:28.784775 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2bmhl_329e0949-4197-4a29-afd8-1300e76b75cf/kube-rbac-proxy/0.log" Jun 11 21:47:29.868457 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:29.868428 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/ovn-controller/0.log" Jun 11 21:47:29.890411 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:29.890385 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/ovn-acl-logging/0.log" Jun 11 21:47:29.890544 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:29.890494 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/ovn-acl-logging/1.log" Jun 11 21:47:29.913134 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:29.913109 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/kube-rbac-proxy-node/0.log" Jun 11 21:47:29.935542 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:29.935518 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/kube-rbac-proxy-ovn-metrics/0.log" Jun 11 21:47:29.954085 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:29.954055 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/northd/0.log" Jun 11 21:47:29.979938 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:29.979914 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/nbdb/0.log" Jun 11 21:47:30.001688 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:30.001666 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/sbdb/0.log" Jun 11 21:47:30.040377 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:30.040353 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b4gmk_c7849da4-f326-4b8c-a436-5b0d68958170/ovnkube-controller/0.log" Jun 11 21:47:31.226856 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:31.226826 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-shxrd_54688996-c5d4-445d-a6ff-4997a19fe1d6/network-check-target-container/0.log" Jun 11 21:47:32.133421 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:32.133397 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-cqksf_71fed145-1fbc-4433-bfb0-394c5631bec6/iptables-alerter/0.log" Jun 11 21:47:32.875162 ip-10-0-143-6 kubenswrapper[2570]: I0611 21:47:32.875118 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-fhhkw_275b0bd4-429e-4171-b5f8-70a3bb42f08f/tuned/0.log"