Sep 10 12:17:51.890092 ip-10-0-143-110 systemd[1]: Starting Kubernetes Kubelet... Sep 10 12:17:52.261450 ip-10-0-143-110 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 12:17:52.261450 ip-10-0-143-110 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 10 12:17:52.261450 ip-10-0-143-110 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 12:17:52.261450 ip-10-0-143-110 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 10 12:17:52.261450 ip-10-0-143-110 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 12:17:52.262117 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.261490 2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 10 12:17:52.265026 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265011 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 12:17:52.265026 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265025 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 12:17:52.265094 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265030 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 12:17:52.265094 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265034 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 12:17:52.265094 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265038 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 12:17:52.265094 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265041 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 12:17:52.265094 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265044 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 12:17:52.265094 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265047 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 12:17:52.265094 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265050 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 12:17:52.265094 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265052 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 12:17:52.265094 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265056 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 12:17:52.265094 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265058 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 12:17:52.265094 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265061 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 12:17:52.265094 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265064 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 12:17:52.265094 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265066 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 12:17:52.265094 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265069 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 12:17:52.265094 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265072 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 12:17:52.265094 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265074 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 12:17:52.265094 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265077 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 12:17:52.265094 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265081 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 12:17:52.265094 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265083 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 12:17:52.265094 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265086 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 12:17:52.265570 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265088 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 12:17:52.265570 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265091 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 12:17:52.265570 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265093 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 12:17:52.265570 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265096 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 12:17:52.265570 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265099 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 12:17:52.265570 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265101 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 12:17:52.265570 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265104 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 12:17:52.265570 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265109 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 12:17:52.265570 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265111 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 12:17:52.265570 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265114 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 12:17:52.265570 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265117 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 12:17:52.265570 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265119 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 12:17:52.265570 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265122 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 12:17:52.265570 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265124 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 12:17:52.265570 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265127 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 12:17:52.265570 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265129 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 12:17:52.265570 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265132 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 12:17:52.265570 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265134 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 12:17:52.265570 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265137 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 12:17:52.265570 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265139 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 12:17:52.266065 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265142 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 12:17:52.266065 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265144 2583 feature_gate.go:328] unrecognized feature gate: Example Sep 10 12:17:52.266065 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265147 2583 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 12:17:52.266065 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265150 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 12:17:52.266065 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265153 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 12:17:52.266065 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265155 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 12:17:52.266065 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265158 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 12:17:52.266065 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265160 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 12:17:52.266065 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265163 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 12:17:52.266065 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265165 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 12:17:52.266065 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265168 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 12:17:52.266065 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265171 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 12:17:52.266065 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265173 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 12:17:52.266065 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265176 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 12:17:52.266065 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265182 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 12:17:52.266065 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265185 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 12:17:52.266065 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265188 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 12:17:52.266065 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265191 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 12:17:52.266065 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265194 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 12:17:52.266065 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265197 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 12:17:52.266548 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265199 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 12:17:52.266548 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265202 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 12:17:52.266548 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265205 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 12:17:52.266548 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265207 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 12:17:52.266548 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265210 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 12:17:52.266548 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265212 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 12:17:52.266548 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265215 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 12:17:52.266548 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265217 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 12:17:52.266548 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265220 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 12:17:52.266548 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265222 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 12:17:52.266548 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265225 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 12:17:52.266548 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265227 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 12:17:52.266548 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265229 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 12:17:52.266548 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265232 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 12:17:52.266548 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265235 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 12:17:52.266548 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265238 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 12:17:52.266548 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265242 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 12:17:52.266548 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265246 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 12:17:52.266548 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265250 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 12:17:52.267014 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265253 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 12:17:52.267014 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265256 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 12:17:52.267014 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265259 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 12:17:52.267014 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265262 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 12:17:52.267014 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265265 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 12:17:52.267014 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265267 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 12:17:52.267014 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265270 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 12:17:52.267014 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265274 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 12:17:52.267014 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265277 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 12:17:52.267014 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265280 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 12:17:52.267014 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265282 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 12:17:52.267014 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265285 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 12:17:52.267014 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265288 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 12:17:52.267014 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265290 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 12:17:52.267014 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265293 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 12:17:52.267014 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265296 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 12:17:52.267014 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265298 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 12:17:52.267014 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.265301 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 12:17:52.267014 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266031 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 12:17:52.267014 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266039 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 12:17:52.267497 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266042 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 12:17:52.267497 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266044 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 12:17:52.267497 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266047 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 12:17:52.267497 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266050 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 12:17:52.267497 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266052 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 12:17:52.267497 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266055 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 12:17:52.267497 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266059 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 12:17:52.267497 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266063 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 12:17:52.267497 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266066 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 12:17:52.267497 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266070 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 12:17:52.267497 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266073 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 12:17:52.267497 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266076 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 12:17:52.267497 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266079 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 12:17:52.267497 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266081 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 12:17:52.267497 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266084 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 12:17:52.267497 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266086 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 12:17:52.267497 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266089 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 12:17:52.267497 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266092 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 12:17:52.267497 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266095 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 12:17:52.267976 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266097 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 12:17:52.267976 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266101 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 12:17:52.267976 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266105 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 12:17:52.267976 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266108 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 12:17:52.267976 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266111 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 12:17:52.267976 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266114 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 12:17:52.267976 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266117 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 12:17:52.267976 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266119 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 12:17:52.267976 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266122 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 12:17:52.267976 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266125 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 12:17:52.267976 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266128 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 12:17:52.267976 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266130 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 12:17:52.267976 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266133 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 12:17:52.267976 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266137 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 12:17:52.267976 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266139 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 12:17:52.267976 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266142 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 12:17:52.267976 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266145 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 12:17:52.267976 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266147 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 12:17:52.267976 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266150 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 12:17:52.267976 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266152 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 12:17:52.268457 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266155 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 12:17:52.268457 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266158 2583 feature_gate.go:328] unrecognized feature gate: Example Sep 10 12:17:52.268457 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266161 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 12:17:52.268457 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266165 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 12:17:52.268457 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266168 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 12:17:52.268457 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266172 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 12:17:52.268457 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266175 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 12:17:52.268457 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266179 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 12:17:52.268457 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266181 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 12:17:52.268457 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266184 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 12:17:52.268457 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266187 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 12:17:52.268457 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266189 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 12:17:52.268457 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266192 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 12:17:52.268457 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266195 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 12:17:52.268457 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266197 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 12:17:52.268457 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266207 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 12:17:52.268457 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266211 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 12:17:52.268457 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266214 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 12:17:52.268457 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266217 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 12:17:52.268457 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266220 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 12:17:52.268955 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266223 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 12:17:52.268955 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266226 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 12:17:52.268955 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266228 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 12:17:52.268955 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266232 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 12:17:52.268955 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266234 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 12:17:52.268955 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266237 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 12:17:52.268955 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266240 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 12:17:52.268955 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266242 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 12:17:52.268955 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266244 2583 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 12:17:52.268955 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266247 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 12:17:52.268955 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266249 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 12:17:52.268955 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266252 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 12:17:52.268955 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266254 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 12:17:52.268955 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266257 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 12:17:52.268955 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266260 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 12:17:52.268955 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266262 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 12:17:52.268955 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266266 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 12:17:52.268955 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266269 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 12:17:52.268955 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266271 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 12:17:52.268955 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266274 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 12:17:52.269442 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266276 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 12:17:52.269442 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266279 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 12:17:52.269442 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266281 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 12:17:52.269442 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266285 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 12:17:52.269442 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266287 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 12:17:52.269442 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266290 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 12:17:52.269442 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266292 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 12:17:52.269442 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266295 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 12:17:52.269442 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266298 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 12:17:52.269442 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266300 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 12:17:52.269442 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266303 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 12:17:52.269442 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266305 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 12:17:52.269442 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266308 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 12:17:52.269442 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266310 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 12:17:52.269442 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266313 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 12:17:52.269442 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266315 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 12:17:52.269442 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266318 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 12:17:52.269442 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266321 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 12:17:52.269442 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266669 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 12:17:52.270006 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266673 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 12:17:52.270006 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266676 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 12:17:52.270006 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266679 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 12:17:52.270006 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266682 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 12:17:52.270006 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266684 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 12:17:52.270006 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266687 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 12:17:52.270006 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266690 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 12:17:52.270006 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266693 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 12:17:52.270006 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266695 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 12:17:52.270006 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266698 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 12:17:52.270006 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266701 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 12:17:52.270006 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266703 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 12:17:52.270006 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266706 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 12:17:52.270006 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266709 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 12:17:52.270006 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266711 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 12:17:52.270006 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266714 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 12:17:52.270006 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266717 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 12:17:52.270006 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266719 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 12:17:52.270006 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266722 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 12:17:52.270006 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266725 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 12:17:52.270539 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266728 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 12:17:52.270539 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266730 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 12:17:52.270539 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266733 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 12:17:52.270539 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266735 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 12:17:52.270539 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266737 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 12:17:52.270539 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266740 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 12:17:52.270539 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266742 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 12:17:52.270539 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266746 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 12:17:52.270539 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266750 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 12:17:52.270539 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266753 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 12:17:52.270539 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266755 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 12:17:52.270539 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266757 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 12:17:52.270539 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266760 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 12:17:52.270539 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266762 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 12:17:52.270539 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266765 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 12:17:52.270539 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266768 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 12:17:52.270539 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266770 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 12:17:52.270539 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266773 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 12:17:52.270539 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266775 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 12:17:52.271025 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266778 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 12:17:52.271025 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266780 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 12:17:52.271025 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266783 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 12:17:52.271025 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266786 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 12:17:52.271025 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266788 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 12:17:52.271025 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266791 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 12:17:52.271025 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266793 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 12:17:52.271025 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266796 2583 feature_gate.go:328] unrecognized feature gate: Example Sep 10 12:17:52.271025 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266798 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 12:17:52.271025 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266801 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 12:17:52.271025 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266804 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 12:17:52.271025 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266806 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 12:17:52.271025 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266809 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 12:17:52.271025 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266811 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 12:17:52.271025 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266814 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 12:17:52.271025 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266816 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 12:17:52.271025 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266818 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 12:17:52.271025 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266821 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 12:17:52.271025 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266823 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 12:17:52.271025 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266826 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 12:17:52.271509 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266829 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 12:17:52.271509 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266831 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 12:17:52.271509 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266848 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 12:17:52.271509 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266852 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 12:17:52.271509 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266855 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 12:17:52.271509 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266858 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 12:17:52.271509 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266863 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 12:17:52.271509 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266866 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 12:17:52.271509 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266872 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 12:17:52.271509 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266875 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 12:17:52.271509 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266877 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 12:17:52.271509 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266880 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 12:17:52.271509 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266883 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 12:17:52.271509 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266885 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 12:17:52.271509 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266888 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 12:17:52.271509 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266890 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 12:17:52.271509 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266893 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 12:17:52.271509 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266896 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 12:17:52.271509 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266899 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 12:17:52.272034 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266901 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 12:17:52.272034 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266904 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 12:17:52.272034 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266907 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 12:17:52.272034 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266909 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 12:17:52.272034 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266912 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 12:17:52.272034 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266914 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 12:17:52.272034 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266917 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 12:17:52.272034 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266919 2583 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 12:17:52.272034 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266922 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 12:17:52.272034 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266924 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 12:17:52.272034 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266927 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 12:17:52.272034 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266929 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 12:17:52.272034 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266932 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 12:17:52.272034 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266934 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 12:17:52.272034 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266937 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 12:17:52.272034 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266939 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 12:17:52.272034 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266942 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 12:17:52.272034 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266946 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 12:17:52.272034 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266948 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 12:17:52.272034 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.266951 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 12:17:52.272512 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267021 2583 flags.go:64] FLAG: --address="0.0.0.0" Sep 10 12:17:52.272512 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267053 2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 10 12:17:52.272512 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267059 2583 flags.go:64] FLAG: --anonymous-auth="true" Sep 10 12:17:52.272512 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267064 2583 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 10 12:17:52.272512 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267067 2583 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 10 12:17:52.272512 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267071 2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 10 12:17:52.272512 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267075 2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 10 12:17:52.272512 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267079 2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 10 12:17:52.272512 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267082 2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 10 12:17:52.272512 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267085 2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 10 12:17:52.272512 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267093 2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 10 12:17:52.272512 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267097 2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 10 12:17:52.272512 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267100 2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 10 12:17:52.272512 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267103 2583 flags.go:64] FLAG: --cgroup-root="" Sep 10 12:17:52.272512 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267106 2583 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 10 12:17:52.272512 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267109 2583 flags.go:64] FLAG: --client-ca-file="" Sep 10 12:17:52.272512 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267111 2583 flags.go:64] FLAG: --cloud-provider="external" Sep 10 12:17:52.272512 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267114 2583 flags.go:64] FLAG: --cluster-dns="[]" Sep 10 12:17:52.272512 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267119 2583 flags.go:64] FLAG: --cluster-domain="" Sep 10 12:17:52.272512 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267122 2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 10 12:17:52.272512 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267125 2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 10 12:17:52.272512 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267128 2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 10 12:17:52.272512 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267131 2583 flags.go:64] FLAG: --container-log-max-files="5" Sep 10 12:17:52.272512 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267135 2583 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267138 2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267141 2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267144 2583 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267147 2583 flags.go:64] FLAG: --contention-profiling="false" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267150 2583 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267154 2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267157 2583 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267160 2583 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267164 2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267168 2583 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267171 2583 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267174 2583 flags.go:64] FLAG: --enable-load-reader="false" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267177 2583 flags.go:64] FLAG: --enable-server="true" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267180 2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267184 2583 flags.go:64] FLAG: --event-burst="100" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267187 2583 flags.go:64] FLAG: --event-qps="50" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267190 2583 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267193 2583 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267196 2583 flags.go:64] FLAG: --eviction-hard="" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267200 2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267203 2583 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267206 2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267208 2583 flags.go:64] FLAG: --eviction-soft="" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267211 2583 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 10 12:17:52.273093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267214 2583 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 10 12:17:52.273733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267217 2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 10 12:17:52.273733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267220 2583 flags.go:64] FLAG: --experimental-mounter-path="" Sep 10 12:17:52.273733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267223 2583 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 10 12:17:52.273733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267226 2583 flags.go:64] FLAG: --fail-swap-on="true" Sep 10 12:17:52.273733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267229 2583 flags.go:64] FLAG: --feature-gates="" Sep 10 12:17:52.273733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267232 2583 flags.go:64] FLAG: --file-check-frequency="20s" Sep 10 12:17:52.273733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267235 2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 10 12:17:52.273733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267238 2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 10 12:17:52.273733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267241 2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 10 12:17:52.273733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267244 2583 flags.go:64] FLAG: --healthz-port="10248" Sep 10 12:17:52.273733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267247 2583 flags.go:64] FLAG: --help="false" Sep 10 12:17:52.273733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267250 2583 flags.go:64] FLAG: --hostname-override="ip-10-0-143-110.ec2.internal" Sep 10 12:17:52.273733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267254 2583 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 10 12:17:52.273733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267257 2583 flags.go:64] FLAG: --http-check-frequency="20s" Sep 10 12:17:52.273733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267260 2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 10 12:17:52.273733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267263 2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 10 12:17:52.273733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267268 2583 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 10 12:17:52.273733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267270 2583 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 10 12:17:52.273733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267273 2583 flags.go:64] FLAG: --image-service-endpoint="" Sep 10 12:17:52.273733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267276 2583 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 10 12:17:52.273733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267279 2583 flags.go:64] FLAG: --kube-api-burst="100" Sep 10 12:17:52.273733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267282 2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 10 12:17:52.273733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267285 2583 flags.go:64] FLAG: --kube-api-qps="50" Sep 10 12:17:52.273733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267287 2583 flags.go:64] FLAG: --kube-reserved="" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267290 2583 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267293 2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267296 2583 flags.go:64] FLAG: --kubelet-cgroups="" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267299 2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267302 2583 flags.go:64] FLAG: --lock-file="" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267305 2583 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267308 2583 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267311 2583 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267316 2583 flags.go:64] FLAG: --log-json-split-stream="false" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267318 2583 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267321 2583 flags.go:64] FLAG: --log-text-split-stream="false" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267324 2583 flags.go:64] FLAG: --logging-format="text" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267326 2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267329 2583 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267332 2583 flags.go:64] FLAG: --manifest-url="" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267335 2583 flags.go:64] FLAG: --manifest-url-header="" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267339 2583 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267341 2583 flags.go:64] FLAG: --max-open-files="1000000" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267345 2583 flags.go:64] FLAG: --max-pods="110" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267348 2583 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267352 2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267355 2583 flags.go:64] FLAG: --memory-manager-policy="None" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267358 2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267360 2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 10 12:17:52.274374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267365 2583 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267367 2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267378 2583 flags.go:64] FLAG: --node-status-max-images="50" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267381 2583 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267384 2583 flags.go:64] FLAG: --oom-score-adj="-999" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267387 2583 flags.go:64] FLAG: --pod-cidr="" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267390 2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267396 2583 flags.go:64] FLAG: --pod-manifest-path="" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267399 2583 flags.go:64] FLAG: --pod-max-pids="-1" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267402 2583 flags.go:64] FLAG: --pods-per-core="0" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267405 2583 flags.go:64] FLAG: --port="10250" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267407 2583 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267411 2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f7a099faed981870" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267414 2583 flags.go:64] FLAG: --qos-reserved="" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267417 2583 flags.go:64] FLAG: --read-only-port="10255" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267420 2583 flags.go:64] FLAG: --register-node="true" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267423 2583 flags.go:64] FLAG: --register-with-taints="" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267427 2583 flags.go:64] FLAG: --registry-burst="10" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267429 2583 flags.go:64] FLAG: --registry-qps="5" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267432 2583 flags.go:64] FLAG: --reserved-cpus="" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267435 2583 flags.go:64] FLAG: --reserved-memory="" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267439 2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267442 2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267445 2583 flags.go:64] FLAG: --rotate-certificates="false" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267447 2583 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 10 12:17:52.275016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267450 2583 flags.go:64] FLAG: --runonce="false" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267453 2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267456 2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267462 2583 flags.go:64] FLAG: --seccomp-default="false" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267465 2583 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267468 2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267471 2583 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267474 2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267478 2583 flags.go:64] FLAG: --storage-driver-password="root" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267481 2583 flags.go:64] FLAG: --storage-driver-secure="false" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267483 2583 flags.go:64] FLAG: --storage-driver-table="stats" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267486 2583 flags.go:64] FLAG: --storage-driver-user="root" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267489 2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267492 2583 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267495 2583 flags.go:64] FLAG: --system-cgroups="" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267497 2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267503 2583 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267506 2583 flags.go:64] FLAG: --tls-cert-file="" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267509 2583 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267513 2583 flags.go:64] FLAG: --tls-min-version="" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267516 2583 flags.go:64] FLAG: --tls-private-key-file="" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267519 2583 flags.go:64] FLAG: --topology-manager-policy="none" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267522 2583 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267525 2583 flags.go:64] FLAG: --topology-manager-scope="container" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267528 2583 flags.go:64] FLAG: --v="2" Sep 10 12:17:52.275620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267531 2583 flags.go:64] FLAG: --version="false" Sep 10 12:17:52.276249 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267535 2583 flags.go:64] FLAG: --vmodule="" Sep 10 12:17:52.276249 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267539 2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 10 12:17:52.276249 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.267542 2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 10 12:17:52.276249 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267652 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 12:17:52.276249 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267656 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 12:17:52.276249 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267659 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 12:17:52.276249 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267662 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 12:17:52.276249 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267665 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 12:17:52.276249 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267668 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 12:17:52.276249 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267671 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 12:17:52.276249 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267675 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 12:17:52.276249 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267677 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 12:17:52.276249 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267681 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 12:17:52.276249 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267685 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 12:17:52.276249 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267689 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 12:17:52.276249 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267692 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 12:17:52.276249 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267695 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 12:17:52.276249 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267698 2583 feature_gate.go:328] unrecognized feature gate: Example Sep 10 12:17:52.276249 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267701 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 12:17:52.276249 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267703 2583 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 12:17:52.276767 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267706 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 12:17:52.276767 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267709 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 12:17:52.276767 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267711 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 12:17:52.276767 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267714 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 12:17:52.276767 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267717 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 12:17:52.276767 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267721 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 12:17:52.276767 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267723 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 12:17:52.276767 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267731 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 12:17:52.276767 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267734 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 12:17:52.276767 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267737 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 12:17:52.276767 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267739 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 12:17:52.276767 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267742 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 12:17:52.276767 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267744 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 12:17:52.276767 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267747 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 12:17:52.276767 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267750 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 12:17:52.276767 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267752 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 12:17:52.276767 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267761 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 12:17:52.276767 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267763 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 12:17:52.276767 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267766 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 12:17:52.276767 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267769 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 12:17:52.277266 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267771 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 12:17:52.277266 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267774 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 12:17:52.277266 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267777 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 12:17:52.277266 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267779 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 12:17:52.277266 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267782 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 12:17:52.277266 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267784 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 12:17:52.277266 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267788 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 12:17:52.277266 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267791 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 12:17:52.277266 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267793 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 12:17:52.277266 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267796 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 12:17:52.277266 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267799 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 12:17:52.277266 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267801 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 12:17:52.277266 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267804 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 12:17:52.277266 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267806 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 12:17:52.277266 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267809 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 12:17:52.277266 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267811 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 12:17:52.277266 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267814 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 12:17:52.277266 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267816 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 12:17:52.277266 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267819 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 12:17:52.277716 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267821 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 12:17:52.277716 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267829 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 12:17:52.277716 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267832 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 12:17:52.277716 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267847 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 12:17:52.277716 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267850 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 12:17:52.277716 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267852 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 12:17:52.277716 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267855 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 12:17:52.277716 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267858 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 12:17:52.277716 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267860 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 12:17:52.277716 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267863 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 12:17:52.277716 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267865 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 12:17:52.277716 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267868 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 12:17:52.277716 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267870 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 12:17:52.277716 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267873 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 12:17:52.277716 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267876 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 12:17:52.277716 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267878 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 12:17:52.277716 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267881 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 12:17:52.277716 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267883 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 12:17:52.277716 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267886 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 12:17:52.277716 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267890 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 12:17:52.278226 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267892 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 12:17:52.278226 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267895 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 12:17:52.278226 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267897 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 12:17:52.278226 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267900 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 12:17:52.278226 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267903 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 12:17:52.278226 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267905 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 12:17:52.278226 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267908 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 12:17:52.278226 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267910 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 12:17:52.278226 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267913 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 12:17:52.278226 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267917 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 12:17:52.278226 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267937 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 12:17:52.278226 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267941 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 12:17:52.278226 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267945 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 12:17:52.278226 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267948 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 12:17:52.278226 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267951 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 12:17:52.278226 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267954 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 12:17:52.278226 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267957 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 12:17:52.278226 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267961 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 12:17:52.278226 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267965 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 12:17:52.278226 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267969 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 12:17:52.278709 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267973 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 12:17:52.278709 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267976 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 12:17:52.278709 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.267979 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 12:17:52.278709 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.268580 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 12:17:52.278709 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.275200 2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 10 12:17:52.278709 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.275214 2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 10 12:17:52.278709 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.276699 2583 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 10 12:17:52.278709 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.276706 2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 10 12:17:52.278709 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276760 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 12:17:52.278709 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276766 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 12:17:52.278709 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276769 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 12:17:52.278709 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276774 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 12:17:52.278709 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276778 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 12:17:52.278709 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276782 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 12:17:52.279098 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276786 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 12:17:52.279098 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276789 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 12:17:52.279098 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276791 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 12:17:52.279098 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276794 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 12:17:52.279098 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276797 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 12:17:52.279098 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276800 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 12:17:52.279098 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276803 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 12:17:52.279098 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276805 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 12:17:52.279098 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276808 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 12:17:52.279098 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276810 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 12:17:52.279098 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276813 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 12:17:52.279098 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276815 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 12:17:52.279098 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276818 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 12:17:52.279098 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276821 2583 feature_gate.go:328] unrecognized feature gate: Example Sep 10 12:17:52.279098 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276823 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 12:17:52.279098 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276826 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 12:17:52.279098 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276828 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 12:17:52.279098 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276831 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 12:17:52.279098 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276834 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 12:17:52.279621 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276849 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 12:17:52.279621 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276853 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 12:17:52.279621 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276855 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 12:17:52.279621 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276858 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 12:17:52.279621 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276863 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 12:17:52.279621 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276866 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 12:17:52.279621 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276868 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 12:17:52.279621 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276871 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 12:17:52.279621 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276873 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 12:17:52.279621 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276876 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 12:17:52.279621 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276879 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 12:17:52.279621 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276881 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 12:17:52.279621 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276884 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 12:17:52.279621 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276887 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 12:17:52.279621 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276890 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 12:17:52.279621 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276892 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 12:17:52.279621 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276895 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 12:17:52.279621 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276897 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 12:17:52.279621 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276900 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 12:17:52.279621 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276902 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 12:17:52.280188 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276905 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 12:17:52.280188 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276907 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 12:17:52.280188 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276910 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 12:17:52.280188 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276912 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 12:17:52.280188 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276915 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 12:17:52.280188 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276919 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 12:17:52.280188 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276923 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 12:17:52.280188 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276926 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 12:17:52.280188 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276928 2583 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 12:17:52.280188 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276931 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 12:17:52.280188 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276934 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 12:17:52.280188 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276936 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 12:17:52.280188 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276939 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 12:17:52.280188 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276941 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 12:17:52.280188 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276944 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 12:17:52.280188 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276946 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 12:17:52.280188 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276949 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 12:17:52.280188 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276953 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 12:17:52.280188 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276955 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 12:17:52.280188 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276958 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 12:17:52.280730 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276960 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 12:17:52.280730 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276963 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 12:17:52.280730 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276965 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 12:17:52.280730 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276968 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 12:17:52.280730 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276970 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 12:17:52.280730 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276973 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 12:17:52.280730 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276975 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 12:17:52.280730 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276978 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 12:17:52.280730 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276980 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 12:17:52.280730 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276983 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 12:17:52.280730 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276985 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 12:17:52.280730 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276988 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 12:17:52.280730 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276991 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 12:17:52.280730 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276993 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 12:17:52.280730 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276996 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 12:17:52.280730 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.276998 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 12:17:52.280730 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277001 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 12:17:52.280730 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277003 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 12:17:52.280730 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277006 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 12:17:52.280730 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277008 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 12:17:52.281314 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277011 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 12:17:52.281314 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277013 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 12:17:52.281314 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277016 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 12:17:52.281314 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277018 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 12:17:52.281314 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277021 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 12:17:52.281314 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277023 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 12:17:52.281314 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277026 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 12:17:52.281314 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277028 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 12:17:52.281314 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277031 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 12:17:52.281314 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277034 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 12:17:52.281314 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277037 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 12:17:52.281314 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277040 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 12:17:52.281314 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277042 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 12:17:52.281314 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277045 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 12:17:52.281314 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.277050 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 12:17:52.281684 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277157 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 12:17:52.281684 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277162 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 12:17:52.281684 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277165 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 12:17:52.281684 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277168 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 12:17:52.281684 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277171 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 12:17:52.281684 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277174 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 12:17:52.281684 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277177 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 12:17:52.281684 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277180 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 12:17:52.281684 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277183 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 12:17:52.281684 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277186 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 12:17:52.281684 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277191 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 12:17:52.281684 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277194 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 12:17:52.281684 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277196 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 12:17:52.281684 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277199 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 12:17:52.281684 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277202 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 12:17:52.281684 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277204 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 12:17:52.281684 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277207 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 12:17:52.281684 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277209 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 12:17:52.281684 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277212 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 12:17:52.282181 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277214 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 12:17:52.282181 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277217 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 12:17:52.282181 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277219 2583 feature_gate.go:328] unrecognized feature gate: Example Sep 10 12:17:52.282181 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277222 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 12:17:52.282181 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277224 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 12:17:52.282181 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277227 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 12:17:52.282181 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277229 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 12:17:52.282181 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277232 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 12:17:52.282181 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277235 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 12:17:52.282181 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277238 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 12:17:52.282181 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277241 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 12:17:52.282181 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277245 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 12:17:52.282181 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277247 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 12:17:52.282181 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277250 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 12:17:52.282181 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277253 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 12:17:52.282181 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277256 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 12:17:52.282181 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277259 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 12:17:52.282181 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277262 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 12:17:52.282181 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277264 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 12:17:52.282181 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277267 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 12:17:52.282678 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277270 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 12:17:52.282678 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277273 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 12:17:52.282678 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277276 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 12:17:52.282678 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277279 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 12:17:52.282678 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277281 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 12:17:52.282678 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277284 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 12:17:52.282678 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277286 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 12:17:52.282678 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277288 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 12:17:52.282678 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277291 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 12:17:52.282678 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277293 2583 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 12:17:52.282678 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277296 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 12:17:52.282678 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277298 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 12:17:52.282678 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277301 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 12:17:52.282678 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277303 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 12:17:52.282678 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277305 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 12:17:52.282678 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277308 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 12:17:52.282678 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277310 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 12:17:52.282678 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277313 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 12:17:52.282678 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277315 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 12:17:52.282678 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277318 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 12:17:52.283291 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277320 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 12:17:52.283291 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277323 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 12:17:52.283291 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277326 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 12:17:52.283291 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277329 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 12:17:52.283291 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277332 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 12:17:52.283291 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277335 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 12:17:52.283291 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277337 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 12:17:52.283291 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277340 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 12:17:52.283291 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277342 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 12:17:52.283291 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277345 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 12:17:52.283291 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277347 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 12:17:52.283291 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277350 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 12:17:52.283291 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277352 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 12:17:52.283291 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277355 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 12:17:52.283291 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277357 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 12:17:52.283291 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277359 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 12:17:52.283291 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277362 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 12:17:52.283291 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277364 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 12:17:52.283291 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277367 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 12:17:52.283291 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277369 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 12:17:52.283825 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277372 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 12:17:52.283825 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277374 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 12:17:52.283825 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277377 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 12:17:52.283825 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277379 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 12:17:52.283825 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277381 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 12:17:52.283825 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277384 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 12:17:52.283825 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277386 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 12:17:52.283825 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277389 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 12:17:52.283825 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277391 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 12:17:52.283825 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277395 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 12:17:52.283825 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277398 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 12:17:52.283825 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277401 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 12:17:52.283825 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277404 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 12:17:52.283825 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277407 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 12:17:52.283825 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277410 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 12:17:52.283825 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277413 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 12:17:52.283825 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277415 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 12:17:52.283825 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277418 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 12:17:52.283825 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277421 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 12:17:52.284498 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:52.277424 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 12:17:52.284498 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.277429 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 12:17:52.284498 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.277523 2583 server.go:962] "Client rotation is on, will bootstrap in background" Sep 10 12:17:52.284498 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.279308 2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 10 12:17:52.284498 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.279963 2583 server.go:1019] "Starting client certificate rotation" Sep 10 12:17:52.284498 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.280076 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 12:17:52.284498 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.280853 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 12:17:52.284498 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.282103 2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 12:17:52.284498 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.283196 2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 12:17:52.293779 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.293761 2583 log.go:25] "Validated CRI v1 runtime API" Sep 10 12:17:52.296027 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.295922 2583 log.go:25] "Validated CRI v1 image API" Sep 10 12:17:52.296930 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.296909 2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 10 12:17:52.300100 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.300081 2583 fs.go:135] Filesystem UUIDs: map[2fe8a6ea-526c-4841-af15-4e22e143ad2d:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a03afb2f-ccbc-4347-ab00-63e62ffdb6bc:/dev/nvme0n1p4] Sep 10 12:17:52.300163 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.300099 2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 10 12:17:52.306300 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.306190 2583 manager.go:216] Machine: {Timestamp:2026-09-10 12:17:52.304649083 +0000 UTC m=+0.309868534 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095980 MemoryCapacity:33164472320 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22331f8fe2a6737fdc23cf8b786e7a SystemUUID:ec22331f-8fe2-a673-7fdc-23cf8b786e7a BootID:14743121-4a32-4daf-9560-f14d6f4e6be0 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2c:57:5f:14:b7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2c:57:5f:14:b7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:e5:d8:c2:86:de Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164472320 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 10 12:17:52.306300 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.306293 2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 10 12:17:52.306406 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.306365 2583 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 10 12:17:52.309882 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.309861 2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 10 12:17:52.310005 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.309884 2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-110.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 10 12:17:52.310639 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.310626 2583 topology_manager.go:138] "Creating topology manager with none policy" Sep 10 12:17:52.310673 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.310643 2583 container_manager_linux.go:309] "Creating device plugin manager" Sep 10 12:17:52.310673 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.310651 2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 12:17:52.311100 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.311089 2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 12:17:52.312360 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.312350 2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 10 12:17:52.316364 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.316350 2583 state_mem.go:36] "Initialized new in-memory state store" Sep 10 12:17:52.316449 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.316434 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 12:17:52.316493 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.316480 2583 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 10 12:17:52.316530 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.316520 2583 kubelet.go:486] "Attempting to sync node with API server" Sep 10 12:17:52.316563 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.316545 2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 10 12:17:52.316563 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.316559 2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 10 12:17:52.316620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.316568 2583 kubelet.go:398] "Adding apiserver pod source" Sep 10 12:17:52.316620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.316575 2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 10 12:17:52.316922 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.316910 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 12:17:52.316968 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.316934 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 12:17:52.319483 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.319465 2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 10 12:17:52.321556 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.321539 2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 10 12:17:52.321649 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.321577 2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 10 12:17:52.322223 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.322209 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 10 12:17:52.322332 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.322231 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 10 12:17:52.322332 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.322240 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 10 12:17:52.322332 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.322250 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 10 12:17:52.322332 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.322311 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 10 12:17:52.322332 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.322324 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 10 12:17:52.322551 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.322412 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 10 12:17:52.322551 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.322423 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 10 12:17:52.322551 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.322432 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 10 12:17:52.322551 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.322441 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 10 12:17:52.322551 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.322454 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 10 12:17:52.322551 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.322465 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 10 12:17:52.323893 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.323879 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 10 12:17:52.323963 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.323895 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 10 12:17:52.327398 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.327382 2583 server.go:1268] "Started kubelet" Sep 10 12:17:52.327509 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.327453 2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 10 12:17:52.327554 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.327488 2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 10 12:17:52.327659 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.327560 2583 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 10 12:17:52.328231 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:52.328189 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-110.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 12:17:52.328235 ip-10-0-143-110 systemd[1]: Started Kubernetes Kubelet. Sep 10 12:17:52.328464 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.328303 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-110.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 12:17:52.328717 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:52.328685 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 12:17:52.328874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.328398 2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 10 12:17:52.333634 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.333614 2583 server.go:310] "Adding debug handlers to kubelet server" Sep 10 12:17:52.334208 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:52.334178 2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 10 12:17:52.334933 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.334916 2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 10 12:17:52.335017 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.334942 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 10 12:17:52.335017 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:52.334983 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-110.ec2.internal\" not found" Sep 10 12:17:52.335439 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.335421 2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 10 12:17:52.335573 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.335479 2583 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 10 12:17:52.335644 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.335591 2583 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 10 12:17:52.335719 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.335706 2583 reconstruct.go:97] "Volume reconstruction finished" Sep 10 12:17:52.335719 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.335716 2583 reconciler.go:29] "Reconciler: start to sync state" Sep 10 12:17:52.335855 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.335824 2583 factory.go:55] Registering systemd factory Sep 10 12:17:52.335916 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.335877 2583 factory.go:223] Registration of the systemd container factory successfully Sep 10 12:17:52.336150 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.336136 2583 factory.go:153] Registering CRI-O factory Sep 10 12:17:52.336150 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.336150 2583 factory.go:223] Registration of the crio container factory successfully Sep 10 12:17:52.336254 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.336197 2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 10 12:17:52.336254 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.336214 2583 factory.go:103] Registering Raw factory Sep 10 12:17:52.336254 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.336228 2583 manager.go:1196] Started watching for new ooms in manager Sep 10 12:17:52.336556 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.336546 2583 manager.go:349] Starting recovery of all containers Sep 10 12:17:52.341751 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:52.341729 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 10 12:17:52.341913 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:52.341878 2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-110.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 10 12:17:52.342979 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:52.341733 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-110.ec2.internal.18d3f4d4ec166bf3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-110.ec2.internal,UID:ip-10-0-143-110.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-110.ec2.internal,},FirstTimestamp:2026-09-10 12:17:52.327355379 +0000 UTC m=+0.332574830,LastTimestamp:2026-09-10 12:17:52.327355379 +0000 UTC m=+0.332574830,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-110.ec2.internal,}" Sep 10 12:17:52.346546 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.346531 2583 manager.go:354] Recovery completed Sep 10 12:17:52.350181 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.350158 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4t8kr" Sep 10 12:17:52.351429 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.351413 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:17:52.352870 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.352854 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:17:52.352935 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.352880 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:17:52.352935 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.352889 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:17:52.353412 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.353399 2583 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 10 12:17:52.353412 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.353406 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:17:52.353412 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.353412 2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 10 12:17:52.353559 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.353428 2583 state_mem.go:36] "Initialized new in-memory state store" Sep 10 12:17:52.355793 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.355643 2583 policy_none.go:49] "None policy: Start" Sep 10 12:17:52.355793 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.355660 2583 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 10 12:17:52.355793 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.355672 2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 10 12:17:52.355793 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.355736 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:17:52.355793 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.355757 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:17:52.355793 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.355768 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:17:52.357788 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.357776 2583 policy_none.go:47] "Start" Sep 10 12:17:52.361641 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:52.361565 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-110.ec2.internal.18d3f4d4ed9bb808 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-110.ec2.internal,UID:ip-10-0-143-110.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-110.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-110.ec2.internal,},FirstTimestamp:2026-09-10 12:17:52.35286836 +0000 UTC m=+0.358087812,LastTimestamp:2026-09-10 12:17:52.35286836 +0000 UTC m=+0.358087812,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-110.ec2.internal,}" Sep 10 12:17:52.363145 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.363127 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4t8kr" Sep 10 12:17:52.406291 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.398369 2583 manager.go:337] "Starting Device Plugin manager" Sep 10 12:17:52.406291 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:52.398398 2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 10 12:17:52.406291 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.398407 2583 server.go:87] "Starting device plugin registration server" Sep 10 12:17:52.406291 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.398671 2583 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 10 12:17:52.406291 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.398682 2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 10 12:17:52.406291 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.398772 2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 10 12:17:52.406291 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.398855 2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 10 12:17:52.406291 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.398864 2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 10 12:17:52.406291 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:52.399295 2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 10 12:17:52.406291 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:52.399344 2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-110.ec2.internal\" not found" Sep 10 12:17:52.462699 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.462672 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 10 12:17:52.463813 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.463791 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 10 12:17:52.463923 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.463858 2583 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 10 12:17:52.463923 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.463889 2583 kubelet.go:2448] "Starting kubelet main sync loop" Sep 10 12:17:52.464011 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:52.463934 2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 10 12:17:52.466617 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.466588 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 12:17:52.498976 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.498949 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:17:52.499862 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.499829 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:17:52.499946 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.499876 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:17:52.499946 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.499891 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:17:52.499946 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.499919 2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-110.ec2.internal" Sep 10 12:17:52.512777 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.512712 2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-110.ec2.internal" Sep 10 12:17:52.512777 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:52.512733 2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-110.ec2.internal\": node \"ip-10-0-143-110.ec2.internal\" not found" Sep 10 12:17:52.531337 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:52.531319 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-110.ec2.internal\" not found" Sep 10 12:17:52.564341 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.564325 2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-143-110.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-110.ec2.internal"] Sep 10 12:17:52.564411 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.564378 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:17:52.565207 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.565195 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:17:52.565268 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.565219 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:17:52.565268 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.565228 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:17:52.567390 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.567379 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:17:52.567505 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.567491 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-110.ec2.internal" Sep 10 12:17:52.567542 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.567521 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:17:52.568062 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.568049 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:17:52.568145 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.568067 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:17:52.568145 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.568076 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:17:52.568145 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.568066 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:17:52.568260 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.568161 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:17:52.568260 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.568172 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:17:52.570108 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.570095 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-110.ec2.internal" Sep 10 12:17:52.570151 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.570118 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:17:52.570759 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.570742 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:17:52.570850 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.570776 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:17:52.570850 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.570792 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:17:52.601355 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:52.601339 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-110.ec2.internal\" not found" node="ip-10-0-143-110.ec2.internal" Sep 10 12:17:52.605748 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:52.605733 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-110.ec2.internal\" not found" node="ip-10-0-143-110.ec2.internal" Sep 10 12:17:52.631627 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:52.631610 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-110.ec2.internal\" not found" Sep 10 12:17:52.637917 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.637899 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de9e1cf5cc38daf6ca22018d91022ea0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-110.ec2.internal\" (UID: \"de9e1cf5cc38daf6ca22018d91022ea0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-110.ec2.internal" Sep 10 12:17:52.637964 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.637921 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/abdf679f5f7e1ecb5f64c1178ad16227-config\") pod \"kube-apiserver-proxy-ip-10-0-143-110.ec2.internal\" (UID: \"abdf679f5f7e1ecb5f64c1178ad16227\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-110.ec2.internal" Sep 10 12:17:52.637964 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.637937 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/de9e1cf5cc38daf6ca22018d91022ea0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-110.ec2.internal\" (UID: \"de9e1cf5cc38daf6ca22018d91022ea0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-110.ec2.internal" Sep 10 12:17:52.732630 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:52.732613 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-110.ec2.internal\" not found" Sep 10 12:17:52.738924 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.738904 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/abdf679f5f7e1ecb5f64c1178ad16227-config\") pod \"kube-apiserver-proxy-ip-10-0-143-110.ec2.internal\" (UID: \"abdf679f5f7e1ecb5f64c1178ad16227\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-110.ec2.internal" Sep 10 12:17:52.739000 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.738985 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/de9e1cf5cc38daf6ca22018d91022ea0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-110.ec2.internal\" (UID: \"de9e1cf5cc38daf6ca22018d91022ea0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-110.ec2.internal" Sep 10 12:17:52.739043 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.738999 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/abdf679f5f7e1ecb5f64c1178ad16227-config\") pod \"kube-apiserver-proxy-ip-10-0-143-110.ec2.internal\" (UID: \"abdf679f5f7e1ecb5f64c1178ad16227\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-110.ec2.internal" Sep 10 12:17:52.739043 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.739022 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de9e1cf5cc38daf6ca22018d91022ea0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-110.ec2.internal\" (UID: \"de9e1cf5cc38daf6ca22018d91022ea0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-110.ec2.internal" Sep 10 12:17:52.739043 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.738987 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/de9e1cf5cc38daf6ca22018d91022ea0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-110.ec2.internal\" (UID: \"de9e1cf5cc38daf6ca22018d91022ea0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-110.ec2.internal" Sep 10 12:17:52.739145 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.739052 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de9e1cf5cc38daf6ca22018d91022ea0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-110.ec2.internal\" (UID: \"de9e1cf5cc38daf6ca22018d91022ea0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-110.ec2.internal" Sep 10 12:17:52.833302 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:52.833263 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-110.ec2.internal\" not found" Sep 10 12:17:52.903843 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.903807 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-110.ec2.internal" Sep 10 12:17:52.908308 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:52.908240 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-110.ec2.internal" Sep 10 12:17:52.934481 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:52.934455 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-110.ec2.internal\" not found" Sep 10 12:17:53.036427 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:53.036384 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-110.ec2.internal\" not found" Sep 10 12:17:53.136865 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:53.136786 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-110.ec2.internal\" not found" Sep 10 12:17:53.236877 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:53.236830 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-110.ec2.internal\" not found" Sep 10 12:17:53.282570 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:53.282548 2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 10 12:17:53.283054 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:53.282736 2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 12:17:53.337244 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:53.337221 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-110.ec2.internal\" not found" Sep 10 12:17:53.365922 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:53.365879 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 12:12:52 +0000 UTC" deadline="2028-05-30 22:08:16.998789782 +0000 UTC" Sep 10 12:17:53.365922 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:53.365910 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15081h50m23.632884524s" Sep 10 12:17:53.388758 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:53.388709 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:17:53.393169 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:53.393152 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:17:53.393276 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:53.393183 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:17:53.393276 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:53.393194 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:17:53.437784 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:53.437750 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-110.ec2.internal\" not found" Sep 10 12:17:53.540314 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:53.540289 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-110.ec2.internal\" not found" Sep 10 12:17:53.641308 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:53.641247 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-110.ec2.internal\" not found" Sep 10 12:17:53.742403 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:53.742234 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-110.ec2.internal\" not found" Sep 10 12:17:53.782077 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:53.782054 2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 12:17:53.792179 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:53.792064 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabdf679f5f7e1ecb5f64c1178ad16227.slice/crio-0c499151eb4c74a63ba1eae141d5e4115ba7c9908e96447645dc32a01826f61d WatchSource:0}: Error finding container 0c499151eb4c74a63ba1eae141d5e4115ba7c9908e96447645dc32a01826f61d: Status 404 returned error can't find the container with id 0c499151eb4c74a63ba1eae141d5e4115ba7c9908e96447645dc32a01826f61d Sep 10 12:17:53.792475 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:53.792440 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde9e1cf5cc38daf6ca22018d91022ea0.slice/crio-15f55518e7ae338f800d07e36ddf5512775601b17a9e1fd4d76427727f36a37b WatchSource:0}: Error finding container 15f55518e7ae338f800d07e36ddf5512775601b17a9e1fd4d76427727f36a37b: Status 404 returned error can't find the container with id 15f55518e7ae338f800d07e36ddf5512775601b17a9e1fd4d76427727f36a37b Sep 10 12:17:53.796858 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:53.796842 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 12:17:53.831342 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:53.831317 2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 12:17:53.835720 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:53.835702 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-110.ec2.internal" Sep 10 12:17:53.846980 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:53.846963 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 12:17:53.848083 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:53.848070 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-110.ec2.internal" Sep 10 12:17:53.856884 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:53.856865 2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 12:17:53.860351 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:53.860337 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 12:17:54.245251 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.245225 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 12:17:54.319521 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.319491 2583 apiserver.go:52] "Watching apiserver" Sep 10 12:17:54.328414 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.328357 2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 10 12:17:54.330918 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.330888 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-4qqqh","kube-system/global-pull-secret-syncer-dgddq","kube-system/konnectivity-agent-qkhpj","kube-system/kube-apiserver-proxy-ip-10-0-143-110.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl","openshift-cluster-node-tuning-operator/tuned-vpqd2","openshift-dns/node-resolver-lm7qv","openshift-multus/multus-smplh","openshift-network-diagnostics/network-check-target-k9bgv","openshift-image-registry/node-ca-vqcn2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-110.ec2.internal","openshift-multus/multus-additional-cni-plugins-5fct5","openshift-multus/network-metrics-daemon-6j4mf","openshift-network-operator/iptables-alerter-wqv99"] Sep 10 12:17:54.333425 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.333400 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vqcn2" Sep 10 12:17:54.335419 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.335396 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 10 12:17:54.335752 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.335729 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:17:54.335840 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:54.335812 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j4mf" podUID="c6ad9571-3788-4c07-8778-c647b1a2cdce" Sep 10 12:17:54.336882 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.336130 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 10 12:17:54.336882 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.336229 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 10 12:17:54.336882 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.336248 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-brs5v\"" Sep 10 12:17:54.336882 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.336491 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 10 12:17:54.340783 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.338357 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qkhpj" Sep 10 12:17:54.341040 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.341022 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 10 12:17:54.341138 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.341116 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-j7smm\"" Sep 10 12:17:54.342102 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.341236 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 10 12:17:54.342102 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.341892 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wqv99" Sep 10 12:17:54.344303 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.344283 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 10 12:17:54.344403 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.344320 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 10 12:17:54.344722 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.344704 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 10 12:17:54.344804 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.344704 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ldw8g\"" Sep 10 12:17:54.346093 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.346071 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" Sep 10 12:17:54.347239 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.347217 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 12:17:54.348428 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.348395 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 10 12:17:54.348539 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.348427 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 10 12:17:54.348608 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.348574 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 10 12:17:54.348738 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.348719 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xqhmd\"" Sep 10 12:17:54.348822 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.348770 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a1521572-7823-459a-a435-0248c1ae1b12-host\") pod \"node-ca-vqcn2\" (UID: \"a1521572-7823-459a-a435-0248c1ae1b12\") " pod="openshift-image-registry/node-ca-vqcn2" Sep 10 12:17:54.348822 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.348797 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a1521572-7823-459a-a435-0248c1ae1b12-serviceca\") pod \"node-ca-vqcn2\" (UID: \"a1521572-7823-459a-a435-0248c1ae1b12\") " pod="openshift-image-registry/node-ca-vqcn2" Sep 10 12:17:54.348961 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.348828 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bm59\" (UniqueName: \"kubernetes.io/projected/a1521572-7823-459a-a435-0248c1ae1b12-kube-api-access-7bm59\") pod \"node-ca-vqcn2\" (UID: \"a1521572-7823-459a-a435-0248c1ae1b12\") " pod="openshift-image-registry/node-ca-vqcn2" Sep 10 12:17:54.348961 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.348855 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6ad9571-3788-4c07-8778-c647b1a2cdce-metrics-certs\") pod \"network-metrics-daemon-6j4mf\" (UID: \"c6ad9571-3788-4c07-8778-c647b1a2cdce\") " pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:17:54.348961 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.348903 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h98g\" (UniqueName: \"kubernetes.io/projected/c6ad9571-3788-4c07-8778-c647b1a2cdce-kube-api-access-9h98g\") pod \"network-metrics-daemon-6j4mf\" (UID: \"c6ad9571-3788-4c07-8778-c647b1a2cdce\") " pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:17:54.349389 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.349371 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.352118 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.352015 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:17:54.352118 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:54.352070 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9bgv" podUID="1aa57da6-29e7-4701-8b1b-be81cf4ebed6" Sep 10 12:17:54.352118 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.352089 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2zcc8\"" Sep 10 12:17:54.352362 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.352341 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 10 12:17:54.352440 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.352378 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 10 12:17:54.354490 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.354467 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lm7qv" Sep 10 12:17:54.356711 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.356568 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 10 12:17:54.356804 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.356744 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-l6jq4\"" Sep 10 12:17:54.356866 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.356803 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 10 12:17:54.357008 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.356992 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:17:54.357168 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:54.357115 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgddq" podUID="6960f97e-f623-4368-b27a-41d81ac1be1f" Sep 10 12:17:54.359647 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.359630 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.362659 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.362636 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 10 12:17:54.362659 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.362648 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 10 12:17:54.363037 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.363013 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 10 12:17:54.363126 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.363062 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 10 12:17:54.363308 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.363290 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-m9z4m\"" Sep 10 12:17:54.363548 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.363528 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 10 12:17:54.363631 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.363292 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-smplh" Sep 10 12:17:54.363690 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.363647 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 10 12:17:54.365153 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.365130 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5fct5" Sep 10 12:17:54.366025 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.366008 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 10 12:17:54.366291 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.366270 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 10 12:17:54.366547 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.366528 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 10 12:17:54.366643 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.366588 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-h8c5f\"" Sep 10 12:17:54.366705 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.366535 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 10 12:17:54.367270 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.367249 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5bknk\"" Sep 10 12:17:54.367550 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.367531 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 10 12:17:54.368038 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.368017 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 10 12:17:54.373520 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.373493 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-l5zcz" Sep 10 12:17:54.382784 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.382765 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-l5zcz" Sep 10 12:17:54.436738 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.436713 2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 10 12:17:54.449138 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449113 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bm59\" (UniqueName: \"kubernetes.io/projected/a1521572-7823-459a-a435-0248c1ae1b12-kube-api-access-7bm59\") pod \"node-ca-vqcn2\" (UID: \"a1521572-7823-459a-a435-0248c1ae1b12\") " pod="openshift-image-registry/node-ca-vqcn2" Sep 10 12:17:54.449255 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449154 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-etc-sysctl-conf\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.449255 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449181 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-multus-conf-dir\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.449255 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449205 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-host-run-multus-certs\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.449255 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449233 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xfzh\" (UniqueName: \"kubernetes.io/projected/adee8f79-7570-4cb5-be43-089a5a464107-kube-api-access-6xfzh\") pod \"iptables-alerter-wqv99\" (UID: \"adee8f79-7570-4cb5-be43-089a5a464107\") " pod="openshift-network-operator/iptables-alerter-wqv99" Sep 10 12:17:54.449478 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449267 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7244cbf-0a28-477e-9012-5dbe58beaa77-tmp-dir\") pod \"node-resolver-lm7qv\" (UID: \"f7244cbf-0a28-477e-9012-5dbe58beaa77\") " pod="openshift-dns/node-resolver-lm7qv" Sep 10 12:17:54.449478 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449292 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckl4q\" (UniqueName: \"kubernetes.io/projected/f7244cbf-0a28-477e-9012-5dbe58beaa77-kube-api-access-ckl4q\") pod \"node-resolver-lm7qv\" (UID: \"f7244cbf-0a28-477e-9012-5dbe58beaa77\") " pod="openshift-dns/node-resolver-lm7qv" Sep 10 12:17:54.449478 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449317 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-env-overrides\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.449478 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449345 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-ovnkube-script-lib\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.449478 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449370 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-multus-cni-dir\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.449478 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449405 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/57d3f064-15f5-4771-b729-cdae4f1ae208-konnectivity-ca\") pod \"konnectivity-agent-qkhpj\" (UID: \"57d3f064-15f5-4771-b729-cdae4f1ae208\") " pod="kube-system/konnectivity-agent-qkhpj" Sep 10 12:17:54.449478 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449429 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-host-kubelet\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.449478 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449452 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-run-ovn\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.449478 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449478 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-log-socket\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.449951 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449502 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-ovnkube-config\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.449951 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449527 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-hostroot\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.449951 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449551 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/adee8f79-7570-4cb5-be43-089a5a464107-iptables-alerter-script\") pod \"iptables-alerter-wqv99\" (UID: \"adee8f79-7570-4cb5-be43-089a5a464107\") " pod="openshift-network-operator/iptables-alerter-wqv99" Sep 10 12:17:54.449951 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449584 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/be18f526-aa95-4956-9d00-8e31388c5c26-tmp\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.449951 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449612 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a1521572-7823-459a-a435-0248c1ae1b12-host\") pod \"node-ca-vqcn2\" (UID: \"a1521572-7823-459a-a435-0248c1ae1b12\") " pod="openshift-image-registry/node-ca-vqcn2" Sep 10 12:17:54.449951 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449636 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-etc-modprobe-d\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.449951 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449660 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-etc-sysctl-d\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.449951 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449685 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-host-slash\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.449951 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449724 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h98g\" (UniqueName: \"kubernetes.io/projected/c6ad9571-3788-4c07-8778-c647b1a2cdce-kube-api-access-9h98g\") pod \"network-metrics-daemon-6j4mf\" (UID: \"c6ad9571-3788-4c07-8778-c647b1a2cdce\") " pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:17:54.449951 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449752 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3e79b263-6658-4ca2-8d1f-df5cfc8c24da-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vwvsl\" (UID: \"3e79b263-6658-4ca2-8d1f-df5cfc8c24da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" Sep 10 12:17:54.449951 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449778 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-host-run-netns\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.449951 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449813 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-host-var-lib-cni-bin\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.449951 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449848 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d34fb4ae-61a3-4533-a553-9dc4e2f6daaf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5fct5\" (UID: \"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf\") " pod="openshift-multus/multus-additional-cni-plugins-5fct5" Sep 10 12:17:54.449951 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449876 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3e79b263-6658-4ca2-8d1f-df5cfc8c24da-socket-dir\") pod \"aws-ebs-csi-driver-node-vwvsl\" (UID: \"3e79b263-6658-4ca2-8d1f-df5cfc8c24da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" Sep 10 12:17:54.449951 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449904 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr48g\" (UniqueName: \"kubernetes.io/projected/1aa57da6-29e7-4701-8b1b-be81cf4ebed6-kube-api-access-xr48g\") pod \"network-check-target-k9bgv\" (UID: \"1aa57da6-29e7-4701-8b1b-be81cf4ebed6\") " pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:17:54.449951 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449944 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-systemd-units\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.450737 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.449978 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-cnibin\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.450737 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450021 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-host-run-k8s-cni-cncf-io\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.450737 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450049 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a1521572-7823-459a-a435-0248c1ae1b12-serviceca\") pod \"node-ca-vqcn2\" (UID: \"a1521572-7823-459a-a435-0248c1ae1b12\") " pod="openshift-image-registry/node-ca-vqcn2" Sep 10 12:17:54.450737 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450073 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3e79b263-6658-4ca2-8d1f-df5cfc8c24da-sys-fs\") pod \"aws-ebs-csi-driver-node-vwvsl\" (UID: \"3e79b263-6658-4ca2-8d1f-df5cfc8c24da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" Sep 10 12:17:54.450737 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450098 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-lib-modules\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.450737 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450123 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6960f97e-f623-4368-b27a-41d81ac1be1f-kubelet-config\") pod \"global-pull-secret-syncer-dgddq\" (UID: \"6960f97e-f623-4368-b27a-41d81ac1be1f\") " pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:17:54.450737 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450151 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-multus-socket-dir-parent\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.450737 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450175 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/be18f526-aa95-4956-9d00-8e31388c5c26-etc-tuned\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.450737 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450213 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3e79b263-6658-4ca2-8d1f-df5cfc8c24da-device-dir\") pod \"aws-ebs-csi-driver-node-vwvsl\" (UID: \"3e79b263-6658-4ca2-8d1f-df5cfc8c24da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" Sep 10 12:17:54.450737 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450239 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-run\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.450737 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450239 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a1521572-7823-459a-a435-0248c1ae1b12-host\") pod \"node-ca-vqcn2\" (UID: \"a1521572-7823-459a-a435-0248c1ae1b12\") " pod="openshift-image-registry/node-ca-vqcn2" Sep 10 12:17:54.450737 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450262 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-host\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.450737 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450288 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-host-run-netns\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.450737 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450313 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-var-lib-openvswitch\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.450737 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450338 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-host-cni-bin\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.450737 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450363 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-system-cni-dir\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.450737 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450391 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-etc-systemd\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.451635 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450415 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-sys\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.451635 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450442 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf2dv\" (UniqueName: \"kubernetes.io/projected/be18f526-aa95-4956-9d00-8e31388c5c26-kube-api-access-tf2dv\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.451635 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450472 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f7244cbf-0a28-477e-9012-5dbe58beaa77-hosts-file\") pod \"node-resolver-lm7qv\" (UID: \"f7244cbf-0a28-477e-9012-5dbe58beaa77\") " pod="openshift-dns/node-resolver-lm7qv" Sep 10 12:17:54.451635 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450499 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6960f97e-f623-4368-b27a-41d81ac1be1f-dbus\") pod \"global-pull-secret-syncer-dgddq\" (UID: \"6960f97e-f623-4368-b27a-41d81ac1be1f\") " pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:17:54.451635 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450704 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-ovn-node-metrics-cert\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.451635 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450734 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d34fb4ae-61a3-4533-a553-9dc4e2f6daaf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5fct5\" (UID: \"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf\") " pod="openshift-multus/multus-additional-cni-plugins-5fct5" Sep 10 12:17:54.451635 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450757 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-var-lib-kubelet\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.451635 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450781 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d34fb4ae-61a3-4533-a553-9dc4e2f6daaf-system-cni-dir\") pod \"multus-additional-cni-plugins-5fct5\" (UID: \"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf\") " pod="openshift-multus/multus-additional-cni-plugins-5fct5" Sep 10 12:17:54.451635 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450806 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/adee8f79-7570-4cb5-be43-089a5a464107-host-slash\") pod \"iptables-alerter-wqv99\" (UID: \"adee8f79-7570-4cb5-be43-089a5a464107\") " pod="openshift-network-operator/iptables-alerter-wqv99" Sep 10 12:17:54.451635 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450830 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3e79b263-6658-4ca2-8d1f-df5cfc8c24da-registration-dir\") pod \"aws-ebs-csi-driver-node-vwvsl\" (UID: \"3e79b263-6658-4ca2-8d1f-df5cfc8c24da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" Sep 10 12:17:54.451635 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450854 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5wbt\" (UniqueName: \"kubernetes.io/projected/3e79b263-6658-4ca2-8d1f-df5cfc8c24da-kube-api-access-z5wbt\") pod \"aws-ebs-csi-driver-node-vwvsl\" (UID: \"3e79b263-6658-4ca2-8d1f-df5cfc8c24da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" Sep 10 12:17:54.451635 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450878 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-node-log\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.451635 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450904 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97xfj\" (UniqueName: \"kubernetes.io/projected/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-kube-api-access-97xfj\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.451635 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450964 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm6z9\" (UniqueName: \"kubernetes.io/projected/d34fb4ae-61a3-4533-a553-9dc4e2f6daaf-kube-api-access-hm6z9\") pod \"multus-additional-cni-plugins-5fct5\" (UID: \"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf\") " pod="openshift-multus/multus-additional-cni-plugins-5fct5" Sep 10 12:17:54.451635 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.450992 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.451635 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.451025 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-multus-daemon-config\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.452377 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.451057 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d34fb4ae-61a3-4533-a553-9dc4e2f6daaf-cnibin\") pod \"multus-additional-cni-plugins-5fct5\" (UID: \"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf\") " pod="openshift-multus/multus-additional-cni-plugins-5fct5" Sep 10 12:17:54.452377 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.451081 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/57d3f064-15f5-4771-b729-cdae4f1ae208-agent-certs\") pod \"konnectivity-agent-qkhpj\" (UID: \"57d3f064-15f5-4771-b729-cdae4f1ae208\") " pod="kube-system/konnectivity-agent-qkhpj" Sep 10 12:17:54.452377 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.451111 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6ad9571-3788-4c07-8778-c647b1a2cdce-metrics-certs\") pod \"network-metrics-daemon-6j4mf\" (UID: \"c6ad9571-3788-4c07-8778-c647b1a2cdce\") " pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:17:54.452377 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.451136 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-etc-sysconfig\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.452377 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.451163 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxl9w\" (UniqueName: \"kubernetes.io/projected/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-kube-api-access-lxl9w\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.452377 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.451202 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-cni-binary-copy\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.452377 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:54.451214 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:17:54.452377 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.451226 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d34fb4ae-61a3-4533-a553-9dc4e2f6daaf-cni-binary-copy\") pod \"multus-additional-cni-plugins-5fct5\" (UID: \"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf\") " pod="openshift-multus/multus-additional-cni-plugins-5fct5" Sep 10 12:17:54.452377 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.451251 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6960f97e-f623-4368-b27a-41d81ac1be1f-original-pull-secret\") pod \"global-pull-secret-syncer-dgddq\" (UID: \"6960f97e-f623-4368-b27a-41d81ac1be1f\") " pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:17:54.452377 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:54.451287 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6ad9571-3788-4c07-8778-c647b1a2cdce-metrics-certs podName:c6ad9571-3788-4c07-8778-c647b1a2cdce nodeName:}" failed. No retries permitted until 2026-09-10 12:17:54.951261882 +0000 UTC m=+2.956481339 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c6ad9571-3788-4c07-8778-c647b1a2cdce-metrics-certs") pod "network-metrics-daemon-6j4mf" (UID: "c6ad9571-3788-4c07-8778-c647b1a2cdce") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:17:54.452377 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.451311 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-run-systemd\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.452377 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.451343 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-etc-openvswitch\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.452377 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.451368 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-etc-kubernetes\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.452377 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.451391 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-run-openvswitch\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.452377 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.451415 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-host-run-ovn-kubernetes\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.452377 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.451440 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-host-cni-netd\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.453216 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.451463 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-host-var-lib-cni-multus\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.453216 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.451488 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-host-var-lib-kubelet\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.453216 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.451534 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3e79b263-6658-4ca2-8d1f-df5cfc8c24da-etc-selinux\") pod \"aws-ebs-csi-driver-node-vwvsl\" (UID: \"3e79b263-6658-4ca2-8d1f-df5cfc8c24da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" Sep 10 12:17:54.453216 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.451561 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a1521572-7823-459a-a435-0248c1ae1b12-serviceca\") pod \"node-ca-vqcn2\" (UID: \"a1521572-7823-459a-a435-0248c1ae1b12\") " pod="openshift-image-registry/node-ca-vqcn2" Sep 10 12:17:54.453216 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.451570 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-etc-kubernetes\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.453216 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.451600 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d34fb4ae-61a3-4533-a553-9dc4e2f6daaf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5fct5\" (UID: \"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf\") " pod="openshift-multus/multus-additional-cni-plugins-5fct5" Sep 10 12:17:54.457447 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.457421 2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 10 12:17:54.462319 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.462295 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bm59\" (UniqueName: \"kubernetes.io/projected/a1521572-7823-459a-a435-0248c1ae1b12-kube-api-access-7bm59\") pod \"node-ca-vqcn2\" (UID: \"a1521572-7823-459a-a435-0248c1ae1b12\") " pod="openshift-image-registry/node-ca-vqcn2" Sep 10 12:17:54.467604 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.467584 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h98g\" (UniqueName: \"kubernetes.io/projected/c6ad9571-3788-4c07-8778-c647b1a2cdce-kube-api-access-9h98g\") pod \"network-metrics-daemon-6j4mf\" (UID: \"c6ad9571-3788-4c07-8778-c647b1a2cdce\") " pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:17:54.471092 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.471050 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-110.ec2.internal" event={"ID":"abdf679f5f7e1ecb5f64c1178ad16227","Type":"ContainerStarted","Data":"0c499151eb4c74a63ba1eae141d5e4115ba7c9908e96447645dc32a01826f61d"} Sep 10 12:17:54.473448 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.473420 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-110.ec2.internal" event={"ID":"de9e1cf5cc38daf6ca22018d91022ea0","Type":"ContainerStarted","Data":"15f55518e7ae338f800d07e36ddf5512775601b17a9e1fd4d76427727f36a37b"} Sep 10 12:17:54.552520 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.552487 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-host-var-lib-cni-multus\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.552744 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.552527 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-host-var-lib-kubelet\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.552744 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.552551 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3e79b263-6658-4ca2-8d1f-df5cfc8c24da-etc-selinux\") pod \"aws-ebs-csi-driver-node-vwvsl\" (UID: \"3e79b263-6658-4ca2-8d1f-df5cfc8c24da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" Sep 10 12:17:54.552744 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.552572 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-etc-kubernetes\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.552744 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.552608 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d34fb4ae-61a3-4533-a553-9dc4e2f6daaf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5fct5\" (UID: \"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf\") " pod="openshift-multus/multus-additional-cni-plugins-5fct5" Sep 10 12:17:54.552744 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.552635 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-etc-sysctl-conf\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.552744 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.552659 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-multus-conf-dir\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.552744 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.552683 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-host-run-multus-certs\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.552744 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.552708 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xfzh\" (UniqueName: \"kubernetes.io/projected/adee8f79-7570-4cb5-be43-089a5a464107-kube-api-access-6xfzh\") pod \"iptables-alerter-wqv99\" (UID: \"adee8f79-7570-4cb5-be43-089a5a464107\") " pod="openshift-network-operator/iptables-alerter-wqv99" Sep 10 12:17:54.552744 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.552732 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7244cbf-0a28-477e-9012-5dbe58beaa77-tmp-dir\") pod \"node-resolver-lm7qv\" (UID: \"f7244cbf-0a28-477e-9012-5dbe58beaa77\") " pod="openshift-dns/node-resolver-lm7qv" Sep 10 12:17:54.553281 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.552755 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckl4q\" (UniqueName: \"kubernetes.io/projected/f7244cbf-0a28-477e-9012-5dbe58beaa77-kube-api-access-ckl4q\") pod \"node-resolver-lm7qv\" (UID: \"f7244cbf-0a28-477e-9012-5dbe58beaa77\") " pod="openshift-dns/node-resolver-lm7qv" Sep 10 12:17:54.553281 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.552780 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-env-overrides\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.553281 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.552806 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-ovnkube-script-lib\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.553281 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.552831 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-multus-cni-dir\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.553281 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.552857 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/57d3f064-15f5-4771-b729-cdae4f1ae208-konnectivity-ca\") pod \"konnectivity-agent-qkhpj\" (UID: \"57d3f064-15f5-4771-b729-cdae4f1ae208\") " pod="kube-system/konnectivity-agent-qkhpj" Sep 10 12:17:54.553281 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.552902 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-host-kubelet\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.553281 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.552927 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-run-ovn\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.553281 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.552956 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-log-socket\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.553281 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.552981 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-ovnkube-config\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.553281 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553008 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-hostroot\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.553281 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553076 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/adee8f79-7570-4cb5-be43-089a5a464107-iptables-alerter-script\") pod \"iptables-alerter-wqv99\" (UID: \"adee8f79-7570-4cb5-be43-089a5a464107\") " pod="openshift-network-operator/iptables-alerter-wqv99" Sep 10 12:17:54.553281 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553103 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/be18f526-aa95-4956-9d00-8e31388c5c26-tmp\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.553281 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553156 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-etc-modprobe-d\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.553281 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553183 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-etc-sysctl-d\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.553281 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553208 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-host-slash\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.553281 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553236 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3e79b263-6658-4ca2-8d1f-df5cfc8c24da-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vwvsl\" (UID: \"3e79b263-6658-4ca2-8d1f-df5cfc8c24da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" Sep 10 12:17:54.553281 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553269 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-host-run-netns\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.554192 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553293 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-host-var-lib-cni-bin\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.554192 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553318 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d34fb4ae-61a3-4533-a553-9dc4e2f6daaf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5fct5\" (UID: \"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf\") " pod="openshift-multus/multus-additional-cni-plugins-5fct5" Sep 10 12:17:54.554192 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553345 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3e79b263-6658-4ca2-8d1f-df5cfc8c24da-socket-dir\") pod \"aws-ebs-csi-driver-node-vwvsl\" (UID: \"3e79b263-6658-4ca2-8d1f-df5cfc8c24da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" Sep 10 12:17:54.554192 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553371 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr48g\" (UniqueName: \"kubernetes.io/projected/1aa57da6-29e7-4701-8b1b-be81cf4ebed6-kube-api-access-xr48g\") pod \"network-check-target-k9bgv\" (UID: \"1aa57da6-29e7-4701-8b1b-be81cf4ebed6\") " pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:17:54.554192 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553401 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-systemd-units\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.554192 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553427 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-cnibin\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.554192 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553458 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-host-run-k8s-cni-cncf-io\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.554192 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553485 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3e79b263-6658-4ca2-8d1f-df5cfc8c24da-sys-fs\") pod \"aws-ebs-csi-driver-node-vwvsl\" (UID: \"3e79b263-6658-4ca2-8d1f-df5cfc8c24da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" Sep 10 12:17:54.554192 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553509 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-lib-modules\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.554192 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553535 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6960f97e-f623-4368-b27a-41d81ac1be1f-kubelet-config\") pod \"global-pull-secret-syncer-dgddq\" (UID: \"6960f97e-f623-4368-b27a-41d81ac1be1f\") " pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:17:54.554192 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553562 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-multus-socket-dir-parent\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.554192 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553587 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/be18f526-aa95-4956-9d00-8e31388c5c26-etc-tuned\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.554192 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553613 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3e79b263-6658-4ca2-8d1f-df5cfc8c24da-device-dir\") pod \"aws-ebs-csi-driver-node-vwvsl\" (UID: \"3e79b263-6658-4ca2-8d1f-df5cfc8c24da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" Sep 10 12:17:54.554192 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553638 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-run\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.554192 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553661 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-host\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.554192 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553687 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-host-run-netns\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.554192 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553713 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-var-lib-openvswitch\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.555037 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553739 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-host-cni-bin\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.555037 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553764 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-system-cni-dir\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.555037 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553790 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-etc-systemd\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.555037 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553822 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-sys\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.555037 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553848 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf2dv\" (UniqueName: \"kubernetes.io/projected/be18f526-aa95-4956-9d00-8e31388c5c26-kube-api-access-tf2dv\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.555037 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553874 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f7244cbf-0a28-477e-9012-5dbe58beaa77-hosts-file\") pod \"node-resolver-lm7qv\" (UID: \"f7244cbf-0a28-477e-9012-5dbe58beaa77\") " pod="openshift-dns/node-resolver-lm7qv" Sep 10 12:17:54.555037 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553900 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6960f97e-f623-4368-b27a-41d81ac1be1f-dbus\") pod \"global-pull-secret-syncer-dgddq\" (UID: \"6960f97e-f623-4368-b27a-41d81ac1be1f\") " pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:17:54.555037 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553946 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-ovn-node-metrics-cert\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.555037 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.553974 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d34fb4ae-61a3-4533-a553-9dc4e2f6daaf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5fct5\" (UID: \"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf\") " pod="openshift-multus/multus-additional-cni-plugins-5fct5" Sep 10 12:17:54.555037 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554009 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-var-lib-kubelet\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.555037 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554037 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d34fb4ae-61a3-4533-a553-9dc4e2f6daaf-system-cni-dir\") pod \"multus-additional-cni-plugins-5fct5\" (UID: \"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf\") " pod="openshift-multus/multus-additional-cni-plugins-5fct5" Sep 10 12:17:54.555037 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554065 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/adee8f79-7570-4cb5-be43-089a5a464107-host-slash\") pod \"iptables-alerter-wqv99\" (UID: \"adee8f79-7570-4cb5-be43-089a5a464107\") " pod="openshift-network-operator/iptables-alerter-wqv99" Sep 10 12:17:54.555037 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554092 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3e79b263-6658-4ca2-8d1f-df5cfc8c24da-registration-dir\") pod \"aws-ebs-csi-driver-node-vwvsl\" (UID: \"3e79b263-6658-4ca2-8d1f-df5cfc8c24da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" Sep 10 12:17:54.555037 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554121 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5wbt\" (UniqueName: \"kubernetes.io/projected/3e79b263-6658-4ca2-8d1f-df5cfc8c24da-kube-api-access-z5wbt\") pod \"aws-ebs-csi-driver-node-vwvsl\" (UID: \"3e79b263-6658-4ca2-8d1f-df5cfc8c24da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" Sep 10 12:17:54.555037 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554148 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-node-log\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.555037 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554204 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97xfj\" (UniqueName: \"kubernetes.io/projected/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-kube-api-access-97xfj\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.555037 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554234 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm6z9\" (UniqueName: \"kubernetes.io/projected/d34fb4ae-61a3-4533-a553-9dc4e2f6daaf-kube-api-access-hm6z9\") pod \"multus-additional-cni-plugins-5fct5\" (UID: \"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf\") " pod="openshift-multus/multus-additional-cni-plugins-5fct5" Sep 10 12:17:54.556003 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554265 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.556003 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554297 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-multus-daemon-config\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.556003 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554326 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d34fb4ae-61a3-4533-a553-9dc4e2f6daaf-cnibin\") pod \"multus-additional-cni-plugins-5fct5\" (UID: \"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf\") " pod="openshift-multus/multus-additional-cni-plugins-5fct5" Sep 10 12:17:54.556003 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554353 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/57d3f064-15f5-4771-b729-cdae4f1ae208-agent-certs\") pod \"konnectivity-agent-qkhpj\" (UID: \"57d3f064-15f5-4771-b729-cdae4f1ae208\") " pod="kube-system/konnectivity-agent-qkhpj" Sep 10 12:17:54.556003 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554397 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-etc-sysconfig\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.556003 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554425 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxl9w\" (UniqueName: \"kubernetes.io/projected/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-kube-api-access-lxl9w\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.556003 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554467 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-cni-binary-copy\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.556003 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554496 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d34fb4ae-61a3-4533-a553-9dc4e2f6daaf-cni-binary-copy\") pod \"multus-additional-cni-plugins-5fct5\" (UID: \"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf\") " pod="openshift-multus/multus-additional-cni-plugins-5fct5" Sep 10 12:17:54.556003 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554526 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6960f97e-f623-4368-b27a-41d81ac1be1f-original-pull-secret\") pod \"global-pull-secret-syncer-dgddq\" (UID: \"6960f97e-f623-4368-b27a-41d81ac1be1f\") " pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:17:54.556003 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554554 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-run-systemd\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.556003 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554582 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-etc-openvswitch\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.556003 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554609 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-etc-kubernetes\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.556003 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554637 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-run-openvswitch\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.556003 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554664 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-host-run-ovn-kubernetes\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.556003 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554691 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-host-cni-netd\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.556003 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554803 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-host-cni-netd\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.556003 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554853 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-host-var-lib-cni-multus\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.556903 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.554905 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-host-var-lib-kubelet\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.556903 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.555038 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3e79b263-6658-4ca2-8d1f-df5cfc8c24da-etc-selinux\") pod \"aws-ebs-csi-driver-node-vwvsl\" (UID: \"3e79b263-6658-4ca2-8d1f-df5cfc8c24da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" Sep 10 12:17:54.556903 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.555083 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-etc-kubernetes\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.556903 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.555663 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d34fb4ae-61a3-4533-a553-9dc4e2f6daaf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5fct5\" (UID: \"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf\") " pod="openshift-multus/multus-additional-cni-plugins-5fct5" Sep 10 12:17:54.556903 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.555819 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-etc-sysctl-conf\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.556903 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.555871 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-multus-conf-dir\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.556903 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.555934 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-host-run-multus-certs\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.556903 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.556448 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7244cbf-0a28-477e-9012-5dbe58beaa77-tmp-dir\") pod \"node-resolver-lm7qv\" (UID: \"f7244cbf-0a28-477e-9012-5dbe58beaa77\") " pod="openshift-dns/node-resolver-lm7qv" Sep 10 12:17:54.557398 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.557139 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-env-overrides\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.557680 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.557656 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-ovnkube-script-lib\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.557906 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.557887 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-multus-cni-dir\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.558434 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.558410 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/57d3f064-15f5-4771-b729-cdae4f1ae208-konnectivity-ca\") pod \"konnectivity-agent-qkhpj\" (UID: \"57d3f064-15f5-4771-b729-cdae4f1ae208\") " pod="kube-system/konnectivity-agent-qkhpj" Sep 10 12:17:54.558521 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.558482 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-host-kubelet\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.558584 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.558531 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-run-ovn\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.558584 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.558578 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-log-socket\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.559069 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.559046 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-ovnkube-config\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.559163 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.559115 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-hostroot\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.560433 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.560411 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-sys\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.560875 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.560853 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.561147 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.561127 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f7244cbf-0a28-477e-9012-5dbe58beaa77-hosts-file\") pod \"node-resolver-lm7qv\" (UID: \"f7244cbf-0a28-477e-9012-5dbe58beaa77\") " pod="openshift-dns/node-resolver-lm7qv" Sep 10 12:17:54.561234 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.561201 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6960f97e-f623-4368-b27a-41d81ac1be1f-dbus\") pod \"global-pull-secret-syncer-dgddq\" (UID: \"6960f97e-f623-4368-b27a-41d81ac1be1f\") " pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:17:54.564410 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.562122 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3e79b263-6658-4ca2-8d1f-df5cfc8c24da-registration-dir\") pod \"aws-ebs-csi-driver-node-vwvsl\" (UID: \"3e79b263-6658-4ca2-8d1f-df5cfc8c24da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" Sep 10 12:17:54.564410 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.562132 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-var-lib-kubelet\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.564410 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.562154 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d34fb4ae-61a3-4533-a553-9dc4e2f6daaf-system-cni-dir\") pod \"multus-additional-cni-plugins-5fct5\" (UID: \"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf\") " pod="openshift-multus/multus-additional-cni-plugins-5fct5" Sep 10 12:17:54.564410 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.562195 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/adee8f79-7570-4cb5-be43-089a5a464107-host-slash\") pod \"iptables-alerter-wqv99\" (UID: \"adee8f79-7570-4cb5-be43-089a5a464107\") " pod="openshift-network-operator/iptables-alerter-wqv99" Sep 10 12:17:54.564410 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.562205 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-node-log\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.564410 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.562264 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3e79b263-6658-4ca2-8d1f-df5cfc8c24da-sys-fs\") pod \"aws-ebs-csi-driver-node-vwvsl\" (UID: \"3e79b263-6658-4ca2-8d1f-df5cfc8c24da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" Sep 10 12:17:54.564410 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.562339 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-etc-sysctl-d\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.564410 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.562393 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-host-slash\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.564410 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.562444 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3e79b263-6658-4ca2-8d1f-df5cfc8c24da-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vwvsl\" (UID: \"3e79b263-6658-4ca2-8d1f-df5cfc8c24da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" Sep 10 12:17:54.564410 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.562490 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-host-run-netns\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.564410 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.562532 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-host-var-lib-cni-bin\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.564410 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.562537 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-etc-modprobe-d\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.564410 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.562599 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d34fb4ae-61a3-4533-a553-9dc4e2f6daaf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5fct5\" (UID: \"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf\") " pod="openshift-multus/multus-additional-cni-plugins-5fct5" Sep 10 12:17:54.564410 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.562708 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d34fb4ae-61a3-4533-a553-9dc4e2f6daaf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5fct5\" (UID: \"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf\") " pod="openshift-multus/multus-additional-cni-plugins-5fct5" Sep 10 12:17:54.564410 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.562773 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-host-run-k8s-cni-cncf-io\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.564410 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.562796 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-cnibin\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.564410 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.562818 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3e79b263-6658-4ca2-8d1f-df5cfc8c24da-socket-dir\") pod \"aws-ebs-csi-driver-node-vwvsl\" (UID: \"3e79b263-6658-4ca2-8d1f-df5cfc8c24da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" Sep 10 12:17:54.565247 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.562849 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-systemd-units\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.565247 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.563581 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d34fb4ae-61a3-4533-a553-9dc4e2f6daaf-cni-binary-copy\") pod \"multus-additional-cni-plugins-5fct5\" (UID: \"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf\") " pod="openshift-multus/multus-additional-cni-plugins-5fct5" Sep 10 12:17:54.565247 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.563663 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-etc-sysconfig\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.565247 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.565146 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-etc-openvswitch\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.569605 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.569583 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-cni-binary-copy\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.570088 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.570068 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/adee8f79-7570-4cb5-be43-089a5a464107-iptables-alerter-script\") pod \"iptables-alerter-wqv99\" (UID: \"adee8f79-7570-4cb5-be43-089a5a464107\") " pod="openshift-network-operator/iptables-alerter-wqv99" Sep 10 12:17:54.573261 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.571195 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-etc-kubernetes\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.573261 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.571336 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-run-openvswitch\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.573261 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.571449 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-host-run-ovn-kubernetes\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.573261 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.572466 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-multus-daemon-config\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.573261 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.572581 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d34fb4ae-61a3-4533-a553-9dc4e2f6daaf-cnibin\") pod \"multus-additional-cni-plugins-5fct5\" (UID: \"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf\") " pod="openshift-multus/multus-additional-cni-plugins-5fct5" Sep 10 12:17:54.573261 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.572659 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-run-systemd\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.573261 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:54.572773 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:17:54.573261 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:54.572840 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6960f97e-f623-4368-b27a-41d81ac1be1f-original-pull-secret podName:6960f97e-f623-4368-b27a-41d81ac1be1f nodeName:}" failed. No retries permitted until 2026-09-10 12:17:55.072819382 +0000 UTC m=+3.078038840 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6960f97e-f623-4368-b27a-41d81ac1be1f-original-pull-secret") pod "global-pull-secret-syncer-dgddq" (UID: "6960f97e-f623-4368-b27a-41d81ac1be1f") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:17:54.573739 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.573535 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/be18f526-aa95-4956-9d00-8e31388c5c26-tmp\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.573739 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.573687 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckl4q\" (UniqueName: \"kubernetes.io/projected/f7244cbf-0a28-477e-9012-5dbe58beaa77-kube-api-access-ckl4q\") pod \"node-resolver-lm7qv\" (UID: \"f7244cbf-0a28-477e-9012-5dbe58beaa77\") " pod="openshift-dns/node-resolver-lm7qv" Sep 10 12:17:54.573855 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.573803 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-ovn-node-metrics-cert\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.574446 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.574006 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-run\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.574446 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.574065 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/57d3f064-15f5-4771-b729-cdae4f1ae208-agent-certs\") pod \"konnectivity-agent-qkhpj\" (UID: \"57d3f064-15f5-4771-b729-cdae4f1ae208\") " pod="kube-system/konnectivity-agent-qkhpj" Sep 10 12:17:54.574446 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.574143 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-lib-modules\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.574446 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.574149 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-multus-socket-dir-parent\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.574446 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.574211 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6960f97e-f623-4368-b27a-41d81ac1be1f-kubelet-config\") pod \"global-pull-secret-syncer-dgddq\" (UID: \"6960f97e-f623-4368-b27a-41d81ac1be1f\") " pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:17:54.574446 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.574274 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3e79b263-6658-4ca2-8d1f-df5cfc8c24da-device-dir\") pod \"aws-ebs-csi-driver-node-vwvsl\" (UID: \"3e79b263-6658-4ca2-8d1f-df5cfc8c24da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" Sep 10 12:17:54.574446 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.574329 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-var-lib-openvswitch\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.574446 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.574396 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-host\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.574446 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.574447 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-host-run-netns\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.574978 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:54.574563 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:17:54.574978 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:54.574586 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:17:54.574978 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:54.574599 2583 projected.go:196] Error preparing data for projected volume kube-api-access-xr48g for pod openshift-network-diagnostics/network-check-target-k9bgv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:17:54.574978 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:54.574669 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1aa57da6-29e7-4701-8b1b-be81cf4ebed6-kube-api-access-xr48g podName:1aa57da6-29e7-4701-8b1b-be81cf4ebed6 nodeName:}" failed. No retries permitted until 2026-09-10 12:17:55.07464481 +0000 UTC m=+3.079864271 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xr48g" (UniqueName: "kubernetes.io/projected/1aa57da6-29e7-4701-8b1b-be81cf4ebed6-kube-api-access-xr48g") pod "network-check-target-k9bgv" (UID: "1aa57da6-29e7-4701-8b1b-be81cf4ebed6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:17:54.576471 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.576010 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-system-cni-dir\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.576471 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.576079 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-host-cni-bin\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.576471 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.576141 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/be18f526-aa95-4956-9d00-8e31388c5c26-etc-systemd\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.577143 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.577114 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5wbt\" (UniqueName: \"kubernetes.io/projected/3e79b263-6658-4ca2-8d1f-df5cfc8c24da-kube-api-access-z5wbt\") pod \"aws-ebs-csi-driver-node-vwvsl\" (UID: \"3e79b263-6658-4ca2-8d1f-df5cfc8c24da\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" Sep 10 12:17:54.579819 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.579243 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/be18f526-aa95-4956-9d00-8e31388c5c26-etc-tuned\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.579819 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.579267 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf2dv\" (UniqueName: \"kubernetes.io/projected/be18f526-aa95-4956-9d00-8e31388c5c26-kube-api-access-tf2dv\") pod \"tuned-vpqd2\" (UID: \"be18f526-aa95-4956-9d00-8e31388c5c26\") " pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.579819 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.579617 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97xfj\" (UniqueName: \"kubernetes.io/projected/3ab78f8a-127b-4d1a-9d6f-0b9e06251841-kube-api-access-97xfj\") pod \"multus-smplh\" (UID: \"3ab78f8a-127b-4d1a-9d6f-0b9e06251841\") " pod="openshift-multus/multus-smplh" Sep 10 12:17:54.579819 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.579697 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xfzh\" (UniqueName: \"kubernetes.io/projected/adee8f79-7570-4cb5-be43-089a5a464107-kube-api-access-6xfzh\") pod \"iptables-alerter-wqv99\" (UID: \"adee8f79-7570-4cb5-be43-089a5a464107\") " pod="openshift-network-operator/iptables-alerter-wqv99" Sep 10 12:17:54.581887 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.581819 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm6z9\" (UniqueName: \"kubernetes.io/projected/d34fb4ae-61a3-4533-a553-9dc4e2f6daaf-kube-api-access-hm6z9\") pod \"multus-additional-cni-plugins-5fct5\" (UID: \"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf\") " pod="openshift-multus/multus-additional-cni-plugins-5fct5" Sep 10 12:17:54.583071 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.583029 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxl9w\" (UniqueName: \"kubernetes.io/projected/7bc3c1d2-e9d8-4fd4-bd94-878d61758b97-kube-api-access-lxl9w\") pod \"ovnkube-node-4qqqh\" (UID: \"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97\") " pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.645466 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.645438 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vqcn2" Sep 10 12:17:54.655663 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.655637 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qkhpj" Sep 10 12:17:54.666003 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.665981 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wqv99" Sep 10 12:17:54.673276 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.673249 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" Sep 10 12:17:54.676238 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:54.676208 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57d3f064_15f5_4771_b729_cdae4f1ae208.slice/crio-320e92149ebac40c4987ab3b64f7df14c0c6992d06b0f8d31c0bf67026ea01e9 WatchSource:0}: Error finding container 320e92149ebac40c4987ab3b64f7df14c0c6992d06b0f8d31c0bf67026ea01e9: Status 404 returned error can't find the container with id 320e92149ebac40c4987ab3b64f7df14c0c6992d06b0f8d31c0bf67026ea01e9 Sep 10 12:17:54.683533 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.683286 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" Sep 10 12:17:54.691077 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.691046 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lm7qv" Sep 10 12:17:54.694620 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:54.694591 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e79b263_6658_4ca2_8d1f_df5cfc8c24da.slice/crio-361aa9d4d939d236f4d1c78e67839b462e32a51471bdfb2270b929343cf28121 WatchSource:0}: Error finding container 361aa9d4d939d236f4d1c78e67839b462e32a51471bdfb2270b929343cf28121: Status 404 returned error can't find the container with id 361aa9d4d939d236f4d1c78e67839b462e32a51471bdfb2270b929343cf28121 Sep 10 12:17:54.698704 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.698683 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:17:54.700102 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:54.700078 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe18f526_aa95_4956_9d00_8e31388c5c26.slice/crio-5be6b03bfacaf4a6aa42a3781b61fb22255b06281587e161d5e81c5fb58253c1 WatchSource:0}: Error finding container 5be6b03bfacaf4a6aa42a3781b61fb22255b06281587e161d5e81c5fb58253c1: Status 404 returned error can't find the container with id 5be6b03bfacaf4a6aa42a3781b61fb22255b06281587e161d5e81c5fb58253c1 Sep 10 12:17:54.707263 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.707240 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-smplh" Sep 10 12:17:54.714978 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:54.714956 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7244cbf_0a28_477e_9012_5dbe58beaa77.slice/crio-03bae9927cefbcdae8088854811c3eb51bb7a9b503d11190994d0321777f2985 WatchSource:0}: Error finding container 03bae9927cefbcdae8088854811c3eb51bb7a9b503d11190994d0321777f2985: Status 404 returned error can't find the container with id 03bae9927cefbcdae8088854811c3eb51bb7a9b503d11190994d0321777f2985 Sep 10 12:17:54.715825 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.715807 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5fct5" Sep 10 12:17:54.718227 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:54.718204 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bc3c1d2_e9d8_4fd4_bd94_878d61758b97.slice/crio-9dddd01e22cefe123819a98c619d3997d15b4faf777c76d0e17fa8bb5eb9c7ce WatchSource:0}: Error finding container 9dddd01e22cefe123819a98c619d3997d15b4faf777c76d0e17fa8bb5eb9c7ce: Status 404 returned error can't find the container with id 9dddd01e22cefe123819a98c619d3997d15b4faf777c76d0e17fa8bb5eb9c7ce Sep 10 12:17:54.725835 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:17:54.725734 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ab78f8a_127b_4d1a_9d6f_0b9e06251841.slice/crio-10f6320f59a823f0936fd079b56b20ec60aa494d8dabf666f9dcc55de05f727b WatchSource:0}: Error finding container 10f6320f59a823f0936fd079b56b20ec60aa494d8dabf666f9dcc55de05f727b: Status 404 returned error can't find the container with id 10f6320f59a823f0936fd079b56b20ec60aa494d8dabf666f9dcc55de05f727b Sep 10 12:17:54.959367 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:54.959292 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6ad9571-3788-4c07-8778-c647b1a2cdce-metrics-certs\") pod \"network-metrics-daemon-6j4mf\" (UID: \"c6ad9571-3788-4c07-8778-c647b1a2cdce\") " pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:17:54.959517 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:54.959418 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:17:54.959517 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:54.959495 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6ad9571-3788-4c07-8778-c647b1a2cdce-metrics-certs podName:c6ad9571-3788-4c07-8778-c647b1a2cdce nodeName:}" failed. No retries permitted until 2026-09-10 12:17:55.959473901 +0000 UTC m=+3.964693390 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c6ad9571-3788-4c07-8778-c647b1a2cdce-metrics-certs") pod "network-metrics-daemon-6j4mf" (UID: "c6ad9571-3788-4c07-8778-c647b1a2cdce") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:17:55.160432 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:55.160395 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr48g\" (UniqueName: \"kubernetes.io/projected/1aa57da6-29e7-4701-8b1b-be81cf4ebed6-kube-api-access-xr48g\") pod \"network-check-target-k9bgv\" (UID: \"1aa57da6-29e7-4701-8b1b-be81cf4ebed6\") " pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:17:55.160644 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:55.160487 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6960f97e-f623-4368-b27a-41d81ac1be1f-original-pull-secret\") pod \"global-pull-secret-syncer-dgddq\" (UID: \"6960f97e-f623-4368-b27a-41d81ac1be1f\") " pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:17:55.160644 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:55.160595 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:17:55.160644 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:55.160623 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:17:55.160644 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:55.160637 2583 projected.go:196] Error preparing data for projected volume kube-api-access-xr48g for pod openshift-network-diagnostics/network-check-target-k9bgv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:17:55.160849 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:55.160707 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1aa57da6-29e7-4701-8b1b-be81cf4ebed6-kube-api-access-xr48g podName:1aa57da6-29e7-4701-8b1b-be81cf4ebed6 nodeName:}" failed. No retries permitted until 2026-09-10 12:17:56.160684729 +0000 UTC m=+4.165904214 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xr48g" (UniqueName: "kubernetes.io/projected/1aa57da6-29e7-4701-8b1b-be81cf4ebed6-kube-api-access-xr48g") pod "network-check-target-k9bgv" (UID: "1aa57da6-29e7-4701-8b1b-be81cf4ebed6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:17:55.160849 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:55.160635 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:17:55.160849 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:55.160782 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6960f97e-f623-4368-b27a-41d81ac1be1f-original-pull-secret podName:6960f97e-f623-4368-b27a-41d81ac1be1f nodeName:}" failed. No retries permitted until 2026-09-10 12:17:56.160765294 +0000 UTC m=+4.165984749 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6960f97e-f623-4368-b27a-41d81ac1be1f-original-pull-secret") pod "global-pull-secret-syncer-dgddq" (UID: "6960f97e-f623-4368-b27a-41d81ac1be1f") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:17:55.386825 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:55.386772 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 12:12:54 +0000 UTC" deadline="2028-02-05 08:09:42.287946907 +0000 UTC" Sep 10 12:17:55.386825 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:55.386810 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12307h51m46.901141369s" Sep 10 12:17:55.480363 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:55.480213 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" event={"ID":"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97","Type":"ContainerStarted","Data":"9dddd01e22cefe123819a98c619d3997d15b4faf777c76d0e17fa8bb5eb9c7ce"} Sep 10 12:17:55.483857 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:55.483807 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" event={"ID":"be18f526-aa95-4956-9d00-8e31388c5c26","Type":"ContainerStarted","Data":"5be6b03bfacaf4a6aa42a3781b61fb22255b06281587e161d5e81c5fb58253c1"} Sep 10 12:17:55.485807 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:55.485782 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5fct5" event={"ID":"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf","Type":"ContainerStarted","Data":"9fbb95af96b51ed21bc069cd135654cc6f64e4e5ccf0a1df68ec772fa1ccf075"} Sep 10 12:17:55.487321 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:55.487288 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-smplh" event={"ID":"3ab78f8a-127b-4d1a-9d6f-0b9e06251841","Type":"ContainerStarted","Data":"10f6320f59a823f0936fd079b56b20ec60aa494d8dabf666f9dcc55de05f727b"} Sep 10 12:17:55.490872 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:55.490843 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lm7qv" event={"ID":"f7244cbf-0a28-477e-9012-5dbe58beaa77","Type":"ContainerStarted","Data":"03bae9927cefbcdae8088854811c3eb51bb7a9b503d11190994d0321777f2985"} Sep 10 12:17:55.493825 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:55.493795 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" event={"ID":"3e79b263-6658-4ca2-8d1f-df5cfc8c24da","Type":"ContainerStarted","Data":"361aa9d4d939d236f4d1c78e67839b462e32a51471bdfb2270b929343cf28121"} Sep 10 12:17:55.496458 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:55.496409 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wqv99" event={"ID":"adee8f79-7570-4cb5-be43-089a5a464107","Type":"ContainerStarted","Data":"0b8d0d19c9b2c78276c63a0448fb234046d38fd10d35da8418f7bf6f09d135d2"} Sep 10 12:17:55.504661 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:55.504607 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qkhpj" event={"ID":"57d3f064-15f5-4771-b729-cdae4f1ae208","Type":"ContainerStarted","Data":"320e92149ebac40c4987ab3b64f7df14c0c6992d06b0f8d31c0bf67026ea01e9"} Sep 10 12:17:55.506510 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:55.506486 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vqcn2" event={"ID":"a1521572-7823-459a-a435-0248c1ae1b12","Type":"ContainerStarted","Data":"9571b41ef1ca5cc94f02e2e28d4971ed5a2aebd52a4c7a159fc1a30e3b4ec7ea"} Sep 10 12:17:55.967432 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:55.967392 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6ad9571-3788-4c07-8778-c647b1a2cdce-metrics-certs\") pod \"network-metrics-daemon-6j4mf\" (UID: \"c6ad9571-3788-4c07-8778-c647b1a2cdce\") " pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:17:55.967613 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:55.967562 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:17:55.967702 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:55.967631 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6ad9571-3788-4c07-8778-c647b1a2cdce-metrics-certs podName:c6ad9571-3788-4c07-8778-c647b1a2cdce nodeName:}" failed. No retries permitted until 2026-09-10 12:17:57.967610132 +0000 UTC m=+5.972829597 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c6ad9571-3788-4c07-8778-c647b1a2cdce-metrics-certs") pod "network-metrics-daemon-6j4mf" (UID: "c6ad9571-3788-4c07-8778-c647b1a2cdce") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:17:56.169628 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:56.169590 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6960f97e-f623-4368-b27a-41d81ac1be1f-original-pull-secret\") pod \"global-pull-secret-syncer-dgddq\" (UID: \"6960f97e-f623-4368-b27a-41d81ac1be1f\") " pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:17:56.169811 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:56.169655 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr48g\" (UniqueName: \"kubernetes.io/projected/1aa57da6-29e7-4701-8b1b-be81cf4ebed6-kube-api-access-xr48g\") pod \"network-check-target-k9bgv\" (UID: \"1aa57da6-29e7-4701-8b1b-be81cf4ebed6\") " pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:17:56.169811 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:56.169786 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:17:56.169811 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:56.169808 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:17:56.170016 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:56.169821 2583 projected.go:196] Error preparing data for projected volume kube-api-access-xr48g for pod openshift-network-diagnostics/network-check-target-k9bgv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:17:56.170016 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:56.169891 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1aa57da6-29e7-4701-8b1b-be81cf4ebed6-kube-api-access-xr48g podName:1aa57da6-29e7-4701-8b1b-be81cf4ebed6 nodeName:}" failed. No retries permitted until 2026-09-10 12:17:58.169873747 +0000 UTC m=+6.175093203 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xr48g" (UniqueName: "kubernetes.io/projected/1aa57da6-29e7-4701-8b1b-be81cf4ebed6-kube-api-access-xr48g") pod "network-check-target-k9bgv" (UID: "1aa57da6-29e7-4701-8b1b-be81cf4ebed6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:17:56.170295 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:56.170272 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:17:56.170370 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:56.170342 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6960f97e-f623-4368-b27a-41d81ac1be1f-original-pull-secret podName:6960f97e-f623-4368-b27a-41d81ac1be1f nodeName:}" failed. No retries permitted until 2026-09-10 12:17:58.170325725 +0000 UTC m=+6.175545178 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6960f97e-f623-4368-b27a-41d81ac1be1f-original-pull-secret") pod "global-pull-secret-syncer-dgddq" (UID: "6960f97e-f623-4368-b27a-41d81ac1be1f") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:17:56.387292 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:56.387225 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 12:12:54 +0000 UTC" deadline="2028-06-11 10:39:04.221163372 +0000 UTC" Sep 10 12:17:56.387292 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:56.387264 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15358h21m7.833903936s" Sep 10 12:17:56.466005 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:56.465976 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:17:56.466160 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:56.466108 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9bgv" podUID="1aa57da6-29e7-4701-8b1b-be81cf4ebed6" Sep 10 12:17:56.466522 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:56.466499 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:17:56.466621 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:56.466594 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgddq" podUID="6960f97e-f623-4368-b27a-41d81ac1be1f" Sep 10 12:17:56.468386 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:56.468205 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:17:56.468386 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:56.468306 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j4mf" podUID="c6ad9571-3788-4c07-8778-c647b1a2cdce" Sep 10 12:17:57.520200 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:57.520163 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-110.ec2.internal" event={"ID":"de9e1cf5cc38daf6ca22018d91022ea0","Type":"ContainerStarted","Data":"09d5e556f6bc100c7a31e4c29afc6bded0f0233888d233ae8c43fa57677c6952"} Sep 10 12:17:57.527066 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:57.526808 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-110.ec2.internal" event={"ID":"abdf679f5f7e1ecb5f64c1178ad16227","Type":"ContainerStarted","Data":"6bde034c58682083f8d7727e7c32fbc7570ad1c086a499abf41a288f9db6a5a5"} Sep 10 12:17:57.585664 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:57.585558 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-110.ec2.internal" podStartSLOduration=3.585536735 podStartE2EDuration="3.585536735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:17:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:17:57.584981575 +0000 UTC m=+5.590201028" watchObservedRunningTime="2026-09-10 12:17:57.585536735 +0000 UTC m=+5.590756196" Sep 10 12:17:57.987242 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:57.987163 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6ad9571-3788-4c07-8778-c647b1a2cdce-metrics-certs\") pod \"network-metrics-daemon-6j4mf\" (UID: \"c6ad9571-3788-4c07-8778-c647b1a2cdce\") " pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:17:57.987403 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:57.987308 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:17:57.987403 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:57.987365 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6ad9571-3788-4c07-8778-c647b1a2cdce-metrics-certs podName:c6ad9571-3788-4c07-8778-c647b1a2cdce nodeName:}" failed. No retries permitted until 2026-09-10 12:18:01.987347667 +0000 UTC m=+9.992567108 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c6ad9571-3788-4c07-8778-c647b1a2cdce-metrics-certs") pod "network-metrics-daemon-6j4mf" (UID: "c6ad9571-3788-4c07-8778-c647b1a2cdce") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:17:58.189473 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:58.188727 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr48g\" (UniqueName: \"kubernetes.io/projected/1aa57da6-29e7-4701-8b1b-be81cf4ebed6-kube-api-access-xr48g\") pod \"network-check-target-k9bgv\" (UID: \"1aa57da6-29e7-4701-8b1b-be81cf4ebed6\") " pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:17:58.189473 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:58.188803 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6960f97e-f623-4368-b27a-41d81ac1be1f-original-pull-secret\") pod \"global-pull-secret-syncer-dgddq\" (UID: \"6960f97e-f623-4368-b27a-41d81ac1be1f\") " pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:17:58.189473 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:58.188952 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:17:58.189473 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:58.189011 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6960f97e-f623-4368-b27a-41d81ac1be1f-original-pull-secret podName:6960f97e-f623-4368-b27a-41d81ac1be1f nodeName:}" failed. No retries permitted until 2026-09-10 12:18:02.188992502 +0000 UTC m=+10.194211963 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6960f97e-f623-4368-b27a-41d81ac1be1f-original-pull-secret") pod "global-pull-secret-syncer-dgddq" (UID: "6960f97e-f623-4368-b27a-41d81ac1be1f") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:17:58.189473 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:58.189097 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:17:58.189473 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:58.189113 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:17:58.189473 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:58.189124 2583 projected.go:196] Error preparing data for projected volume kube-api-access-xr48g for pod openshift-network-diagnostics/network-check-target-k9bgv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:17:58.189473 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:58.189161 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1aa57da6-29e7-4701-8b1b-be81cf4ebed6-kube-api-access-xr48g podName:1aa57da6-29e7-4701-8b1b-be81cf4ebed6 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:02.189149453 +0000 UTC m=+10.194368906 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xr48g" (UniqueName: "kubernetes.io/projected/1aa57da6-29e7-4701-8b1b-be81cf4ebed6-kube-api-access-xr48g") pod "network-check-target-k9bgv" (UID: "1aa57da6-29e7-4701-8b1b-be81cf4ebed6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:17:58.465718 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:58.464964 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:17:58.465718 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:58.464983 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:17:58.465718 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:58.465116 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j4mf" podUID="c6ad9571-3788-4c07-8778-c647b1a2cdce" Sep 10 12:17:58.465718 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:58.465189 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9bgv" podUID="1aa57da6-29e7-4701-8b1b-be81cf4ebed6" Sep 10 12:17:58.465718 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:17:58.465248 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:17:58.465718 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:17:58.465317 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgddq" podUID="6960f97e-f623-4368-b27a-41d81ac1be1f" Sep 10 12:18:00.465030 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:00.464952 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:18:00.465030 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:00.464967 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:18:00.465518 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:00.465088 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9bgv" podUID="1aa57da6-29e7-4701-8b1b-be81cf4ebed6" Sep 10 12:18:00.465518 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:00.465346 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:18:00.465518 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:00.465452 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgddq" podUID="6960f97e-f623-4368-b27a-41d81ac1be1f" Sep 10 12:18:00.465678 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:00.465571 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j4mf" podUID="c6ad9571-3788-4c07-8778-c647b1a2cdce" Sep 10 12:18:00.539857 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:00.539809 2583 generic.go:356] "Generic (PLEG): container finished" podID="de9e1cf5cc38daf6ca22018d91022ea0" containerID="09d5e556f6bc100c7a31e4c29afc6bded0f0233888d233ae8c43fa57677c6952" exitCode=0 Sep 10 12:18:00.540013 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:00.539890 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-110.ec2.internal" event={"ID":"de9e1cf5cc38daf6ca22018d91022ea0","Type":"ContainerDied","Data":"09d5e556f6bc100c7a31e4c29afc6bded0f0233888d233ae8c43fa57677c6952"} Sep 10 12:18:02.019380 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:02.019338 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6ad9571-3788-4c07-8778-c647b1a2cdce-metrics-certs\") pod \"network-metrics-daemon-6j4mf\" (UID: \"c6ad9571-3788-4c07-8778-c647b1a2cdce\") " pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:18:02.019792 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:02.019510 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:18:02.019792 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:02.019577 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6ad9571-3788-4c07-8778-c647b1a2cdce-metrics-certs podName:c6ad9571-3788-4c07-8778-c647b1a2cdce nodeName:}" failed. No retries permitted until 2026-09-10 12:18:10.019556792 +0000 UTC m=+18.024776246 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c6ad9571-3788-4c07-8778-c647b1a2cdce-metrics-certs") pod "network-metrics-daemon-6j4mf" (UID: "c6ad9571-3788-4c07-8778-c647b1a2cdce") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:18:02.221545 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:02.221506 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr48g\" (UniqueName: \"kubernetes.io/projected/1aa57da6-29e7-4701-8b1b-be81cf4ebed6-kube-api-access-xr48g\") pod \"network-check-target-k9bgv\" (UID: \"1aa57da6-29e7-4701-8b1b-be81cf4ebed6\") " pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:18:02.221729 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:02.221693 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6960f97e-f623-4368-b27a-41d81ac1be1f-original-pull-secret\") pod \"global-pull-secret-syncer-dgddq\" (UID: \"6960f97e-f623-4368-b27a-41d81ac1be1f\") " pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:18:02.221799 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:02.221700 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:18:02.221799 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:02.221751 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:18:02.221799 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:02.221764 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:18:02.221799 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:02.221780 2583 projected.go:196] Error preparing data for projected volume kube-api-access-xr48g for pod openshift-network-diagnostics/network-check-target-k9bgv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:18:02.222004 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:02.221811 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6960f97e-f623-4368-b27a-41d81ac1be1f-original-pull-secret podName:6960f97e-f623-4368-b27a-41d81ac1be1f nodeName:}" failed. No retries permitted until 2026-09-10 12:18:10.221795486 +0000 UTC m=+18.227014924 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6960f97e-f623-4368-b27a-41d81ac1be1f-original-pull-secret") pod "global-pull-secret-syncer-dgddq" (UID: "6960f97e-f623-4368-b27a-41d81ac1be1f") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:18:02.222004 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:02.221830 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1aa57da6-29e7-4701-8b1b-be81cf4ebed6-kube-api-access-xr48g podName:1aa57da6-29e7-4701-8b1b-be81cf4ebed6 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:10.221820117 +0000 UTC m=+18.227039555 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xr48g" (UniqueName: "kubernetes.io/projected/1aa57da6-29e7-4701-8b1b-be81cf4ebed6-kube-api-access-xr48g") pod "network-check-target-k9bgv" (UID: "1aa57da6-29e7-4701-8b1b-be81cf4ebed6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:18:02.465643 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:02.465600 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:18:02.465807 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:02.465704 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:18:02.465807 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:02.465713 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j4mf" podUID="c6ad9571-3788-4c07-8778-c647b1a2cdce" Sep 10 12:18:02.465944 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:02.465806 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgddq" podUID="6960f97e-f623-4368-b27a-41d81ac1be1f" Sep 10 12:18:02.465944 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:02.465874 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:18:02.465944 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:02.465937 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9bgv" podUID="1aa57da6-29e7-4701-8b1b-be81cf4ebed6" Sep 10 12:18:04.465085 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:04.465051 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:18:04.465463 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:04.465051 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:18:04.465463 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:04.465185 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j4mf" podUID="c6ad9571-3788-4c07-8778-c647b1a2cdce" Sep 10 12:18:04.465463 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:04.465251 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9bgv" podUID="1aa57da6-29e7-4701-8b1b-be81cf4ebed6" Sep 10 12:18:04.465463 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:04.465054 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:18:04.465463 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:04.465347 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgddq" podUID="6960f97e-f623-4368-b27a-41d81ac1be1f" Sep 10 12:18:06.465037 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:06.465000 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:18:06.465524 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:06.465000 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:18:06.465524 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:06.465139 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j4mf" podUID="c6ad9571-3788-4c07-8778-c647b1a2cdce" Sep 10 12:18:06.465524 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:06.465177 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:18:06.465524 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:06.465275 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgddq" podUID="6960f97e-f623-4368-b27a-41d81ac1be1f" Sep 10 12:18:06.465524 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:06.465348 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9bgv" podUID="1aa57da6-29e7-4701-8b1b-be81cf4ebed6" Sep 10 12:18:08.464523 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:08.464489 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:18:08.464523 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:08.464505 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:18:08.464981 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:08.464601 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgddq" podUID="6960f97e-f623-4368-b27a-41d81ac1be1f" Sep 10 12:18:08.464981 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:08.464672 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:18:08.464981 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:08.464675 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j4mf" podUID="c6ad9571-3788-4c07-8778-c647b1a2cdce" Sep 10 12:18:08.464981 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:08.464751 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9bgv" podUID="1aa57da6-29e7-4701-8b1b-be81cf4ebed6" Sep 10 12:18:10.080693 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:10.080657 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6ad9571-3788-4c07-8778-c647b1a2cdce-metrics-certs\") pod \"network-metrics-daemon-6j4mf\" (UID: \"c6ad9571-3788-4c07-8778-c647b1a2cdce\") " pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:18:10.081192 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:10.080794 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:18:10.081192 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:10.080864 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6ad9571-3788-4c07-8778-c647b1a2cdce-metrics-certs podName:c6ad9571-3788-4c07-8778-c647b1a2cdce nodeName:}" failed. No retries permitted until 2026-09-10 12:18:26.080849164 +0000 UTC m=+34.086068614 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c6ad9571-3788-4c07-8778-c647b1a2cdce-metrics-certs") pod "network-metrics-daemon-6j4mf" (UID: "c6ad9571-3788-4c07-8778-c647b1a2cdce") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:18:10.282228 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:10.282185 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6960f97e-f623-4368-b27a-41d81ac1be1f-original-pull-secret\") pod \"global-pull-secret-syncer-dgddq\" (UID: \"6960f97e-f623-4368-b27a-41d81ac1be1f\") " pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:18:10.282414 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:10.282263 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr48g\" (UniqueName: \"kubernetes.io/projected/1aa57da6-29e7-4701-8b1b-be81cf4ebed6-kube-api-access-xr48g\") pod \"network-check-target-k9bgv\" (UID: \"1aa57da6-29e7-4701-8b1b-be81cf4ebed6\") " pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:18:10.282414 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:10.282304 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:18:10.282414 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:10.282395 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:18:10.282414 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:10.282415 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:18:10.282614 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:10.282428 2583 projected.go:196] Error preparing data for projected volume kube-api-access-xr48g for pod openshift-network-diagnostics/network-check-target-k9bgv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:18:10.282614 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:10.282401 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6960f97e-f623-4368-b27a-41d81ac1be1f-original-pull-secret podName:6960f97e-f623-4368-b27a-41d81ac1be1f nodeName:}" failed. No retries permitted until 2026-09-10 12:18:26.282372567 +0000 UTC m=+34.287592007 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6960f97e-f623-4368-b27a-41d81ac1be1f-original-pull-secret") pod "global-pull-secret-syncer-dgddq" (UID: "6960f97e-f623-4368-b27a-41d81ac1be1f") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:18:10.282614 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:10.282492 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1aa57da6-29e7-4701-8b1b-be81cf4ebed6-kube-api-access-xr48g podName:1aa57da6-29e7-4701-8b1b-be81cf4ebed6 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:26.282478727 +0000 UTC m=+34.287698166 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xr48g" (UniqueName: "kubernetes.io/projected/1aa57da6-29e7-4701-8b1b-be81cf4ebed6-kube-api-access-xr48g") pod "network-check-target-k9bgv" (UID: "1aa57da6-29e7-4701-8b1b-be81cf4ebed6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:18:10.464544 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:10.464463 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:18:10.464544 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:10.464498 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:18:10.464544 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:10.464466 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:18:10.466149 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:10.466099 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j4mf" podUID="c6ad9571-3788-4c07-8778-c647b1a2cdce" Sep 10 12:18:10.466597 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:10.466512 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgddq" podUID="6960f97e-f623-4368-b27a-41d81ac1be1f" Sep 10 12:18:10.466704 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:10.466685 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9bgv" podUID="1aa57da6-29e7-4701-8b1b-be81cf4ebed6" Sep 10 12:18:12.465494 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:12.465433 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:18:12.465910 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:12.465549 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:18:12.465910 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:12.465556 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgddq" podUID="6960f97e-f623-4368-b27a-41d81ac1be1f" Sep 10 12:18:12.465910 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:12.465643 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j4mf" podUID="c6ad9571-3788-4c07-8778-c647b1a2cdce" Sep 10 12:18:12.465910 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:12.465668 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:18:12.465910 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:12.465780 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9bgv" podUID="1aa57da6-29e7-4701-8b1b-be81cf4ebed6" Sep 10 12:18:13.565222 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:13.564907 2583 generic.go:356] "Generic (PLEG): container finished" podID="d34fb4ae-61a3-4533-a553-9dc4e2f6daaf" containerID="05383d8c83f731c418a1d4846e6cb6e99a503ab72f102a026d0596f1c8aeefcb" exitCode=0 Sep 10 12:18:13.565707 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:13.564994 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5fct5" event={"ID":"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf","Type":"ContainerDied","Data":"05383d8c83f731c418a1d4846e6cb6e99a503ab72f102a026d0596f1c8aeefcb"} Sep 10 12:18:13.569102 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:13.569073 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-smplh" event={"ID":"3ab78f8a-127b-4d1a-9d6f-0b9e06251841","Type":"ContainerStarted","Data":"d6a4f1f4ddd2293bc5e180b00953b12a19fdcda0a588f2d20d546eb6977dab02"} Sep 10 12:18:13.570755 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:13.570732 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lm7qv" event={"ID":"f7244cbf-0a28-477e-9012-5dbe58beaa77","Type":"ContainerStarted","Data":"797c2413711a57510bbb9e3a15d160ed142fe4921d1bfbe7af9a51fd6d01062d"} Sep 10 12:18:13.572558 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:13.572530 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" event={"ID":"3e79b263-6658-4ca2-8d1f-df5cfc8c24da","Type":"ContainerStarted","Data":"40970a82435bf1e2475235499c23afd4da2c747fd9f451a9e955c64d3c8db9d6"} Sep 10 12:18:13.574370 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:13.574348 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qkhpj" event={"ID":"57d3f064-15f5-4771-b729-cdae4f1ae208","Type":"ContainerStarted","Data":"6f559167d0e85d622e7b308947fa1a01f43e8c3a0dcfbab42b053313f9edec66"} Sep 10 12:18:13.576110 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:13.576092 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vqcn2" event={"ID":"a1521572-7823-459a-a435-0248c1ae1b12","Type":"ContainerStarted","Data":"ddcf04f3297cbb8fdc7bffb446b8eefa8a7e25b6cf78c9f0b2d30ee62b5621b1"} Sep 10 12:18:13.578615 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:13.578592 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-110.ec2.internal" event={"ID":"de9e1cf5cc38daf6ca22018d91022ea0","Type":"ContainerStarted","Data":"7f3ca3b69401c3e3a96b72609a47e2bb36e6f718f5582b11a5936eeed243ef38"} Sep 10 12:18:13.583679 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:13.583662 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/ovn-acl-logging/0.log" Sep 10 12:18:13.584114 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:13.584093 2583 generic.go:356] "Generic (PLEG): container finished" podID="7bc3c1d2-e9d8-4fd4-bd94-878d61758b97" containerID="f4f98df1a7ea3a45fb1f703a88435b120c926dbaa2dbb7d547bd5267676cd992" exitCode=1 Sep 10 12:18:13.584207 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:13.584155 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" event={"ID":"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97","Type":"ContainerStarted","Data":"a41beae1da8695f0fade7c5d8d9115dc2032e91140ba5019fd73fb016626d672"} Sep 10 12:18:13.584268 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:13.584251 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" event={"ID":"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97","Type":"ContainerStarted","Data":"168c68c220325172a2f11622b92fcec9fb456042810ae737264a9b1bc76158b7"} Sep 10 12:18:13.584326 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:13.584313 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" event={"ID":"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97","Type":"ContainerDied","Data":"f4f98df1a7ea3a45fb1f703a88435b120c926dbaa2dbb7d547bd5267676cd992"} Sep 10 12:18:13.584385 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:13.584371 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" event={"ID":"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97","Type":"ContainerStarted","Data":"341bb53c9b87363a1e7597d1944efd3ca59b9d545338e879ee6fd20734589c3a"} Sep 10 12:18:13.585816 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:13.585792 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" event={"ID":"be18f526-aa95-4956-9d00-8e31388c5c26","Type":"ContainerStarted","Data":"52eed09bc023b9bc5c5d3407e8c37d1f0802babe64dbced06373b3dd4ca36786"} Sep 10 12:18:13.629334 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:13.629293 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vqcn2" podStartSLOduration=3.52791618 podStartE2EDuration="21.629278256s" totalImagesPullingTime="18.101362076s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:17:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:13.628765006 +0000 UTC m=+21.633984469" watchObservedRunningTime="2026-09-10 12:18:13.629278256 +0000 UTC m=+21.634497716" Sep 10 12:18:13.629449 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:13.629364 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-110.ec2.internal" podStartSLOduration=19.629359681 podStartE2EDuration="19.629359681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:17:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:13.612987604 +0000 UTC m=+21.618207065" watchObservedRunningTime="2026-09-10 12:18:13.629359681 +0000 UTC m=+21.634579140" Sep 10 12:18:13.648406 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:13.648352 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vpqd2" podStartSLOduration=3.5117031709999997 podStartE2EDuration="21.648332795s" totalImagesPullingTime="18.136629624s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:17:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:13.647068732 +0000 UTC m=+21.652288192" watchObservedRunningTime="2026-09-10 12:18:13.648332795 +0000 UTC m=+21.653552255" Sep 10 12:18:13.668286 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:13.668241 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lm7qv" podStartSLOduration=3.5382637409999997 podStartE2EDuration="21.668226427s" totalImagesPullingTime="18.129962686s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:17:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:13.667871126 +0000 UTC m=+21.673090586" watchObservedRunningTime="2026-09-10 12:18:13.668226427 +0000 UTC m=+21.673445887" Sep 10 12:18:13.713544 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:13.713489 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-smplh" podStartSLOduration=2.960533057 podStartE2EDuration="21.713470715s" totalImagesPullingTime="18.752937658s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:17:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:13.712689238 +0000 UTC m=+21.717908699" watchObservedRunningTime="2026-09-10 12:18:13.713470715 +0000 UTC m=+21.718690177" Sep 10 12:18:13.714230 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:13.714191 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qkhpj" podStartSLOduration=3.571986264 podStartE2EDuration="21.714176286s" totalImagesPullingTime="18.142190022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:17:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:13.689989752 +0000 UTC m=+21.695209379" watchObservedRunningTime="2026-09-10 12:18:13.714176286 +0000 UTC m=+21.719395748" Sep 10 12:18:14.235332 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:14.235311 2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 10 12:18:14.407916 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:14.407785 2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T12:18:14.235328657Z","UUID":"4546f985-5c9e-49f8-adb2-cda119b016ff","Handler":null,"Name":"","Endpoint":""} Sep 10 12:18:14.409068 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:14.409053 2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 10 12:18:14.409150 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:14.409079 2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 10 12:18:14.464460 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:14.464440 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:18:14.464460 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:14.464450 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:18:14.464582 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:14.464527 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgddq" podUID="6960f97e-f623-4368-b27a-41d81ac1be1f" Sep 10 12:18:14.464630 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:14.464600 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j4mf" podUID="c6ad9571-3788-4c07-8778-c647b1a2cdce" Sep 10 12:18:14.464630 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:14.464621 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:18:14.464703 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:14.464688 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9bgv" podUID="1aa57da6-29e7-4701-8b1b-be81cf4ebed6" Sep 10 12:18:14.589594 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:14.589572 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" event={"ID":"3e79b263-6658-4ca2-8d1f-df5cfc8c24da","Type":"ContainerStarted","Data":"eda60b53093902305df03704fb11c080ce1db448bb47b613cd310ee8ad0099a1"} Sep 10 12:18:14.591021 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:14.590997 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wqv99" event={"ID":"adee8f79-7570-4cb5-be43-089a5a464107","Type":"ContainerStarted","Data":"ba4ec7d07fd0c9b8702798dafcd7157fe1a25c785464ab92c057f069165644d2"} Sep 10 12:18:14.593441 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:14.593426 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/ovn-acl-logging/0.log" Sep 10 12:18:14.593783 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:14.593753 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" event={"ID":"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97","Type":"ContainerStarted","Data":"3ea3b251252a791c90a1232ae35f05261006b0eeadbd8723ca083894941b56e6"} Sep 10 12:18:14.593896 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:14.593849 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" event={"ID":"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97","Type":"ContainerStarted","Data":"c70a0f625a7153ab0c8e024d598f9c3b7bddd467e8053bf33dc18b37824a634f"} Sep 10 12:18:14.608138 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:14.608100 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wqv99" podStartSLOduration=4.452838728 podStartE2EDuration="22.608084419s" totalImagesPullingTime="18.155245691s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:17:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:14.607977335 +0000 UTC m=+22.613196795" watchObservedRunningTime="2026-09-10 12:18:14.608084419 +0000 UTC m=+22.613303879" Sep 10 12:18:15.598930 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:15.598890 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" event={"ID":"3e79b263-6658-4ca2-8d1f-df5cfc8c24da","Type":"ContainerStarted","Data":"f510bf89cb9eceafd569309826265654e2937fd6639090cfe1d287ecb96c4e3b"} Sep 10 12:18:15.638406 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:15.638353 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vwvsl" podStartSLOduration=3.714135039 podStartE2EDuration="23.638333666s" totalImagesPullingTime="19.924198627s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:17:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:15.637552017 +0000 UTC m=+23.642771478" watchObservedRunningTime="2026-09-10 12:18:15.638333666 +0000 UTC m=+23.643553127" Sep 10 12:18:16.257722 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:16.257686 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qkhpj" Sep 10 12:18:16.258378 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:16.258357 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qkhpj" Sep 10 12:18:16.464416 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:16.464323 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:18:16.464581 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:16.464439 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:18:16.464581 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:16.464446 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9bgv" podUID="1aa57da6-29e7-4701-8b1b-be81cf4ebed6" Sep 10 12:18:16.464581 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:16.464560 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j4mf" podUID="c6ad9571-3788-4c07-8778-c647b1a2cdce" Sep 10 12:18:16.464746 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:16.464613 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:18:16.464746 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:16.464675 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgddq" podUID="6960f97e-f623-4368-b27a-41d81ac1be1f" Sep 10 12:18:16.604783 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:16.604749 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/ovn-acl-logging/0.log" Sep 10 12:18:16.605215 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:16.605152 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" event={"ID":"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97","Type":"ContainerStarted","Data":"2801432f5d92cf2f9ce8eaaa28e0dbcf9203f9289b5a01a81f32bff75ae1bcfb"} Sep 10 12:18:16.605710 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:16.605686 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qkhpj" Sep 10 12:18:16.605877 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:16.605849 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qkhpj" Sep 10 12:18:18.465083 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:18.464943 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:18:18.465464 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:18.464954 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:18:18.465464 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:18.465190 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j4mf" podUID="c6ad9571-3788-4c07-8778-c647b1a2cdce" Sep 10 12:18:18.465464 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:18.464989 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:18:18.465464 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:18.465258 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9bgv" podUID="1aa57da6-29e7-4701-8b1b-be81cf4ebed6" Sep 10 12:18:18.465464 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:18.465348 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgddq" podUID="6960f97e-f623-4368-b27a-41d81ac1be1f" Sep 10 12:18:18.612058 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:18.612039 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/ovn-acl-logging/0.log" Sep 10 12:18:18.612384 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:18.612362 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" event={"ID":"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97","Type":"ContainerStarted","Data":"08deda6988c8f8cb756bb71a92deb71f9fe4fbaedaf0a3e60ac787043c1fabf0"} Sep 10 12:18:18.612808 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:18.612791 2583 scope.go:117] "RemoveContainer" containerID="f4f98df1a7ea3a45fb1f703a88435b120c926dbaa2dbb7d547bd5267676cd992" Sep 10 12:18:19.616168 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:19.616133 2583 generic.go:356] "Generic (PLEG): container finished" podID="d34fb4ae-61a3-4533-a553-9dc4e2f6daaf" containerID="2cef99fe97f01024641e0de85e20689d14cae08f2867906bc21567f36974132a" exitCode=0 Sep 10 12:18:19.616817 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:19.616217 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5fct5" event={"ID":"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf","Type":"ContainerDied","Data":"2cef99fe97f01024641e0de85e20689d14cae08f2867906bc21567f36974132a"} Sep 10 12:18:19.620122 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:19.620105 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/ovn-acl-logging/0.log" Sep 10 12:18:19.620473 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:19.620451 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" event={"ID":"7bc3c1d2-e9d8-4fd4-bd94-878d61758b97","Type":"ContainerStarted","Data":"9c97ec47b53a5927032f0069067c8b597805261faf4789fbf1877bdd817b84d8"} Sep 10 12:18:19.620727 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:19.620709 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:18:19.620830 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:19.620785 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:18:19.620905 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:19.620833 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:18:19.636052 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:19.636030 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:18:19.636142 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:19.636127 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:18:19.675032 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:19.674992 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" podStartSLOduration=9.46119836 podStartE2EDuration="27.674976102s" totalImagesPullingTime="18.213777742s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:17:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:19.674512457 +0000 UTC m=+27.679731917" watchObservedRunningTime="2026-09-10 12:18:19.674976102 +0000 UTC m=+27.680195586" Sep 10 12:18:20.464759 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:20.464619 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:18:20.464865 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:20.464619 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:18:20.464865 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:20.464824 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9bgv" podUID="1aa57da6-29e7-4701-8b1b-be81cf4ebed6" Sep 10 12:18:20.464865 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:20.464622 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:18:20.464970 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:20.464936 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j4mf" podUID="c6ad9571-3788-4c07-8778-c647b1a2cdce" Sep 10 12:18:20.465018 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:20.464998 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgddq" podUID="6960f97e-f623-4368-b27a-41d81ac1be1f" Sep 10 12:18:20.619199 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:20.619157 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k9bgv"] Sep 10 12:18:20.621684 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:20.621660 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dgddq"] Sep 10 12:18:20.622885 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:20.622861 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6j4mf"] Sep 10 12:18:20.627623 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:20.627600 2583 generic.go:356] "Generic (PLEG): container finished" podID="d34fb4ae-61a3-4533-a553-9dc4e2f6daaf" containerID="a5680a7baa40c92ed5880862a8e2c8608a3e495e4c14c9e479a9ac2d50efb392" exitCode=0 Sep 10 12:18:20.627746 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:20.627720 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5fct5" event={"ID":"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf","Type":"ContainerDied","Data":"a5680a7baa40c92ed5880862a8e2c8608a3e495e4c14c9e479a9ac2d50efb392"} Sep 10 12:18:20.627816 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:20.627770 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:18:20.627943 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:20.627779 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:18:20.628129 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:20.627957 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j4mf" podUID="c6ad9571-3788-4c07-8778-c647b1a2cdce" Sep 10 12:18:20.628340 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:20.628316 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9bgv" podUID="1aa57da6-29e7-4701-8b1b-be81cf4ebed6" Sep 10 12:18:20.629204 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:20.628494 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:18:20.629204 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:20.628579 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgddq" podUID="6960f97e-f623-4368-b27a-41d81ac1be1f" Sep 10 12:18:21.631696 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:21.631655 2583 generic.go:356] "Generic (PLEG): container finished" podID="d34fb4ae-61a3-4533-a553-9dc4e2f6daaf" containerID="2a376b681d0a89d2eb81cf2d2ca3addea996afc0fba9f95820ca1d002dbbb0e4" exitCode=0 Sep 10 12:18:21.632210 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:21.631735 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5fct5" event={"ID":"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf","Type":"ContainerDied","Data":"2a376b681d0a89d2eb81cf2d2ca3addea996afc0fba9f95820ca1d002dbbb0e4"} Sep 10 12:18:22.466181 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:22.466001 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:18:22.466337 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:22.466071 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:18:22.466337 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:22.466095 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:18:22.466337 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:22.466278 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgddq" podUID="6960f97e-f623-4368-b27a-41d81ac1be1f" Sep 10 12:18:22.466470 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:22.466401 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9bgv" podUID="1aa57da6-29e7-4701-8b1b-be81cf4ebed6" Sep 10 12:18:22.466519 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:22.466501 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j4mf" podUID="c6ad9571-3788-4c07-8778-c647b1a2cdce" Sep 10 12:18:24.468659 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:24.468632 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:18:24.468659 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:24.468648 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:18:24.469119 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:24.468748 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k9bgv" podUID="1aa57da6-29e7-4701-8b1b-be81cf4ebed6" Sep 10 12:18:24.469119 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:24.468806 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgddq" podUID="6960f97e-f623-4368-b27a-41d81ac1be1f" Sep 10 12:18:24.469119 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:24.468817 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:18:24.469119 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:24.468933 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6j4mf" podUID="c6ad9571-3788-4c07-8778-c647b1a2cdce" Sep 10 12:18:26.092655 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.092617 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6ad9571-3788-4c07-8778-c647b1a2cdce-metrics-certs\") pod \"network-metrics-daemon-6j4mf\" (UID: \"c6ad9571-3788-4c07-8778-c647b1a2cdce\") " pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:18:26.093142 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:26.092796 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:18:26.093142 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:26.092880 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6ad9571-3788-4c07-8778-c647b1a2cdce-metrics-certs podName:c6ad9571-3788-4c07-8778-c647b1a2cdce nodeName:}" failed. No retries permitted until 2026-09-10 12:18:58.09286094 +0000 UTC m=+66.098080398 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c6ad9571-3788-4c07-8778-c647b1a2cdce-metrics-certs") pod "network-metrics-daemon-6j4mf" (UID: "c6ad9571-3788-4c07-8778-c647b1a2cdce") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:18:26.222303 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.222227 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-110.ec2.internal" event="NodeReady" Sep 10 12:18:26.222451 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.222368 2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 10 12:18:26.270801 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.270768 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dssrq"] Sep 10 12:18:26.293916 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.293889 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6960f97e-f623-4368-b27a-41d81ac1be1f-original-pull-secret\") pod \"global-pull-secret-syncer-dgddq\" (UID: \"6960f97e-f623-4368-b27a-41d81ac1be1f\") " pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:18:26.294091 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.293939 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr48g\" (UniqueName: \"kubernetes.io/projected/1aa57da6-29e7-4701-8b1b-be81cf4ebed6-kube-api-access-xr48g\") pod \"network-check-target-k9bgv\" (UID: \"1aa57da6-29e7-4701-8b1b-be81cf4ebed6\") " pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:18:26.294091 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:26.294030 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:18:26.294091 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:26.294049 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:18:26.294091 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:26.294072 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:18:26.294091 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:26.294085 2583 projected.go:196] Error preparing data for projected volume kube-api-access-xr48g for pod openshift-network-diagnostics/network-check-target-k9bgv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:18:26.294338 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:26.294100 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6960f97e-f623-4368-b27a-41d81ac1be1f-original-pull-secret podName:6960f97e-f623-4368-b27a-41d81ac1be1f nodeName:}" failed. No retries permitted until 2026-09-10 12:18:58.294082155 +0000 UTC m=+66.299301606 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6960f97e-f623-4368-b27a-41d81ac1be1f-original-pull-secret") pod "global-pull-secret-syncer-dgddq" (UID: "6960f97e-f623-4368-b27a-41d81ac1be1f") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:18:26.294338 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:26.294129 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1aa57da6-29e7-4701-8b1b-be81cf4ebed6-kube-api-access-xr48g podName:1aa57da6-29e7-4701-8b1b-be81cf4ebed6 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:58.29411582 +0000 UTC m=+66.299335285 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xr48g" (UniqueName: "kubernetes.io/projected/1aa57da6-29e7-4701-8b1b-be81cf4ebed6-kube-api-access-xr48g") pod "network-check-target-k9bgv" (UID: "1aa57da6-29e7-4701-8b1b-be81cf4ebed6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:18:26.296676 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.296657 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kgd4g"] Sep 10 12:18:26.296803 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.296789 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dssrq" Sep 10 12:18:26.299388 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.299366 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 10 12:18:26.299388 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.299375 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 10 12:18:26.299650 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.299633 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lhd7h\"" Sep 10 12:18:26.319532 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.317085 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dssrq"] Sep 10 12:18:26.319532 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.317145 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kgd4g"] Sep 10 12:18:26.319532 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.317246 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kgd4g" Sep 10 12:18:26.321414 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.321396 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 10 12:18:26.321719 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.321702 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xh5g6\"" Sep 10 12:18:26.321952 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.321931 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 10 12:18:26.322474 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.322450 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 10 12:18:26.394642 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.394606 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-tmp-dir\") pod \"dns-default-dssrq\" (UID: \"3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2\") " pod="openshift-dns/dns-default-dssrq" Sep 10 12:18:26.394642 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.394647 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8ppp\" (UniqueName: \"kubernetes.io/projected/d295dd14-e004-4178-9b90-086fe26e6778-kube-api-access-s8ppp\") pod \"ingress-canary-kgd4g\" (UID: \"d295dd14-e004-4178-9b90-086fe26e6778\") " pod="openshift-ingress-canary/ingress-canary-kgd4g" Sep 10 12:18:26.394902 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.394696 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d295dd14-e004-4178-9b90-086fe26e6778-cert\") pod \"ingress-canary-kgd4g\" (UID: \"d295dd14-e004-4178-9b90-086fe26e6778\") " pod="openshift-ingress-canary/ingress-canary-kgd4g" Sep 10 12:18:26.394902 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.394764 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-config-volume\") pod \"dns-default-dssrq\" (UID: \"3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2\") " pod="openshift-dns/dns-default-dssrq" Sep 10 12:18:26.394902 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.394818 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8bzz\" (UniqueName: \"kubernetes.io/projected/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-kube-api-access-b8bzz\") pod \"dns-default-dssrq\" (UID: \"3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2\") " pod="openshift-dns/dns-default-dssrq" Sep 10 12:18:26.394902 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.394883 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-metrics-tls\") pod \"dns-default-dssrq\" (UID: \"3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2\") " pod="openshift-dns/dns-default-dssrq" Sep 10 12:18:26.465189 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.465141 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:18:26.465362 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.465160 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:18:26.465425 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.465384 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:18:26.467947 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.467924 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 12:18:26.468061 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.467965 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 12:18:26.468061 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.467935 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 12:18:26.468291 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.468212 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cfjr5\"" Sep 10 12:18:26.468291 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.468234 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 12:18:26.468534 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.468511 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wdbdh\"" Sep 10 12:18:26.496062 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.496037 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8ppp\" (UniqueName: \"kubernetes.io/projected/d295dd14-e004-4178-9b90-086fe26e6778-kube-api-access-s8ppp\") pod \"ingress-canary-kgd4g\" (UID: \"d295dd14-e004-4178-9b90-086fe26e6778\") " pod="openshift-ingress-canary/ingress-canary-kgd4g" Sep 10 12:18:26.496159 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.496094 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d295dd14-e004-4178-9b90-086fe26e6778-cert\") pod \"ingress-canary-kgd4g\" (UID: \"d295dd14-e004-4178-9b90-086fe26e6778\") " pod="openshift-ingress-canary/ingress-canary-kgd4g" Sep 10 12:18:26.496159 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.496119 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-config-volume\") pod \"dns-default-dssrq\" (UID: \"3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2\") " pod="openshift-dns/dns-default-dssrq" Sep 10 12:18:26.496159 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.496145 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8bzz\" (UniqueName: \"kubernetes.io/projected/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-kube-api-access-b8bzz\") pod \"dns-default-dssrq\" (UID: \"3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2\") " pod="openshift-dns/dns-default-dssrq" Sep 10 12:18:26.496324 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.496180 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-metrics-tls\") pod \"dns-default-dssrq\" (UID: \"3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2\") " pod="openshift-dns/dns-default-dssrq" Sep 10 12:18:26.496324 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.496210 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-tmp-dir\") pod \"dns-default-dssrq\" (UID: \"3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2\") " pod="openshift-dns/dns-default-dssrq" Sep 10 12:18:26.496418 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:26.496356 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 12:18:26.496467 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:26.496430 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-metrics-tls podName:3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:26.996406623 +0000 UTC m=+35.001626074 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-metrics-tls") pod "dns-default-dssrq" (UID: "3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2") : secret "dns-default-metrics-tls" not found Sep 10 12:18:26.496523 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.496506 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-tmp-dir\") pod \"dns-default-dssrq\" (UID: \"3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2\") " pod="openshift-dns/dns-default-dssrq" Sep 10 12:18:26.496714 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.496691 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-config-volume\") pod \"dns-default-dssrq\" (UID: \"3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2\") " pod="openshift-dns/dns-default-dssrq" Sep 10 12:18:26.501286 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.501267 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d295dd14-e004-4178-9b90-086fe26e6778-cert\") pod \"ingress-canary-kgd4g\" (UID: \"d295dd14-e004-4178-9b90-086fe26e6778\") " pod="openshift-ingress-canary/ingress-canary-kgd4g" Sep 10 12:18:26.507733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.507714 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8bzz\" (UniqueName: \"kubernetes.io/projected/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-kube-api-access-b8bzz\") pod \"dns-default-dssrq\" (UID: \"3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2\") " pod="openshift-dns/dns-default-dssrq" Sep 10 12:18:26.508177 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.508143 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8ppp\" (UniqueName: \"kubernetes.io/projected/d295dd14-e004-4178-9b90-086fe26e6778-kube-api-access-s8ppp\") pod \"ingress-canary-kgd4g\" (UID: \"d295dd14-e004-4178-9b90-086fe26e6778\") " pod="openshift-ingress-canary/ingress-canary-kgd4g" Sep 10 12:18:26.629437 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:26.629399 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kgd4g" Sep 10 12:18:27.000725 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:27.000693 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-metrics-tls\") pod \"dns-default-dssrq\" (UID: \"3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2\") " pod="openshift-dns/dns-default-dssrq" Sep 10 12:18:27.000922 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:27.000893 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 12:18:27.000996 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:27.000983 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-metrics-tls podName:3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:28.000958424 +0000 UTC m=+36.006177878 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-metrics-tls") pod "dns-default-dssrq" (UID: "3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2") : secret "dns-default-metrics-tls" not found Sep 10 12:18:27.349560 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:27.349523 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kgd4g"] Sep 10 12:18:27.388276 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:18:27.388240 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd295dd14_e004_4178_9b90_086fe26e6778.slice/crio-88a3c32c080214e95c57fa41daf48240d7eeb39cadebd863967eb3a3cd361dc4 WatchSource:0}: Error finding container 88a3c32c080214e95c57fa41daf48240d7eeb39cadebd863967eb3a3cd361dc4: Status 404 returned error can't find the container with id 88a3c32c080214e95c57fa41daf48240d7eeb39cadebd863967eb3a3cd361dc4 Sep 10 12:18:27.646731 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:27.646476 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kgd4g" event={"ID":"d295dd14-e004-4178-9b90-086fe26e6778","Type":"ContainerStarted","Data":"88a3c32c080214e95c57fa41daf48240d7eeb39cadebd863967eb3a3cd361dc4"} Sep 10 12:18:27.649658 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:27.649634 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5fct5" event={"ID":"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf","Type":"ContainerStarted","Data":"967965faf82e3034f5a21fe3d4746e8c63227f7b408576cf071eebd71844eeca"} Sep 10 12:18:28.007205 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:28.007131 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-metrics-tls\") pod \"dns-default-dssrq\" (UID: \"3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2\") " pod="openshift-dns/dns-default-dssrq" Sep 10 12:18:28.007357 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:28.007245 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 12:18:28.007357 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:28.007296 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-metrics-tls podName:3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:30.007281212 +0000 UTC m=+38.012500651 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-metrics-tls") pod "dns-default-dssrq" (UID: "3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2") : secret "dns-default-metrics-tls" not found Sep 10 12:18:28.656367 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:28.656330 2583 generic.go:356] "Generic (PLEG): container finished" podID="d34fb4ae-61a3-4533-a553-9dc4e2f6daaf" containerID="967965faf82e3034f5a21fe3d4746e8c63227f7b408576cf071eebd71844eeca" exitCode=0 Sep 10 12:18:28.656876 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:28.656407 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5fct5" event={"ID":"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf","Type":"ContainerDied","Data":"967965faf82e3034f5a21fe3d4746e8c63227f7b408576cf071eebd71844eeca"} Sep 10 12:18:29.662762 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:29.662720 2583 generic.go:356] "Generic (PLEG): container finished" podID="d34fb4ae-61a3-4533-a553-9dc4e2f6daaf" containerID="9fe41f23005d32db6998d3ec1338295dbdbd60c783dc87f52e14ab21ffbe9d4e" exitCode=0 Sep 10 12:18:29.663167 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:29.662763 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5fct5" event={"ID":"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf","Type":"ContainerDied","Data":"9fe41f23005d32db6998d3ec1338295dbdbd60c783dc87f52e14ab21ffbe9d4e"} Sep 10 12:18:30.022314 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.022289 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-metrics-tls\") pod \"dns-default-dssrq\" (UID: \"3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2\") " pod="openshift-dns/dns-default-dssrq" Sep 10 12:18:30.022446 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:30.022429 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 12:18:30.022506 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:30.022497 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-metrics-tls podName:3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:34.022482928 +0000 UTC m=+42.027702366 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-metrics-tls") pod "dns-default-dssrq" (UID: "3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2") : secret "dns-default-metrics-tls" not found Sep 10 12:18:30.568581 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.568545 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-kmrbv"] Sep 10 12:18:30.592505 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.592477 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-kmrbv"] Sep 10 12:18:30.592611 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.592569 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-kmrbv" Sep 10 12:18:30.595831 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.595806 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 10 12:18:30.595966 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.595814 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 10 12:18:30.596516 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.596499 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 10 12:18:30.596583 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.596502 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-r427m\"" Sep 10 12:18:30.596682 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.596665 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 10 12:18:30.666750 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.666722 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kgd4g" event={"ID":"d295dd14-e004-4178-9b90-086fe26e6778","Type":"ContainerStarted","Data":"08e4399257b1932afd0f6db7f7d154e86cd5e28eaf60631feb28835be9c370fe"} Sep 10 12:18:30.670586 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.670567 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5fct5" event={"ID":"d34fb4ae-61a3-4533-a553-9dc4e2f6daaf","Type":"ContainerStarted","Data":"40598310c47027380c86898c69014fa44784f9666804855fbdf3d9c2c73e4101"} Sep 10 12:18:30.683591 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.683573 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wftws"] Sep 10 12:18:30.693976 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.693938 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kgd4g" podStartSLOduration=1.969899089 podStartE2EDuration="4.693925832s" totalImagesPullingTime="2.724026743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:30.69190868 +0000 UTC m=+38.697128140" watchObservedRunningTime="2026-09-10 12:18:30.693925832 +0000 UTC m=+38.699145291" Sep 10 12:18:30.703622 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.703605 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-bf6588865-dxmvn"] Sep 10 12:18:30.703749 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.703733 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wftws" Sep 10 12:18:30.707420 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.707401 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 10 12:18:30.708128 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.708113 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 10 12:18:30.709826 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.709806 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 10 12:18:30.711487 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.710914 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-26848\"" Sep 10 12:18:30.721593 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.721570 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wftws"] Sep 10 12:18:30.721593 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.721602 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bf6588865-dxmvn"] Sep 10 12:18:30.721729 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.721682 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:30.723672 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.723625 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5fct5" podStartSLOduration=8.546358224 podStartE2EDuration="38.723609314s" totalImagesPullingTime="30.17725109s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:17:52 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:30.72306561 +0000 UTC m=+38.728285071" watchObservedRunningTime="2026-09-10 12:18:30.723609314 +0000 UTC m=+38.728828775" Sep 10 12:18:30.724549 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.724529 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 10 12:18:30.724725 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.724710 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 10 12:18:30.724793 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.724774 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bkkdl\"" Sep 10 12:18:30.724960 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.724944 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 10 12:18:30.725863 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.725820 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bkjc\" (UniqueName: \"kubernetes.io/projected/5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2-kube-api-access-8bkjc\") pod \"service-ca-operator-859f94c4b-kmrbv\" (UID: \"5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-kmrbv" Sep 10 12:18:30.725946 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.725871 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2-serving-cert\") pod \"service-ca-operator-859f94c4b-kmrbv\" (UID: \"5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-kmrbv" Sep 10 12:18:30.725946 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.725923 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2-config\") pod \"service-ca-operator-859f94c4b-kmrbv\" (UID: \"5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-kmrbv" Sep 10 12:18:30.731642 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.731625 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 10 12:18:30.782342 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.782315 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cq4k5"] Sep 10 12:18:30.802616 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.802593 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-85f68cb7f6-gsvtn"] Sep 10 12:18:30.802749 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.802722 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cq4k5" Sep 10 12:18:30.805590 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.805570 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 10 12:18:30.805692 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.805600 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-b5hvj\"" Sep 10 12:18:30.806602 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.806580 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 10 12:18:30.806713 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.806621 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 10 12:18:30.818242 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.818224 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 10 12:18:30.826746 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.826691 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d2f46f3-7193-4c2a-be48-f2458d2a0960-installation-pull-secrets\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:30.826746 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.826719 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bkjc\" (UniqueName: \"kubernetes.io/projected/5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2-kube-api-access-8bkjc\") pod \"service-ca-operator-859f94c4b-kmrbv\" (UID: \"5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-kmrbv" Sep 10 12:18:30.826746 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.826737 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d2f46f3-7193-4c2a-be48-f2458d2a0960-image-registry-private-configuration\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:30.826946 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.826769 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2-serving-cert\") pod \"service-ca-operator-859f94c4b-kmrbv\" (UID: \"5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-kmrbv" Sep 10 12:18:30.827035 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.827018 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-certificates\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:30.827146 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.827127 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2-config\") pod \"service-ca-operator-859f94c4b-kmrbv\" (UID: \"5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-kmrbv" Sep 10 12:18:30.827212 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.827196 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1727f13e-a084-4cc0-a35c-d7fc811a4c40-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-wftws\" (UID: \"1727f13e-a084-4cc0-a35c-d7fc811a4c40\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wftws" Sep 10 12:18:30.827387 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.827359 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-bound-sa-token\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:30.827485 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.827399 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6f7x\" (UniqueName: \"kubernetes.io/projected/1727f13e-a084-4cc0-a35c-d7fc811a4c40-kube-api-access-x6f7x\") pod \"cluster-samples-operator-7cdcfcf76c-wftws\" (UID: \"1727f13e-a084-4cc0-a35c-d7fc811a4c40\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wftws" Sep 10 12:18:30.827485 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.827424 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d2f46f3-7193-4c2a-be48-f2458d2a0960-trusted-ca\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:30.827485 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.827451 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwwd9\" (UniqueName: \"kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-kube-api-access-hwwd9\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:30.827597 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.827569 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-tls\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:30.827641 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.827605 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d2f46f3-7193-4c2a-be48-f2458d2a0960-ca-trust-extracted\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:30.827641 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.827620 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2-config\") pod \"service-ca-operator-859f94c4b-kmrbv\" (UID: \"5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-kmrbv" Sep 10 12:18:30.828281 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.828266 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cq4k5"] Sep 10 12:18:30.828345 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.828296 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-85f68cb7f6-gsvtn"] Sep 10 12:18:30.828390 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.828368 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:30.830695 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.830563 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2-serving-cert\") pod \"service-ca-operator-859f94c4b-kmrbv\" (UID: \"5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-kmrbv" Sep 10 12:18:30.830695 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.830689 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 10 12:18:30.830828 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.830592 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 10 12:18:30.831074 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.831057 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 10 12:18:30.831329 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.831305 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 10 12:18:30.831402 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.831337 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 10 12:18:30.831402 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.831361 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 10 12:18:30.831402 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.831385 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-br5wb\"" Sep 10 12:18:30.835865 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.835832 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bkjc\" (UniqueName: \"kubernetes.io/projected/5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2-kube-api-access-8bkjc\") pod \"service-ca-operator-859f94c4b-kmrbv\" (UID: \"5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-kmrbv" Sep 10 12:18:30.901770 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.901747 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-kmrbv" Sep 10 12:18:30.928490 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.928466 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z2cb\" (UniqueName: \"kubernetes.io/projected/a6364599-ee05-41d5-a75c-c9fae216b75d-kube-api-access-7z2cb\") pod \"cluster-monitoring-operator-6cdcc96965-cq4k5\" (UID: \"a6364599-ee05-41d5-a75c-c9fae216b75d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cq4k5" Sep 10 12:18:30.928584 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.928501 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d174be6e-11f3-4679-a7d9-13ad404888f2-default-certificate\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:30.928584 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.928532 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d2f46f3-7193-4c2a-be48-f2458d2a0960-installation-pull-secrets\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:30.928584 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.928561 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d2f46f3-7193-4c2a-be48-f2458d2a0960-image-registry-private-configuration\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:30.928703 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.928587 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-certificates\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:30.928703 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.928611 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a6364599-ee05-41d5-a75c-c9fae216b75d-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-cq4k5\" (UID: \"a6364599-ee05-41d5-a75c-c9fae216b75d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cq4k5" Sep 10 12:18:30.928703 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.928682 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1727f13e-a084-4cc0-a35c-d7fc811a4c40-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-wftws\" (UID: \"1727f13e-a084-4cc0-a35c-d7fc811a4c40\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wftws" Sep 10 12:18:30.928834 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.928712 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d174be6e-11f3-4679-a7d9-13ad404888f2-metrics-certs\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:30.928834 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.928746 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d174be6e-11f3-4679-a7d9-13ad404888f2-service-ca-bundle\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:30.928834 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.928771 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d174be6e-11f3-4679-a7d9-13ad404888f2-stats-auth\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:30.928834 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:30.928802 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 12:18:30.929057 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.928862 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a6364599-ee05-41d5-a75c-c9fae216b75d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cq4k5\" (UID: \"a6364599-ee05-41d5-a75c-c9fae216b75d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cq4k5" Sep 10 12:18:30.929057 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:30.928884 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1727f13e-a084-4cc0-a35c-d7fc811a4c40-samples-operator-tls podName:1727f13e-a084-4cc0-a35c-d7fc811a4c40 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:31.428862512 +0000 UTC m=+39.434081963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1727f13e-a084-4cc0-a35c-d7fc811a4c40-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-wftws" (UID: "1727f13e-a084-4cc0-a35c-d7fc811a4c40") : secret "samples-operator-tls" not found Sep 10 12:18:30.929057 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.928921 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hngp7\" (UniqueName: \"kubernetes.io/projected/d174be6e-11f3-4679-a7d9-13ad404888f2-kube-api-access-hngp7\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:30.929057 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.928958 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-bound-sa-token\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:30.929057 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.929000 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6f7x\" (UniqueName: \"kubernetes.io/projected/1727f13e-a084-4cc0-a35c-d7fc811a4c40-kube-api-access-x6f7x\") pod \"cluster-samples-operator-7cdcfcf76c-wftws\" (UID: \"1727f13e-a084-4cc0-a35c-d7fc811a4c40\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wftws" Sep 10 12:18:30.929057 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.929025 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d2f46f3-7193-4c2a-be48-f2458d2a0960-trusted-ca\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:30.929057 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.929050 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwwd9\" (UniqueName: \"kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-kube-api-access-hwwd9\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:30.929434 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.929090 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-tls\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:30.929434 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.929117 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d2f46f3-7193-4c2a-be48-f2458d2a0960-ca-trust-extracted\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:30.929434 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.929329 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-certificates\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:30.929562 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:30.929445 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 12:18:30.929562 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:30.929459 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bf6588865-dxmvn: secret "image-registry-tls" not found Sep 10 12:18:30.929562 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:30.929521 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-tls podName:8d2f46f3-7193-4c2a-be48-f2458d2a0960 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:31.429503093 +0000 UTC m=+39.434722548 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-tls") pod "image-registry-bf6588865-dxmvn" (UID: "8d2f46f3-7193-4c2a-be48-f2458d2a0960") : secret "image-registry-tls" not found Sep 10 12:18:30.929712 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.929689 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d2f46f3-7193-4c2a-be48-f2458d2a0960-ca-trust-extracted\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:30.930152 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.930133 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d2f46f3-7193-4c2a-be48-f2458d2a0960-trusted-ca\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:30.931284 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.931265 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d2f46f3-7193-4c2a-be48-f2458d2a0960-installation-pull-secrets\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:30.931350 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.931285 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d2f46f3-7193-4c2a-be48-f2458d2a0960-image-registry-private-configuration\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:30.938771 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.938742 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-bound-sa-token\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:30.938771 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.938759 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6f7x\" (UniqueName: \"kubernetes.io/projected/1727f13e-a084-4cc0-a35c-d7fc811a4c40-kube-api-access-x6f7x\") pod \"cluster-samples-operator-7cdcfcf76c-wftws\" (UID: \"1727f13e-a084-4cc0-a35c-d7fc811a4c40\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wftws" Sep 10 12:18:30.938936 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:30.938792 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwwd9\" (UniqueName: \"kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-kube-api-access-hwwd9\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:31.029857 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:31.029809 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a6364599-ee05-41d5-a75c-c9fae216b75d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cq4k5\" (UID: \"a6364599-ee05-41d5-a75c-c9fae216b75d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cq4k5" Sep 10 12:18:31.029997 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:31.029866 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hngp7\" (UniqueName: \"kubernetes.io/projected/d174be6e-11f3-4679-a7d9-13ad404888f2-kube-api-access-hngp7\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:31.029997 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:31.029938 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z2cb\" (UniqueName: \"kubernetes.io/projected/a6364599-ee05-41d5-a75c-c9fae216b75d-kube-api-access-7z2cb\") pod \"cluster-monitoring-operator-6cdcc96965-cq4k5\" (UID: \"a6364599-ee05-41d5-a75c-c9fae216b75d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cq4k5" Sep 10 12:18:31.029997 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:31.029962 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d174be6e-11f3-4679-a7d9-13ad404888f2-default-certificate\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:31.029997 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:31.029967 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 12:18:31.030197 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:31.030006 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a6364599-ee05-41d5-a75c-c9fae216b75d-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-cq4k5\" (UID: \"a6364599-ee05-41d5-a75c-c9fae216b75d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cq4k5" Sep 10 12:18:31.030197 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:31.030044 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6364599-ee05-41d5-a75c-c9fae216b75d-cluster-monitoring-operator-tls podName:a6364599-ee05-41d5-a75c-c9fae216b75d nodeName:}" failed. No retries permitted until 2026-09-10 12:18:31.530025259 +0000 UTC m=+39.535244717 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a6364599-ee05-41d5-a75c-c9fae216b75d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cq4k5" (UID: "a6364599-ee05-41d5-a75c-c9fae216b75d") : secret "cluster-monitoring-operator-tls" not found Sep 10 12:18:31.030197 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:31.030127 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d174be6e-11f3-4679-a7d9-13ad404888f2-metrics-certs\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:31.030197 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:31.030161 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d174be6e-11f3-4679-a7d9-13ad404888f2-service-ca-bundle\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:31.030197 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:31.030183 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d174be6e-11f3-4679-a7d9-13ad404888f2-stats-auth\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:31.030440 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:31.030243 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 12:18:31.030440 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:31.030318 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d174be6e-11f3-4679-a7d9-13ad404888f2-service-ca-bundle podName:d174be6e-11f3-4679-a7d9-13ad404888f2 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:31.53029798 +0000 UTC m=+39.535517436 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d174be6e-11f3-4679-a7d9-13ad404888f2-service-ca-bundle") pod "router-default-85f68cb7f6-gsvtn" (UID: "d174be6e-11f3-4679-a7d9-13ad404888f2") : configmap references non-existent config key: service-ca.crt Sep 10 12:18:31.030440 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:31.030342 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d174be6e-11f3-4679-a7d9-13ad404888f2-metrics-certs podName:d174be6e-11f3-4679-a7d9-13ad404888f2 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:31.530331722 +0000 UTC m=+39.535551173 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d174be6e-11f3-4679-a7d9-13ad404888f2-metrics-certs") pod "router-default-85f68cb7f6-gsvtn" (UID: "d174be6e-11f3-4679-a7d9-13ad404888f2") : secret "router-metrics-certs-default" not found Sep 10 12:18:31.030822 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:31.030800 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a6364599-ee05-41d5-a75c-c9fae216b75d-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-cq4k5\" (UID: \"a6364599-ee05-41d5-a75c-c9fae216b75d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cq4k5" Sep 10 12:18:31.032267 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:31.032252 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d174be6e-11f3-4679-a7d9-13ad404888f2-stats-auth\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:31.032384 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:31.032365 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d174be6e-11f3-4679-a7d9-13ad404888f2-default-certificate\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:31.040403 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:31.040378 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hngp7\" (UniqueName: \"kubernetes.io/projected/d174be6e-11f3-4679-a7d9-13ad404888f2-kube-api-access-hngp7\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:31.040403 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:31.040394 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z2cb\" (UniqueName: \"kubernetes.io/projected/a6364599-ee05-41d5-a75c-c9fae216b75d-kube-api-access-7z2cb\") pod \"cluster-monitoring-operator-6cdcc96965-cq4k5\" (UID: \"a6364599-ee05-41d5-a75c-c9fae216b75d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cq4k5" Sep 10 12:18:31.187936 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:31.187865 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-kmrbv"] Sep 10 12:18:31.191858 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:18:31.191815 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bf4bb00_f3f9_4b88_8f1d_d08391b9bde2.slice/crio-adf2eb32e9855eecd8b019c555f199b710e834f23ec61b204ce37c5b0e692758 WatchSource:0}: Error finding container adf2eb32e9855eecd8b019c555f199b710e834f23ec61b204ce37c5b0e692758: Status 404 returned error can't find the container with id adf2eb32e9855eecd8b019c555f199b710e834f23ec61b204ce37c5b0e692758 Sep 10 12:18:31.434200 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:31.434167 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1727f13e-a084-4cc0-a35c-d7fc811a4c40-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-wftws\" (UID: \"1727f13e-a084-4cc0-a35c-d7fc811a4c40\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wftws" Sep 10 12:18:31.434323 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:31.434238 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-tls\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:31.434363 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:31.434337 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 12:18:31.434363 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:31.434355 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bf6588865-dxmvn: secret "image-registry-tls" not found Sep 10 12:18:31.434422 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:31.434373 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 12:18:31.434422 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:31.434409 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-tls podName:8d2f46f3-7193-4c2a-be48-f2458d2a0960 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:32.434391818 +0000 UTC m=+40.439611275 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-tls") pod "image-registry-bf6588865-dxmvn" (UID: "8d2f46f3-7193-4c2a-be48-f2458d2a0960") : secret "image-registry-tls" not found Sep 10 12:18:31.434487 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:31.434432 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1727f13e-a084-4cc0-a35c-d7fc811a4c40-samples-operator-tls podName:1727f13e-a084-4cc0-a35c-d7fc811a4c40 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:32.434419322 +0000 UTC m=+40.439638774 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1727f13e-a084-4cc0-a35c-d7fc811a4c40-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-wftws" (UID: "1727f13e-a084-4cc0-a35c-d7fc811a4c40") : secret "samples-operator-tls" not found Sep 10 12:18:31.535376 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:31.535299 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a6364599-ee05-41d5-a75c-c9fae216b75d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cq4k5\" (UID: \"a6364599-ee05-41d5-a75c-c9fae216b75d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cq4k5" Sep 10 12:18:31.535507 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:31.535434 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d174be6e-11f3-4679-a7d9-13ad404888f2-metrics-certs\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:31.535507 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:31.535440 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 12:18:31.535507 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:31.535469 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d174be6e-11f3-4679-a7d9-13ad404888f2-service-ca-bundle\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:31.535507 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:31.535491 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6364599-ee05-41d5-a75c-c9fae216b75d-cluster-monitoring-operator-tls podName:a6364599-ee05-41d5-a75c-c9fae216b75d nodeName:}" failed. No retries permitted until 2026-09-10 12:18:32.535476059 +0000 UTC m=+40.540695513 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a6364599-ee05-41d5-a75c-c9fae216b75d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cq4k5" (UID: "a6364599-ee05-41d5-a75c-c9fae216b75d") : secret "cluster-monitoring-operator-tls" not found Sep 10 12:18:31.535676 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:31.535564 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d174be6e-11f3-4679-a7d9-13ad404888f2-service-ca-bundle podName:d174be6e-11f3-4679-a7d9-13ad404888f2 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:32.535548773 +0000 UTC m=+40.540768211 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d174be6e-11f3-4679-a7d9-13ad404888f2-service-ca-bundle") pod "router-default-85f68cb7f6-gsvtn" (UID: "d174be6e-11f3-4679-a7d9-13ad404888f2") : configmap references non-existent config key: service-ca.crt Sep 10 12:18:31.535676 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:31.535571 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 12:18:31.535676 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:31.535624 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d174be6e-11f3-4679-a7d9-13ad404888f2-metrics-certs podName:d174be6e-11f3-4679-a7d9-13ad404888f2 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:32.535609345 +0000 UTC m=+40.540828800 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d174be6e-11f3-4679-a7d9-13ad404888f2-metrics-certs") pod "router-default-85f68cb7f6-gsvtn" (UID: "d174be6e-11f3-4679-a7d9-13ad404888f2") : secret "router-metrics-certs-default" not found Sep 10 12:18:31.673538 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:31.673510 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-kmrbv" event={"ID":"5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2","Type":"ContainerStarted","Data":"adf2eb32e9855eecd8b019c555f199b710e834f23ec61b204ce37c5b0e692758"} Sep 10 12:18:32.442739 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:32.442705 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1727f13e-a084-4cc0-a35c-d7fc811a4c40-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-wftws\" (UID: \"1727f13e-a084-4cc0-a35c-d7fc811a4c40\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wftws" Sep 10 12:18:32.442993 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:32.442814 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-tls\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:32.442993 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:32.442894 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 12:18:32.442993 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:32.442958 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 12:18:32.442993 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:32.442975 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bf6588865-dxmvn: secret "image-registry-tls" not found Sep 10 12:18:32.442993 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:32.442987 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1727f13e-a084-4cc0-a35c-d7fc811a4c40-samples-operator-tls podName:1727f13e-a084-4cc0-a35c-d7fc811a4c40 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:34.442968078 +0000 UTC m=+42.448187515 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1727f13e-a084-4cc0-a35c-d7fc811a4c40-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-wftws" (UID: "1727f13e-a084-4cc0-a35c-d7fc811a4c40") : secret "samples-operator-tls" not found Sep 10 12:18:32.443240 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:32.443022 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-tls podName:8d2f46f3-7193-4c2a-be48-f2458d2a0960 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:34.443007359 +0000 UTC m=+42.448226822 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-tls") pod "image-registry-bf6588865-dxmvn" (UID: "8d2f46f3-7193-4c2a-be48-f2458d2a0960") : secret "image-registry-tls" not found Sep 10 12:18:32.543758 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:32.543727 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d174be6e-11f3-4679-a7d9-13ad404888f2-metrics-certs\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:32.543758 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:32.543763 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d174be6e-11f3-4679-a7d9-13ad404888f2-service-ca-bundle\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:32.543971 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:32.543789 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a6364599-ee05-41d5-a75c-c9fae216b75d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cq4k5\" (UID: \"a6364599-ee05-41d5-a75c-c9fae216b75d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cq4k5" Sep 10 12:18:32.543971 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:32.543916 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 12:18:32.543971 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:32.543934 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d174be6e-11f3-4679-a7d9-13ad404888f2-service-ca-bundle podName:d174be6e-11f3-4679-a7d9-13ad404888f2 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:34.543918887 +0000 UTC m=+42.549138344 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d174be6e-11f3-4679-a7d9-13ad404888f2-service-ca-bundle") pod "router-default-85f68cb7f6-gsvtn" (UID: "d174be6e-11f3-4679-a7d9-13ad404888f2") : configmap references non-existent config key: service-ca.crt Sep 10 12:18:32.543971 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:32.543956 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6364599-ee05-41d5-a75c-c9fae216b75d-cluster-monitoring-operator-tls podName:a6364599-ee05-41d5-a75c-c9fae216b75d nodeName:}" failed. No retries permitted until 2026-09-10 12:18:34.543949711 +0000 UTC m=+42.549169148 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a6364599-ee05-41d5-a75c-c9fae216b75d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cq4k5" (UID: "a6364599-ee05-41d5-a75c-c9fae216b75d") : secret "cluster-monitoring-operator-tls" not found Sep 10 12:18:32.544116 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:32.543984 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 12:18:32.544116 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:32.544025 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d174be6e-11f3-4679-a7d9-13ad404888f2-metrics-certs podName:d174be6e-11f3-4679-a7d9-13ad404888f2 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:34.544010799 +0000 UTC m=+42.549230243 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d174be6e-11f3-4679-a7d9-13ad404888f2-metrics-certs") pod "router-default-85f68cb7f6-gsvtn" (UID: "d174be6e-11f3-4679-a7d9-13ad404888f2") : secret "router-metrics-certs-default" not found Sep 10 12:18:34.056995 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:34.056970 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-metrics-tls\") pod \"dns-default-dssrq\" (UID: \"3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2\") " pod="openshift-dns/dns-default-dssrq" Sep 10 12:18:34.057317 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:34.057087 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 12:18:34.057317 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:34.057133 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-metrics-tls podName:3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:42.057121558 +0000 UTC m=+50.062341009 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-metrics-tls") pod "dns-default-dssrq" (UID: "3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2") : secret "dns-default-metrics-tls" not found Sep 10 12:18:34.461231 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:34.461201 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1727f13e-a084-4cc0-a35c-d7fc811a4c40-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-wftws\" (UID: \"1727f13e-a084-4cc0-a35c-d7fc811a4c40\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wftws" Sep 10 12:18:34.461392 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:34.461369 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 12:18:34.461438 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:34.461370 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-tls\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:34.461469 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:34.461435 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 12:18:34.461469 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:34.461447 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1727f13e-a084-4cc0-a35c-d7fc811a4c40-samples-operator-tls podName:1727f13e-a084-4cc0-a35c-d7fc811a4c40 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:38.461428239 +0000 UTC m=+46.466647679 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1727f13e-a084-4cc0-a35c-d7fc811a4c40-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-wftws" (UID: "1727f13e-a084-4cc0-a35c-d7fc811a4c40") : secret "samples-operator-tls" not found Sep 10 12:18:34.461469 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:34.461452 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bf6588865-dxmvn: secret "image-registry-tls" not found Sep 10 12:18:34.461562 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:34.461488 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-tls podName:8d2f46f3-7193-4c2a-be48-f2458d2a0960 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:38.461479235 +0000 UTC m=+46.466698676 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-tls") pod "image-registry-bf6588865-dxmvn" (UID: "8d2f46f3-7193-4c2a-be48-f2458d2a0960") : secret "image-registry-tls" not found Sep 10 12:18:34.562318 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:34.561963 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d174be6e-11f3-4679-a7d9-13ad404888f2-metrics-certs\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:34.562520 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:34.562370 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d174be6e-11f3-4679-a7d9-13ad404888f2-service-ca-bundle\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:34.562520 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:34.562386 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 12:18:34.562520 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:34.562428 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a6364599-ee05-41d5-a75c-c9fae216b75d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cq4k5\" (UID: \"a6364599-ee05-41d5-a75c-c9fae216b75d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cq4k5" Sep 10 12:18:34.562520 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:34.562465 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d174be6e-11f3-4679-a7d9-13ad404888f2-metrics-certs podName:d174be6e-11f3-4679-a7d9-13ad404888f2 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:38.562437675 +0000 UTC m=+46.567657116 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d174be6e-11f3-4679-a7d9-13ad404888f2-metrics-certs") pod "router-default-85f68cb7f6-gsvtn" (UID: "d174be6e-11f3-4679-a7d9-13ad404888f2") : secret "router-metrics-certs-default" not found Sep 10 12:18:34.562713 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:34.562528 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 12:18:34.562713 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:34.562585 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6364599-ee05-41d5-a75c-c9fae216b75d-cluster-monitoring-operator-tls podName:a6364599-ee05-41d5-a75c-c9fae216b75d nodeName:}" failed. No retries permitted until 2026-09-10 12:18:38.562563074 +0000 UTC m=+46.567782514 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a6364599-ee05-41d5-a75c-c9fae216b75d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cq4k5" (UID: "a6364599-ee05-41d5-a75c-c9fae216b75d") : secret "cluster-monitoring-operator-tls" not found Sep 10 12:18:34.562713 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:34.562692 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d174be6e-11f3-4679-a7d9-13ad404888f2-service-ca-bundle podName:d174be6e-11f3-4679-a7d9-13ad404888f2 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:38.562668402 +0000 UTC m=+46.567887855 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d174be6e-11f3-4679-a7d9-13ad404888f2-service-ca-bundle") pod "router-default-85f68cb7f6-gsvtn" (UID: "d174be6e-11f3-4679-a7d9-13ad404888f2") : configmap references non-existent config key: service-ca.crt Sep 10 12:18:34.681115 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:34.681082 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-kmrbv" event={"ID":"5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2","Type":"ContainerStarted","Data":"25eedb09e7488aad200199fb6d751d39198c97ce95f9a41dab0a6299a2fe9c99"} Sep 10 12:18:34.698367 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:34.698326 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-kmrbv" podStartSLOduration=1.908305103 podStartE2EDuration="4.698309485s" totalImagesPullingTime="2.790004382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:18:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:34.697385955 +0000 UTC m=+42.702605420" watchObservedRunningTime="2026-09-10 12:18:34.698309485 +0000 UTC m=+42.703528944" Sep 10 12:18:37.135640 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.135606 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pqcdq"] Sep 10 12:18:37.164591 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.164566 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pqcdq"] Sep 10 12:18:37.164722 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.164661 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:37.168177 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.168157 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 10 12:18:37.169180 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.169162 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 10 12:18:37.169325 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.169299 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 10 12:18:37.169325 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.169309 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 10 12:18:37.169477 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.169368 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zcn9n\"" Sep 10 12:18:37.285398 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.285365 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/abfa64ea-18f1-4a06-9292-15a385850835-crio-socket\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:37.285541 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.285428 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abfa64ea-18f1-4a06-9292-15a385850835-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:37.285541 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.285514 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqpkk\" (UniqueName: \"kubernetes.io/projected/abfa64ea-18f1-4a06-9292-15a385850835-kube-api-access-fqpkk\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:37.285633 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.285562 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/abfa64ea-18f1-4a06-9292-15a385850835-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:37.285633 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.285578 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/abfa64ea-18f1-4a06-9292-15a385850835-data-volume\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:37.386190 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.386120 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/abfa64ea-18f1-4a06-9292-15a385850835-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:37.386323 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.386189 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/abfa64ea-18f1-4a06-9292-15a385850835-data-volume\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:37.386323 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.386225 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/abfa64ea-18f1-4a06-9292-15a385850835-crio-socket\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:37.386323 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:37.386239 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 12:18:37.386323 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.386280 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abfa64ea-18f1-4a06-9292-15a385850835-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:37.386323 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:37.386303 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/abfa64ea-18f1-4a06-9292-15a385850835-kube-rbac-proxy-cm podName:abfa64ea-18f1-4a06-9292-15a385850835 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:37.886284149 +0000 UTC m=+45.891503588 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/abfa64ea-18f1-4a06-9292-15a385850835-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pqcdq" (UID: "abfa64ea-18f1-4a06-9292-15a385850835") : configmap "kube-rbac-proxy" not found Sep 10 12:18:37.386578 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:37.386376 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 12:18:37.386578 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.386414 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqpkk\" (UniqueName: \"kubernetes.io/projected/abfa64ea-18f1-4a06-9292-15a385850835-kube-api-access-fqpkk\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:37.386578 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:37.386440 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abfa64ea-18f1-4a06-9292-15a385850835-insights-runtime-extractor-tls podName:abfa64ea-18f1-4a06-9292-15a385850835 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:37.886428275 +0000 UTC m=+45.891647729 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/abfa64ea-18f1-4a06-9292-15a385850835-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pqcdq" (UID: "abfa64ea-18f1-4a06-9292-15a385850835") : secret "insights-runtime-extractor-tls" not found Sep 10 12:18:37.386578 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.386485 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/abfa64ea-18f1-4a06-9292-15a385850835-crio-socket\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:37.386735 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.386622 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/abfa64ea-18f1-4a06-9292-15a385850835-data-volume\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:37.406066 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.406047 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqpkk\" (UniqueName: \"kubernetes.io/projected/abfa64ea-18f1-4a06-9292-15a385850835-kube-api-access-fqpkk\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:37.426812 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.426788 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-2vg96"] Sep 10 12:18:37.450038 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.450012 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-2vg96"] Sep 10 12:18:37.450156 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.450112 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-2vg96" Sep 10 12:18:37.452710 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.452676 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 10 12:18:37.452710 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.452674 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 10 12:18:37.452908 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.452744 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 10 12:18:37.452908 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.452680 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-k2zw7\"" Sep 10 12:18:37.452908 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.452681 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 10 12:18:37.587878 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.587824 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ed074ff5-21b2-4590-9b9f-55cd870f7664-signing-key\") pod \"service-ca-75d99c484-2vg96\" (UID: \"ed074ff5-21b2-4590-9b9f-55cd870f7664\") " pod="openshift-service-ca/service-ca-75d99c484-2vg96" Sep 10 12:18:37.588052 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.587912 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng9bm\" (UniqueName: \"kubernetes.io/projected/ed074ff5-21b2-4590-9b9f-55cd870f7664-kube-api-access-ng9bm\") pod \"service-ca-75d99c484-2vg96\" (UID: \"ed074ff5-21b2-4590-9b9f-55cd870f7664\") " pod="openshift-service-ca/service-ca-75d99c484-2vg96" Sep 10 12:18:37.588052 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.587969 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ed074ff5-21b2-4590-9b9f-55cd870f7664-signing-cabundle\") pod \"service-ca-75d99c484-2vg96\" (UID: \"ed074ff5-21b2-4590-9b9f-55cd870f7664\") " pod="openshift-service-ca/service-ca-75d99c484-2vg96" Sep 10 12:18:37.688941 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.688868 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng9bm\" (UniqueName: \"kubernetes.io/projected/ed074ff5-21b2-4590-9b9f-55cd870f7664-kube-api-access-ng9bm\") pod \"service-ca-75d99c484-2vg96\" (UID: \"ed074ff5-21b2-4590-9b9f-55cd870f7664\") " pod="openshift-service-ca/service-ca-75d99c484-2vg96" Sep 10 12:18:37.688941 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.688933 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ed074ff5-21b2-4590-9b9f-55cd870f7664-signing-cabundle\") pod \"service-ca-75d99c484-2vg96\" (UID: \"ed074ff5-21b2-4590-9b9f-55cd870f7664\") " pod="openshift-service-ca/service-ca-75d99c484-2vg96" Sep 10 12:18:37.689141 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.689012 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ed074ff5-21b2-4590-9b9f-55cd870f7664-signing-key\") pod \"service-ca-75d99c484-2vg96\" (UID: \"ed074ff5-21b2-4590-9b9f-55cd870f7664\") " pod="openshift-service-ca/service-ca-75d99c484-2vg96" Sep 10 12:18:37.689583 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.689562 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ed074ff5-21b2-4590-9b9f-55cd870f7664-signing-cabundle\") pod \"service-ca-75d99c484-2vg96\" (UID: \"ed074ff5-21b2-4590-9b9f-55cd870f7664\") " pod="openshift-service-ca/service-ca-75d99c484-2vg96" Sep 10 12:18:37.691470 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.691447 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ed074ff5-21b2-4590-9b9f-55cd870f7664-signing-key\") pod \"service-ca-75d99c484-2vg96\" (UID: \"ed074ff5-21b2-4590-9b9f-55cd870f7664\") " pod="openshift-service-ca/service-ca-75d99c484-2vg96" Sep 10 12:18:37.702541 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.702513 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng9bm\" (UniqueName: \"kubernetes.io/projected/ed074ff5-21b2-4590-9b9f-55cd870f7664-kube-api-access-ng9bm\") pod \"service-ca-75d99c484-2vg96\" (UID: \"ed074ff5-21b2-4590-9b9f-55cd870f7664\") " pod="openshift-service-ca/service-ca-75d99c484-2vg96" Sep 10 12:18:37.758719 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.758692 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-2vg96" Sep 10 12:18:37.890224 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.890195 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/abfa64ea-18f1-4a06-9292-15a385850835-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:37.890357 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:37.890254 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abfa64ea-18f1-4a06-9292-15a385850835-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:37.890357 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:37.890327 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 12:18:37.890357 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:37.890350 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 12:18:37.890499 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:37.890410 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/abfa64ea-18f1-4a06-9292-15a385850835-kube-rbac-proxy-cm podName:abfa64ea-18f1-4a06-9292-15a385850835 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:38.890386831 +0000 UTC m=+46.895606287 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/abfa64ea-18f1-4a06-9292-15a385850835-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pqcdq" (UID: "abfa64ea-18f1-4a06-9292-15a385850835") : configmap "kube-rbac-proxy" not found Sep 10 12:18:37.890499 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:37.890432 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abfa64ea-18f1-4a06-9292-15a385850835-insights-runtime-extractor-tls podName:abfa64ea-18f1-4a06-9292-15a385850835 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:38.890422766 +0000 UTC m=+46.895642230 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/abfa64ea-18f1-4a06-9292-15a385850835-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pqcdq" (UID: "abfa64ea-18f1-4a06-9292-15a385850835") : secret "insights-runtime-extractor-tls" not found Sep 10 12:18:38.048702 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:38.048673 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-2vg96"] Sep 10 12:18:38.051603 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:18:38.051567 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded074ff5_21b2_4590_9b9f_55cd870f7664.slice/crio-5732e1cfe94d1eaa21bcf734554b203d90cde24ca4d40ea72f7ec03041625ca2 WatchSource:0}: Error finding container 5732e1cfe94d1eaa21bcf734554b203d90cde24ca4d40ea72f7ec03041625ca2: Status 404 returned error can't find the container with id 5732e1cfe94d1eaa21bcf734554b203d90cde24ca4d40ea72f7ec03041625ca2 Sep 10 12:18:38.483855 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:38.483770 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lm7qv_f7244cbf-0a28-477e-9012-5dbe58beaa77/dns-node-resolver/0.log" Sep 10 12:18:38.496196 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:38.496173 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-tls\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:38.496289 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:38.496231 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1727f13e-a084-4cc0-a35c-d7fc811a4c40-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-wftws\" (UID: \"1727f13e-a084-4cc0-a35c-d7fc811a4c40\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wftws" Sep 10 12:18:38.496342 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:38.496328 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 12:18:38.496377 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:38.496345 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bf6588865-dxmvn: secret "image-registry-tls" not found Sep 10 12:18:38.496421 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:38.496404 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 12:18:38.496459 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:38.496407 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-tls podName:8d2f46f3-7193-4c2a-be48-f2458d2a0960 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:46.49639103 +0000 UTC m=+54.501610481 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-tls") pod "image-registry-bf6588865-dxmvn" (UID: "8d2f46f3-7193-4c2a-be48-f2458d2a0960") : secret "image-registry-tls" not found Sep 10 12:18:38.496501 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:38.496476 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1727f13e-a084-4cc0-a35c-d7fc811a4c40-samples-operator-tls podName:1727f13e-a084-4cc0-a35c-d7fc811a4c40 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:46.496459037 +0000 UTC m=+54.501678481 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1727f13e-a084-4cc0-a35c-d7fc811a4c40-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-wftws" (UID: "1727f13e-a084-4cc0-a35c-d7fc811a4c40") : secret "samples-operator-tls" not found Sep 10 12:18:38.597281 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:38.597252 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d174be6e-11f3-4679-a7d9-13ad404888f2-service-ca-bundle\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:38.597400 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:38.597285 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a6364599-ee05-41d5-a75c-c9fae216b75d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cq4k5\" (UID: \"a6364599-ee05-41d5-a75c-c9fae216b75d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cq4k5" Sep 10 12:18:38.597400 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:38.597377 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d174be6e-11f3-4679-a7d9-13ad404888f2-metrics-certs\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:38.597520 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:38.597440 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d174be6e-11f3-4679-a7d9-13ad404888f2-service-ca-bundle podName:d174be6e-11f3-4679-a7d9-13ad404888f2 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:46.597418948 +0000 UTC m=+54.602638409 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d174be6e-11f3-4679-a7d9-13ad404888f2-service-ca-bundle") pod "router-default-85f68cb7f6-gsvtn" (UID: "d174be6e-11f3-4679-a7d9-13ad404888f2") : configmap references non-existent config key: service-ca.crt Sep 10 12:18:38.597520 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:38.597461 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 12:18:38.597520 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:38.597465 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 12:18:38.597520 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:38.597495 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d174be6e-11f3-4679-a7d9-13ad404888f2-metrics-certs podName:d174be6e-11f3-4679-a7d9-13ad404888f2 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:46.597484671 +0000 UTC m=+54.602704127 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d174be6e-11f3-4679-a7d9-13ad404888f2-metrics-certs") pod "router-default-85f68cb7f6-gsvtn" (UID: "d174be6e-11f3-4679-a7d9-13ad404888f2") : secret "router-metrics-certs-default" not found Sep 10 12:18:38.597520 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:38.597515 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6364599-ee05-41d5-a75c-c9fae216b75d-cluster-monitoring-operator-tls podName:a6364599-ee05-41d5-a75c-c9fae216b75d nodeName:}" failed. No retries permitted until 2026-09-10 12:18:46.597500509 +0000 UTC m=+54.602719970 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a6364599-ee05-41d5-a75c-c9fae216b75d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cq4k5" (UID: "a6364599-ee05-41d5-a75c-c9fae216b75d") : secret "cluster-monitoring-operator-tls" not found Sep 10 12:18:38.692415 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:38.692384 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-2vg96" event={"ID":"ed074ff5-21b2-4590-9b9f-55cd870f7664","Type":"ContainerStarted","Data":"5d961808aeae621196198e86aa20f909d871a1dd14d69d98717c52260ea9f366"} Sep 10 12:18:38.692529 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:38.692431 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-2vg96" event={"ID":"ed074ff5-21b2-4590-9b9f-55cd870f7664","Type":"ContainerStarted","Data":"5732e1cfe94d1eaa21bcf734554b203d90cde24ca4d40ea72f7ec03041625ca2"} Sep 10 12:18:38.898731 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:38.898701 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abfa64ea-18f1-4a06-9292-15a385850835-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:38.898889 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:38.898819 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/abfa64ea-18f1-4a06-9292-15a385850835-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:38.898889 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:38.898851 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 12:18:38.898964 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:38.898913 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abfa64ea-18f1-4a06-9292-15a385850835-insights-runtime-extractor-tls podName:abfa64ea-18f1-4a06-9292-15a385850835 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:40.898898637 +0000 UTC m=+48.904118075 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/abfa64ea-18f1-4a06-9292-15a385850835-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pqcdq" (UID: "abfa64ea-18f1-4a06-9292-15a385850835") : secret "insights-runtime-extractor-tls" not found Sep 10 12:18:38.898964 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:38.898927 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 12:18:38.899036 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:38.898965 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/abfa64ea-18f1-4a06-9292-15a385850835-kube-rbac-proxy-cm podName:abfa64ea-18f1-4a06-9292-15a385850835 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:40.898955047 +0000 UTC m=+48.904174497 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/abfa64ea-18f1-4a06-9292-15a385850835-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pqcdq" (UID: "abfa64ea-18f1-4a06-9292-15a385850835") : configmap "kube-rbac-proxy" not found Sep 10 12:18:39.884348 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:39.884317 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vqcn2_a1521572-7823-459a-a435-0248c1ae1b12/node-ca/0.log" Sep 10 12:18:40.299012 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:40.298982 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kgd4g_d295dd14-e004-4178-9b90-086fe26e6778/serve-healthcheck-canary/0.log" Sep 10 12:18:40.914398 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:40.914364 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/abfa64ea-18f1-4a06-9292-15a385850835-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:40.914789 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:40.914431 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abfa64ea-18f1-4a06-9292-15a385850835-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:40.914789 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:40.914482 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 12:18:40.914789 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:40.914550 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/abfa64ea-18f1-4a06-9292-15a385850835-kube-rbac-proxy-cm podName:abfa64ea-18f1-4a06-9292-15a385850835 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:44.914532096 +0000 UTC m=+52.919751557 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/abfa64ea-18f1-4a06-9292-15a385850835-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pqcdq" (UID: "abfa64ea-18f1-4a06-9292-15a385850835") : configmap "kube-rbac-proxy" not found Sep 10 12:18:40.914789 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:40.914570 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 12:18:40.914789 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:40.914621 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abfa64ea-18f1-4a06-9292-15a385850835-insights-runtime-extractor-tls podName:abfa64ea-18f1-4a06-9292-15a385850835 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:44.914608939 +0000 UTC m=+52.919828393 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/abfa64ea-18f1-4a06-9292-15a385850835-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pqcdq" (UID: "abfa64ea-18f1-4a06-9292-15a385850835") : secret "insights-runtime-extractor-tls" not found Sep 10 12:18:42.123410 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:42.123369 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-metrics-tls\") pod \"dns-default-dssrq\" (UID: \"3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2\") " pod="openshift-dns/dns-default-dssrq" Sep 10 12:18:42.123772 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:42.123499 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 12:18:42.123772 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:42.123615 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-metrics-tls podName:3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:58.123593712 +0000 UTC m=+66.128813165 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-metrics-tls") pod "dns-default-dssrq" (UID: "3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2") : secret "dns-default-metrics-tls" not found Sep 10 12:18:44.944441 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:44.944399 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/abfa64ea-18f1-4a06-9292-15a385850835-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:44.944939 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:44.944461 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abfa64ea-18f1-4a06-9292-15a385850835-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:44.944939 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:44.944528 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 12:18:44.944939 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:44.944548 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 12:18:44.944939 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:44.944612 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/abfa64ea-18f1-4a06-9292-15a385850835-kube-rbac-proxy-cm podName:abfa64ea-18f1-4a06-9292-15a385850835 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:52.944589918 +0000 UTC m=+60.949809642 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/abfa64ea-18f1-4a06-9292-15a385850835-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pqcdq" (UID: "abfa64ea-18f1-4a06-9292-15a385850835") : configmap "kube-rbac-proxy" not found Sep 10 12:18:44.944939 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:44.944633 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abfa64ea-18f1-4a06-9292-15a385850835-insights-runtime-extractor-tls podName:abfa64ea-18f1-4a06-9292-15a385850835 nodeName:}" failed. No retries permitted until 2026-09-10 12:18:52.944623962 +0000 UTC m=+60.949843401 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/abfa64ea-18f1-4a06-9292-15a385850835-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pqcdq" (UID: "abfa64ea-18f1-4a06-9292-15a385850835") : secret "insights-runtime-extractor-tls" not found Sep 10 12:18:46.559370 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:46.559342 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-tls\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:46.559862 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:46.559395 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1727f13e-a084-4cc0-a35c-d7fc811a4c40-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-wftws\" (UID: \"1727f13e-a084-4cc0-a35c-d7fc811a4c40\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wftws" Sep 10 12:18:46.561730 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:46.561704 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-tls\") pod \"image-registry-bf6588865-dxmvn\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:46.561730 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:46.561716 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1727f13e-a084-4cc0-a35c-d7fc811a4c40-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-wftws\" (UID: \"1727f13e-a084-4cc0-a35c-d7fc811a4c40\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wftws" Sep 10 12:18:46.616585 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:46.616558 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wftws" Sep 10 12:18:46.630968 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:46.630943 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:46.660224 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:46.660190 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d174be6e-11f3-4679-a7d9-13ad404888f2-metrics-certs\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:46.660390 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:46.660248 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d174be6e-11f3-4679-a7d9-13ad404888f2-service-ca-bundle\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:46.660390 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:46.660282 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a6364599-ee05-41d5-a75c-c9fae216b75d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cq4k5\" (UID: \"a6364599-ee05-41d5-a75c-c9fae216b75d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cq4k5" Sep 10 12:18:46.660516 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:46.660489 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 12:18:46.660578 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:18:46.660553 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6364599-ee05-41d5-a75c-c9fae216b75d-cluster-monitoring-operator-tls podName:a6364599-ee05-41d5-a75c-c9fae216b75d nodeName:}" failed. No retries permitted until 2026-09-10 12:19:02.660533624 +0000 UTC m=+70.665753081 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a6364599-ee05-41d5-a75c-c9fae216b75d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-cq4k5" (UID: "a6364599-ee05-41d5-a75c-c9fae216b75d") : secret "cluster-monitoring-operator-tls" not found Sep 10 12:18:46.661467 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:46.661431 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d174be6e-11f3-4679-a7d9-13ad404888f2-service-ca-bundle\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:46.664774 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:46.663990 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d174be6e-11f3-4679-a7d9-13ad404888f2-metrics-certs\") pod \"router-default-85f68cb7f6-gsvtn\" (UID: \"d174be6e-11f3-4679-a7d9-13ad404888f2\") " pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:46.744290 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:46.744256 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:46.885882 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:46.885768 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-2vg96" podStartSLOduration=9.885750472 podStartE2EDuration="9.885750472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:18:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:38.711402544 +0000 UTC m=+46.716622004" watchObservedRunningTime="2026-09-10 12:18:46.885750472 +0000 UTC m=+54.890969931" Sep 10 12:18:46.886025 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:46.885914 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wftws"] Sep 10 12:18:46.920828 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:46.920803 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bf6588865-dxmvn"] Sep 10 12:18:46.923485 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:18:46.923463 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d2f46f3_7193_4c2a_be48_f2458d2a0960.slice/crio-0eaf70ddad9d154080b5aec06ba9f4fe6603da8f43c2e09bc918df568d763d07 WatchSource:0}: Error finding container 0eaf70ddad9d154080b5aec06ba9f4fe6603da8f43c2e09bc918df568d763d07: Status 404 returned error can't find the container with id 0eaf70ddad9d154080b5aec06ba9f4fe6603da8f43c2e09bc918df568d763d07 Sep 10 12:18:47.016419 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:47.016391 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-85f68cb7f6-gsvtn"] Sep 10 12:18:47.019228 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:18:47.019197 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd174be6e_11f3_4679_a7d9_13ad404888f2.slice/crio-e9fbbf862254e8101da923fad0f8aff651ebc3d8f82f71fbf97d8ae83325f285 WatchSource:0}: Error finding container e9fbbf862254e8101da923fad0f8aff651ebc3d8f82f71fbf97d8ae83325f285: Status 404 returned error can't find the container with id e9fbbf862254e8101da923fad0f8aff651ebc3d8f82f71fbf97d8ae83325f285 Sep 10 12:18:47.722664 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:47.722622 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" event={"ID":"d174be6e-11f3-4679-a7d9-13ad404888f2","Type":"ContainerStarted","Data":"8cf2bfe0bb4324386b770906721adbe5e2bd2e71ff17832f0db5e3b23395d659"} Sep 10 12:18:47.723173 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:47.722725 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" event={"ID":"d174be6e-11f3-4679-a7d9-13ad404888f2","Type":"ContainerStarted","Data":"e9fbbf862254e8101da923fad0f8aff651ebc3d8f82f71fbf97d8ae83325f285"} Sep 10 12:18:47.724574 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:47.724548 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bf6588865-dxmvn" event={"ID":"8d2f46f3-7193-4c2a-be48-f2458d2a0960","Type":"ContainerStarted","Data":"371fe2547bf91ab216b903a908b2e01f1f1e76f7f2eed511a0173ecaafa67a18"} Sep 10 12:18:47.724694 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:47.724592 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bf6588865-dxmvn" event={"ID":"8d2f46f3-7193-4c2a-be48-f2458d2a0960","Type":"ContainerStarted","Data":"0eaf70ddad9d154080b5aec06ba9f4fe6603da8f43c2e09bc918df568d763d07"} Sep 10 12:18:47.724755 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:47.724705 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:18:47.725776 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:47.725754 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wftws" event={"ID":"1727f13e-a084-4cc0-a35c-d7fc811a4c40","Type":"ContainerStarted","Data":"cb8c2828e5ec61daddc8b5f9452690d29d66137529ee71b8636df40c43f7d50b"} Sep 10 12:18:47.745365 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:47.745343 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:47.747015 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:47.746946 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" podStartSLOduration=17.746930671 podStartE2EDuration="17.746930671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:18:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:47.745962413 +0000 UTC m=+55.751181874" watchObservedRunningTime="2026-09-10 12:18:47.746930671 +0000 UTC m=+55.752150129" Sep 10 12:18:47.748383 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:47.748362 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:47.772208 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:47.771947 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-bf6588865-dxmvn" podStartSLOduration=17.771934412 podStartE2EDuration="17.771934412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:18:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:47.771551981 +0000 UTC m=+55.776771443" watchObservedRunningTime="2026-09-10 12:18:47.771934412 +0000 UTC m=+55.777153874" Sep 10 12:18:48.732268 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:48.732222 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wftws" event={"ID":"1727f13e-a084-4cc0-a35c-d7fc811a4c40","Type":"ContainerStarted","Data":"0a53300c4a397b91d4a98915d8a1f62b62d1e7fb99614c55599e27eaa65ff9ed"} Sep 10 12:18:48.732688 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:48.732301 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wftws" event={"ID":"1727f13e-a084-4cc0-a35c-d7fc811a4c40","Type":"ContainerStarted","Data":"9810e531b1d1356a19b37a147756403993e44cf769b5bfe85f7278121b3a1dc8"} Sep 10 12:18:48.732799 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:48.732778 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:48.734087 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:48.734047 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-85f68cb7f6-gsvtn" Sep 10 12:18:48.752387 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:48.752336 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-wftws" podStartSLOduration=17.075881551 podStartE2EDuration="18.752322304s" totalImagesPullingTime="1.676440753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:18:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:48.751452252 +0000 UTC m=+56.756671711" watchObservedRunningTime="2026-09-10 12:18:48.752322304 +0000 UTC m=+56.757541765" Sep 10 12:18:51.643326 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:51.643295 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4qqqh" Sep 10 12:18:53.007933 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:53.007887 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/abfa64ea-18f1-4a06-9292-15a385850835-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:53.008365 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:53.007947 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abfa64ea-18f1-4a06-9292-15a385850835-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:53.008405 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:53.008386 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/abfa64ea-18f1-4a06-9292-15a385850835-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:53.010243 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:53.010221 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/abfa64ea-18f1-4a06-9292-15a385850835-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pqcdq\" (UID: \"abfa64ea-18f1-4a06-9292-15a385850835\") " pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:53.076364 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:53.076339 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zcn9n\"" Sep 10 12:18:53.084236 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:53.084218 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pqcdq" Sep 10 12:18:53.355729 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:53.355691 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pqcdq"] Sep 10 12:18:53.358476 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:18:53.358450 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabfa64ea_18f1_4a06_9292_15a385850835.slice/crio-f2a31e78f9cbfe0edb7370b8c264c3d65a50d6ab8bb8edff026d7314b0cf35a8 WatchSource:0}: Error finding container f2a31e78f9cbfe0edb7370b8c264c3d65a50d6ab8bb8edff026d7314b0cf35a8: Status 404 returned error can't find the container with id f2a31e78f9cbfe0edb7370b8c264c3d65a50d6ab8bb8edff026d7314b0cf35a8 Sep 10 12:18:53.748385 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:53.748303 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pqcdq" event={"ID":"abfa64ea-18f1-4a06-9292-15a385850835","Type":"ContainerStarted","Data":"bc89974759f42451b4806069a5d9bb7759c44b9fbdc86a21f1efe12a04557426"} Sep 10 12:18:53.748385 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:53.748366 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pqcdq" event={"ID":"abfa64ea-18f1-4a06-9292-15a385850835","Type":"ContainerStarted","Data":"f2a31e78f9cbfe0edb7370b8c264c3d65a50d6ab8bb8edff026d7314b0cf35a8"} Sep 10 12:18:54.753657 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:54.753579 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pqcdq" event={"ID":"abfa64ea-18f1-4a06-9292-15a385850835","Type":"ContainerStarted","Data":"b303d1415079181045ac972a61997327c0d294a971fdc4a5665ae3d904de5a97"} Sep 10 12:18:56.763259 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:56.763226 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pqcdq" event={"ID":"abfa64ea-18f1-4a06-9292-15a385850835","Type":"ContainerStarted","Data":"6db6f71401e2f3e4b339db7380eb7d3a3133f53477b04846a0ef41f3e57a8f5a"} Sep 10 12:18:56.787166 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:56.787123 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pqcdq" podStartSLOduration=17.344125449 podStartE2EDuration="19.787108846s" totalImagesPullingTime="2.442983397s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:18:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:56.785963712 +0000 UTC m=+64.791183173" watchObservedRunningTime="2026-09-10 12:18:56.787108846 +0000 UTC m=+64.792328306" Sep 10 12:18:58.148229 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.148197 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6ad9571-3788-4c07-8778-c647b1a2cdce-metrics-certs\") pod \"network-metrics-daemon-6j4mf\" (UID: \"c6ad9571-3788-4c07-8778-c647b1a2cdce\") " pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:18:58.148687 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.148257 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-metrics-tls\") pod \"dns-default-dssrq\" (UID: \"3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2\") " pod="openshift-dns/dns-default-dssrq" Sep 10 12:18:58.150465 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.150443 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2-metrics-tls\") pod \"dns-default-dssrq\" (UID: \"3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2\") " pod="openshift-dns/dns-default-dssrq" Sep 10 12:18:58.151107 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.151086 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 12:18:58.161300 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.161278 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6ad9571-3788-4c07-8778-c647b1a2cdce-metrics-certs\") pod \"network-metrics-daemon-6j4mf\" (UID: \"c6ad9571-3788-4c07-8778-c647b1a2cdce\") " pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:18:58.286529 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.286501 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wdbdh\"" Sep 10 12:18:58.294148 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.294128 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6j4mf" Sep 10 12:18:58.349530 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.349505 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr48g\" (UniqueName: \"kubernetes.io/projected/1aa57da6-29e7-4701-8b1b-be81cf4ebed6-kube-api-access-xr48g\") pod \"network-check-target-k9bgv\" (UID: \"1aa57da6-29e7-4701-8b1b-be81cf4ebed6\") " pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:18:58.349630 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.349554 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6960f97e-f623-4368-b27a-41d81ac1be1f-original-pull-secret\") pod \"global-pull-secret-syncer-dgddq\" (UID: \"6960f97e-f623-4368-b27a-41d81ac1be1f\") " pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:18:58.352870 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.352829 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 12:18:58.352959 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.352828 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 12:18:58.362153 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.362128 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6960f97e-f623-4368-b27a-41d81ac1be1f-original-pull-secret\") pod \"global-pull-secret-syncer-dgddq\" (UID: \"6960f97e-f623-4368-b27a-41d81ac1be1f\") " pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:18:58.362569 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.362549 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 12:18:58.373857 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.373397 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr48g\" (UniqueName: \"kubernetes.io/projected/1aa57da6-29e7-4701-8b1b-be81cf4ebed6-kube-api-access-xr48g\") pod \"network-check-target-k9bgv\" (UID: \"1aa57da6-29e7-4701-8b1b-be81cf4ebed6\") " pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:18:58.411142 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.411082 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lhd7h\"" Sep 10 12:18:58.419293 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.419272 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dssrq" Sep 10 12:18:58.576465 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.576440 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgddq" Sep 10 12:18:58.592308 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.592283 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cfjr5\"" Sep 10 12:18:58.594680 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.594658 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6j4mf"] Sep 10 12:18:58.597209 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:18:58.597185 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6ad9571_3788_4c07_8778_c647b1a2cdce.slice/crio-f0463bb11209ed5b433747b20e52e517759ad67bb4abf2e804e6d1400f38f57e WatchSource:0}: Error finding container f0463bb11209ed5b433747b20e52e517759ad67bb4abf2e804e6d1400f38f57e: Status 404 returned error can't find the container with id f0463bb11209ed5b433747b20e52e517759ad67bb4abf2e804e6d1400f38f57e Sep 10 12:18:58.600132 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.600108 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:18:58.719910 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.719878 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dssrq"] Sep 10 12:18:58.722208 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:18:58.722185 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c3ec4cf_9df3_42a2_829b_27b75b5e0ab2.slice/crio-58e63e95498ad5a8ed0f3b743203de06a8017f33970323701d257b29e0158a39 WatchSource:0}: Error finding container 58e63e95498ad5a8ed0f3b743203de06a8017f33970323701d257b29e0158a39: Status 404 returned error can't find the container with id 58e63e95498ad5a8ed0f3b743203de06a8017f33970323701d257b29e0158a39 Sep 10 12:18:58.770051 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.770020 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dssrq" event={"ID":"3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2","Type":"ContainerStarted","Data":"58e63e95498ad5a8ed0f3b743203de06a8017f33970323701d257b29e0158a39"} Sep 10 12:18:58.771177 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.771157 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6j4mf" event={"ID":"c6ad9571-3788-4c07-8778-c647b1a2cdce","Type":"ContainerStarted","Data":"f0463bb11209ed5b433747b20e52e517759ad67bb4abf2e804e6d1400f38f57e"} Sep 10 12:18:58.867678 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.867656 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dgddq"] Sep 10 12:18:58.870027 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:18:58.870005 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6960f97e_f623_4368_b27a_41d81ac1be1f.slice/crio-0639718fa5766edf59b1da8e64a30fa2df8c1b0206c7aa719fe1a233b0ef6c0a WatchSource:0}: Error finding container 0639718fa5766edf59b1da8e64a30fa2df8c1b0206c7aa719fe1a233b0ef6c0a: Status 404 returned error can't find the container with id 0639718fa5766edf59b1da8e64a30fa2df8c1b0206c7aa719fe1a233b0ef6c0a Sep 10 12:18:58.907076 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:58.907055 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k9bgv"] Sep 10 12:18:58.909960 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:18:58.909936 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aa57da6_29e7_4701_8b1b_be81cf4ebed6.slice/crio-05c04ed66932e825b2a63533efa4ae4722df575873aa760848c41898d23d6915 WatchSource:0}: Error finding container 05c04ed66932e825b2a63533efa4ae4722df575873aa760848c41898d23d6915: Status 404 returned error can't find the container with id 05c04ed66932e825b2a63533efa4ae4722df575873aa760848c41898d23d6915 Sep 10 12:18:59.777722 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:59.777683 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dgddq" event={"ID":"6960f97e-f623-4368-b27a-41d81ac1be1f","Type":"ContainerStarted","Data":"0639718fa5766edf59b1da8e64a30fa2df8c1b0206c7aa719fe1a233b0ef6c0a"} Sep 10 12:18:59.779913 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:18:59.779872 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k9bgv" event={"ID":"1aa57da6-29e7-4701-8b1b-be81cf4ebed6","Type":"ContainerStarted","Data":"05c04ed66932e825b2a63533efa4ae4722df575873aa760848c41898d23d6915"} Sep 10 12:19:00.788066 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:00.787964 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6j4mf" event={"ID":"c6ad9571-3788-4c07-8778-c647b1a2cdce","Type":"ContainerStarted","Data":"fcd2619eb10cdc7a5f6e7b9f30181fe405a2e800557ec00940766b2f1383251b"} Sep 10 12:19:00.788066 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:00.788032 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6j4mf" event={"ID":"c6ad9571-3788-4c07-8778-c647b1a2cdce","Type":"ContainerStarted","Data":"95442b74c33d0fd539549f272377ab2de998f096c57ed3a12159f07a815f5a99"} Sep 10 12:19:00.808650 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:00.807125 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6j4mf" podStartSLOduration=67.517063137 podStartE2EDuration="1m8.807105406s" totalImagesPullingTime="1.290042269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:17:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:00.80661023 +0000 UTC m=+68.811829689" watchObservedRunningTime="2026-09-10 12:19:00.807105406 +0000 UTC m=+68.812324911" Sep 10 12:19:01.793081 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:01.793041 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dssrq" event={"ID":"3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2","Type":"ContainerStarted","Data":"8d0a0a03bb1e721a9bb967a98469472618ea65932b0d4d0588b9ed1c35af8f3a"} Sep 10 12:19:01.793534 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:01.793108 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dssrq" event={"ID":"3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2","Type":"ContainerStarted","Data":"e853e06d40e05bc82c4f197c4db58c4f9994793c2a777e9cbfa468d11ec8aa32"} Sep 10 12:19:01.817950 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:01.817900 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dssrq" podStartSLOduration=33.421658741 podStartE2EDuration="35.817885745s" totalImagesPullingTime="2.396227004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:01.81777813 +0000 UTC m=+69.822997590" watchObservedRunningTime="2026-09-10 12:19:01.817885745 +0000 UTC m=+69.823105204" Sep 10 12:19:02.689771 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:02.689726 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a6364599-ee05-41d5-a75c-c9fae216b75d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cq4k5\" (UID: \"a6364599-ee05-41d5-a75c-c9fae216b75d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cq4k5" Sep 10 12:19:02.692313 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:02.692287 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a6364599-ee05-41d5-a75c-c9fae216b75d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-cq4k5\" (UID: \"a6364599-ee05-41d5-a75c-c9fae216b75d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cq4k5" Sep 10 12:19:02.796815 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:02.796783 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dssrq" Sep 10 12:19:02.915294 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:02.915264 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-b5hvj\"" Sep 10 12:19:02.922532 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:02.922503 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cq4k5" Sep 10 12:19:02.973604 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:02.973533 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc6945cbb-9zpxm"] Sep 10 12:19:02.976576 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:02.976556 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc6945cbb-9zpxm" Sep 10 12:19:02.980509 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:02.980487 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 10 12:19:02.980678 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:02.980496 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 10 12:19:02.980857 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:02.980818 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 10 12:19:02.981176 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:02.981161 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 10 12:19:02.993947 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:02.993893 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc6945cbb-9zpxm"] Sep 10 12:19:03.058444 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.058414 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-bf6588865-dxmvn"] Sep 10 12:19:03.064237 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.064209 2583 patch_prober.go:28] interesting pod/image-registry-bf6588865-dxmvn container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 10 12:19:03.064371 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.064260 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-bf6588865-dxmvn" podUID="8d2f46f3-7193-4c2a-be48-f2458d2a0960" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 12:19:03.070699 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.070674 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-zscmp"] Sep 10 12:19:03.073994 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.073974 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-zscmp" Sep 10 12:19:03.074453 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.074340 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f5f7bdd8d-hx87h"] Sep 10 12:19:03.077399 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.077377 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f5f7bdd8d-hx87h" Sep 10 12:19:03.080188 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.080032 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 10 12:19:03.080485 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.080276 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 10 12:19:03.080485 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.080382 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 10 12:19:03.080485 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.080404 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 10 12:19:03.080821 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.080799 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-cl8sn\"" Sep 10 12:19:03.080931 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.080799 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 10 12:19:03.080931 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.080832 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 10 12:19:03.083811 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.083791 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-4rm6x\"" Sep 10 12:19:03.085615 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.085033 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 10 12:19:03.093378 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.093358 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wmh8\" (UniqueName: \"kubernetes.io/projected/0eb74df4-2791-4b34-9d46-a2af05e48691-kube-api-access-9wmh8\") pod \"klusterlet-addon-workmgr-6bc6945cbb-9zpxm\" (UID: \"0eb74df4-2791-4b34-9d46-a2af05e48691\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc6945cbb-9zpxm" Sep 10 12:19:03.093489 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.093401 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0eb74df4-2791-4b34-9d46-a2af05e48691-tmp\") pod \"klusterlet-addon-workmgr-6bc6945cbb-9zpxm\" (UID: \"0eb74df4-2791-4b34-9d46-a2af05e48691\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc6945cbb-9zpxm" Sep 10 12:19:03.093489 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.093437 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0eb74df4-2791-4b34-9d46-a2af05e48691-klusterlet-config\") pod \"klusterlet-addon-workmgr-6bc6945cbb-9zpxm\" (UID: \"0eb74df4-2791-4b34-9d46-a2af05e48691\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc6945cbb-9zpxm" Sep 10 12:19:03.095480 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.095456 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-zscmp"] Sep 10 12:19:03.110728 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.110700 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f5f7bdd8d-hx87h"] Sep 10 12:19:03.194326 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.194295 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt8wj\" (UniqueName: \"kubernetes.io/projected/02bc865a-2994-4d11-8f15-dc17ef64f9c4-kube-api-access-wt8wj\") pod \"console-6f5f7bdd8d-hx87h\" (UID: \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\") " pod="openshift-console/console-6f5f7bdd8d-hx87h" Sep 10 12:19:03.194498 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.194348 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0eb74df4-2791-4b34-9d46-a2af05e48691-tmp\") pod \"klusterlet-addon-workmgr-6bc6945cbb-9zpxm\" (UID: \"0eb74df4-2791-4b34-9d46-a2af05e48691\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc6945cbb-9zpxm" Sep 10 12:19:03.194498 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.194371 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0eb74df4-2791-4b34-9d46-a2af05e48691-klusterlet-config\") pod \"klusterlet-addon-workmgr-6bc6945cbb-9zpxm\" (UID: \"0eb74df4-2791-4b34-9d46-a2af05e48691\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc6945cbb-9zpxm" Sep 10 12:19:03.194498 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.194398 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/02bc865a-2994-4d11-8f15-dc17ef64f9c4-service-ca\") pod \"console-6f5f7bdd8d-hx87h\" (UID: \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\") " pod="openshift-console/console-6f5f7bdd8d-hx87h" Sep 10 12:19:03.194498 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.194451 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/02bc865a-2994-4d11-8f15-dc17ef64f9c4-oauth-serving-cert\") pod \"console-6f5f7bdd8d-hx87h\" (UID: \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\") " pod="openshift-console/console-6f5f7bdd8d-hx87h" Sep 10 12:19:03.194498 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.194478 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/02bc865a-2994-4d11-8f15-dc17ef64f9c4-console-oauth-config\") pod \"console-6f5f7bdd8d-hx87h\" (UID: \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\") " pod="openshift-console/console-6f5f7bdd8d-hx87h" Sep 10 12:19:03.194804 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.194658 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv4r6\" (UniqueName: \"kubernetes.io/projected/bfa6a4f9-3186-4067-961d-4b9130110942-kube-api-access-pv4r6\") pod \"downloads-85d98c998f-zscmp\" (UID: \"bfa6a4f9-3186-4067-961d-4b9130110942\") " pod="openshift-console/downloads-85d98c998f-zscmp" Sep 10 12:19:03.194804 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.194773 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/02bc865a-2994-4d11-8f15-dc17ef64f9c4-console-serving-cert\") pod \"console-6f5f7bdd8d-hx87h\" (UID: \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\") " pod="openshift-console/console-6f5f7bdd8d-hx87h" Sep 10 12:19:03.194804 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.194776 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0eb74df4-2791-4b34-9d46-a2af05e48691-tmp\") pod \"klusterlet-addon-workmgr-6bc6945cbb-9zpxm\" (UID: \"0eb74df4-2791-4b34-9d46-a2af05e48691\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc6945cbb-9zpxm" Sep 10 12:19:03.194964 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.194796 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/02bc865a-2994-4d11-8f15-dc17ef64f9c4-console-config\") pod \"console-6f5f7bdd8d-hx87h\" (UID: \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\") " pod="openshift-console/console-6f5f7bdd8d-hx87h" Sep 10 12:19:03.194964 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.194914 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wmh8\" (UniqueName: \"kubernetes.io/projected/0eb74df4-2791-4b34-9d46-a2af05e48691-kube-api-access-9wmh8\") pod \"klusterlet-addon-workmgr-6bc6945cbb-9zpxm\" (UID: \"0eb74df4-2791-4b34-9d46-a2af05e48691\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc6945cbb-9zpxm" Sep 10 12:19:03.197201 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.197178 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0eb74df4-2791-4b34-9d46-a2af05e48691-klusterlet-config\") pod \"klusterlet-addon-workmgr-6bc6945cbb-9zpxm\" (UID: \"0eb74df4-2791-4b34-9d46-a2af05e48691\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc6945cbb-9zpxm" Sep 10 12:19:03.220801 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.220775 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wmh8\" (UniqueName: \"kubernetes.io/projected/0eb74df4-2791-4b34-9d46-a2af05e48691-kube-api-access-9wmh8\") pod \"klusterlet-addon-workmgr-6bc6945cbb-9zpxm\" (UID: \"0eb74df4-2791-4b34-9d46-a2af05e48691\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc6945cbb-9zpxm" Sep 10 12:19:03.241428 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.241354 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-cbd876cd-gdc7j"] Sep 10 12:19:03.244766 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.244744 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.276111 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.276076 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-cbd876cd-gdc7j"] Sep 10 12:19:03.287596 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.287567 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc6945cbb-9zpxm" Sep 10 12:19:03.295271 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.295245 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c0707deb-a9da-4cb3-b67e-38ffbee97f42-registry-certificates\") pod \"image-registry-cbd876cd-gdc7j\" (UID: \"c0707deb-a9da-4cb3-b67e-38ffbee97f42\") " pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.295395 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.295281 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/02bc865a-2994-4d11-8f15-dc17ef64f9c4-service-ca\") pod \"console-6f5f7bdd8d-hx87h\" (UID: \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\") " pod="openshift-console/console-6f5f7bdd8d-hx87h" Sep 10 12:19:03.295395 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.295317 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c0707deb-a9da-4cb3-b67e-38ffbee97f42-image-registry-private-configuration\") pod \"image-registry-cbd876cd-gdc7j\" (UID: \"c0707deb-a9da-4cb3-b67e-38ffbee97f42\") " pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.295395 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.295382 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c0707deb-a9da-4cb3-b67e-38ffbee97f42-trusted-ca\") pod \"image-registry-cbd876cd-gdc7j\" (UID: \"c0707deb-a9da-4cb3-b67e-38ffbee97f42\") " pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.295525 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.295441 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c0707deb-a9da-4cb3-b67e-38ffbee97f42-registry-tls\") pod \"image-registry-cbd876cd-gdc7j\" (UID: \"c0707deb-a9da-4cb3-b67e-38ffbee97f42\") " pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.295525 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.295465 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/02bc865a-2994-4d11-8f15-dc17ef64f9c4-oauth-serving-cert\") pod \"console-6f5f7bdd8d-hx87h\" (UID: \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\") " pod="openshift-console/console-6f5f7bdd8d-hx87h" Sep 10 12:19:03.295525 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.295487 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c0707deb-a9da-4cb3-b67e-38ffbee97f42-installation-pull-secrets\") pod \"image-registry-cbd876cd-gdc7j\" (UID: \"c0707deb-a9da-4cb3-b67e-38ffbee97f42\") " pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.295674 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.295535 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/02bc865a-2994-4d11-8f15-dc17ef64f9c4-console-oauth-config\") pod \"console-6f5f7bdd8d-hx87h\" (UID: \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\") " pod="openshift-console/console-6f5f7bdd8d-hx87h" Sep 10 12:19:03.295674 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.295569 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv4r6\" (UniqueName: \"kubernetes.io/projected/bfa6a4f9-3186-4067-961d-4b9130110942-kube-api-access-pv4r6\") pod \"downloads-85d98c998f-zscmp\" (UID: \"bfa6a4f9-3186-4067-961d-4b9130110942\") " pod="openshift-console/downloads-85d98c998f-zscmp" Sep 10 12:19:03.295674 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.295601 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/02bc865a-2994-4d11-8f15-dc17ef64f9c4-console-serving-cert\") pod \"console-6f5f7bdd8d-hx87h\" (UID: \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\") " pod="openshift-console/console-6f5f7bdd8d-hx87h" Sep 10 12:19:03.295674 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.295624 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/02bc865a-2994-4d11-8f15-dc17ef64f9c4-console-config\") pod \"console-6f5f7bdd8d-hx87h\" (UID: \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\") " pod="openshift-console/console-6f5f7bdd8d-hx87h" Sep 10 12:19:03.295674 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.295650 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4k6r\" (UniqueName: \"kubernetes.io/projected/c0707deb-a9da-4cb3-b67e-38ffbee97f42-kube-api-access-q4k6r\") pod \"image-registry-cbd876cd-gdc7j\" (UID: \"c0707deb-a9da-4cb3-b67e-38ffbee97f42\") " pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.295943 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.295686 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c0707deb-a9da-4cb3-b67e-38ffbee97f42-ca-trust-extracted\") pod \"image-registry-cbd876cd-gdc7j\" (UID: \"c0707deb-a9da-4cb3-b67e-38ffbee97f42\") " pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.295943 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.295915 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt8wj\" (UniqueName: \"kubernetes.io/projected/02bc865a-2994-4d11-8f15-dc17ef64f9c4-kube-api-access-wt8wj\") pod \"console-6f5f7bdd8d-hx87h\" (UID: \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\") " pod="openshift-console/console-6f5f7bdd8d-hx87h" Sep 10 12:19:03.296036 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.295955 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c0707deb-a9da-4cb3-b67e-38ffbee97f42-bound-sa-token\") pod \"image-registry-cbd876cd-gdc7j\" (UID: \"c0707deb-a9da-4cb3-b67e-38ffbee97f42\") " pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.296036 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.296009 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/02bc865a-2994-4d11-8f15-dc17ef64f9c4-service-ca\") pod \"console-6f5f7bdd8d-hx87h\" (UID: \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\") " pod="openshift-console/console-6f5f7bdd8d-hx87h" Sep 10 12:19:03.296196 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.296175 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/02bc865a-2994-4d11-8f15-dc17ef64f9c4-oauth-serving-cert\") pod \"console-6f5f7bdd8d-hx87h\" (UID: \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\") " pod="openshift-console/console-6f5f7bdd8d-hx87h" Sep 10 12:19:03.296330 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.296284 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/02bc865a-2994-4d11-8f15-dc17ef64f9c4-console-config\") pod \"console-6f5f7bdd8d-hx87h\" (UID: \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\") " pod="openshift-console/console-6f5f7bdd8d-hx87h" Sep 10 12:19:03.298567 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.298546 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/02bc865a-2994-4d11-8f15-dc17ef64f9c4-console-oauth-config\") pod \"console-6f5f7bdd8d-hx87h\" (UID: \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\") " pod="openshift-console/console-6f5f7bdd8d-hx87h" Sep 10 12:19:03.298667 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.298596 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/02bc865a-2994-4d11-8f15-dc17ef64f9c4-console-serving-cert\") pod \"console-6f5f7bdd8d-hx87h\" (UID: \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\") " pod="openshift-console/console-6f5f7bdd8d-hx87h" Sep 10 12:19:03.318195 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.318175 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt8wj\" (UniqueName: \"kubernetes.io/projected/02bc865a-2994-4d11-8f15-dc17ef64f9c4-kube-api-access-wt8wj\") pod \"console-6f5f7bdd8d-hx87h\" (UID: \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\") " pod="openshift-console/console-6f5f7bdd8d-hx87h" Sep 10 12:19:03.333645 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.333619 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv4r6\" (UniqueName: \"kubernetes.io/projected/bfa6a4f9-3186-4067-961d-4b9130110942-kube-api-access-pv4r6\") pod \"downloads-85d98c998f-zscmp\" (UID: \"bfa6a4f9-3186-4067-961d-4b9130110942\") " pod="openshift-console/downloads-85d98c998f-zscmp" Sep 10 12:19:03.385891 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.385868 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-zscmp" Sep 10 12:19:03.391896 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.391876 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f5f7bdd8d-hx87h" Sep 10 12:19:03.396767 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.396741 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c0707deb-a9da-4cb3-b67e-38ffbee97f42-ca-trust-extracted\") pod \"image-registry-cbd876cd-gdc7j\" (UID: \"c0707deb-a9da-4cb3-b67e-38ffbee97f42\") " pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.396889 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.396781 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c0707deb-a9da-4cb3-b67e-38ffbee97f42-bound-sa-token\") pod \"image-registry-cbd876cd-gdc7j\" (UID: \"c0707deb-a9da-4cb3-b67e-38ffbee97f42\") " pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.396889 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.396814 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c0707deb-a9da-4cb3-b67e-38ffbee97f42-registry-certificates\") pod \"image-registry-cbd876cd-gdc7j\" (UID: \"c0707deb-a9da-4cb3-b67e-38ffbee97f42\") " pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.396889 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.396863 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c0707deb-a9da-4cb3-b67e-38ffbee97f42-image-registry-private-configuration\") pod \"image-registry-cbd876cd-gdc7j\" (UID: \"c0707deb-a9da-4cb3-b67e-38ffbee97f42\") " pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.397062 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.396895 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c0707deb-a9da-4cb3-b67e-38ffbee97f42-trusted-ca\") pod \"image-registry-cbd876cd-gdc7j\" (UID: \"c0707deb-a9da-4cb3-b67e-38ffbee97f42\") " pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.397062 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.396919 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c0707deb-a9da-4cb3-b67e-38ffbee97f42-registry-tls\") pod \"image-registry-cbd876cd-gdc7j\" (UID: \"c0707deb-a9da-4cb3-b67e-38ffbee97f42\") " pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.397062 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.396944 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c0707deb-a9da-4cb3-b67e-38ffbee97f42-installation-pull-secrets\") pod \"image-registry-cbd876cd-gdc7j\" (UID: \"c0707deb-a9da-4cb3-b67e-38ffbee97f42\") " pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.397062 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.396970 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4k6r\" (UniqueName: \"kubernetes.io/projected/c0707deb-a9da-4cb3-b67e-38ffbee97f42-kube-api-access-q4k6r\") pod \"image-registry-cbd876cd-gdc7j\" (UID: \"c0707deb-a9da-4cb3-b67e-38ffbee97f42\") " pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.397254 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.397121 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c0707deb-a9da-4cb3-b67e-38ffbee97f42-ca-trust-extracted\") pod \"image-registry-cbd876cd-gdc7j\" (UID: \"c0707deb-a9da-4cb3-b67e-38ffbee97f42\") " pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.397694 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.397670 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c0707deb-a9da-4cb3-b67e-38ffbee97f42-registry-certificates\") pod \"image-registry-cbd876cd-gdc7j\" (UID: \"c0707deb-a9da-4cb3-b67e-38ffbee97f42\") " pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.397944 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.397916 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c0707deb-a9da-4cb3-b67e-38ffbee97f42-trusted-ca\") pod \"image-registry-cbd876cd-gdc7j\" (UID: \"c0707deb-a9da-4cb3-b67e-38ffbee97f42\") " pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.399401 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.399370 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c0707deb-a9da-4cb3-b67e-38ffbee97f42-image-registry-private-configuration\") pod \"image-registry-cbd876cd-gdc7j\" (UID: \"c0707deb-a9da-4cb3-b67e-38ffbee97f42\") " pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.399757 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.399726 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c0707deb-a9da-4cb3-b67e-38ffbee97f42-registry-tls\") pod \"image-registry-cbd876cd-gdc7j\" (UID: \"c0707deb-a9da-4cb3-b67e-38ffbee97f42\") " pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.399830 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.399811 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c0707deb-a9da-4cb3-b67e-38ffbee97f42-installation-pull-secrets\") pod \"image-registry-cbd876cd-gdc7j\" (UID: \"c0707deb-a9da-4cb3-b67e-38ffbee97f42\") " pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.409613 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.409589 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4k6r\" (UniqueName: \"kubernetes.io/projected/c0707deb-a9da-4cb3-b67e-38ffbee97f42-kube-api-access-q4k6r\") pod \"image-registry-cbd876cd-gdc7j\" (UID: \"c0707deb-a9da-4cb3-b67e-38ffbee97f42\") " pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.409613 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.409606 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c0707deb-a9da-4cb3-b67e-38ffbee97f42-bound-sa-token\") pod \"image-registry-cbd876cd-gdc7j\" (UID: \"c0707deb-a9da-4cb3-b67e-38ffbee97f42\") " pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:03.556427 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:03.556015 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:04.405211 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:04.405184 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-zscmp"] Sep 10 12:19:04.414811 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:04.414751 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f5f7bdd8d-hx87h"] Sep 10 12:19:04.424670 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:04.424635 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-cbd876cd-gdc7j"] Sep 10 12:19:04.428513 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:19:04.428482 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0707deb_a9da_4cb3_b67e_38ffbee97f42.slice/crio-f07917d7707ae1a73e5006727aa73bf2196a20482413e4037ca83cdbd8f4e8cc WatchSource:0}: Error finding container f07917d7707ae1a73e5006727aa73bf2196a20482413e4037ca83cdbd8f4e8cc: Status 404 returned error can't find the container with id f07917d7707ae1a73e5006727aa73bf2196a20482413e4037ca83cdbd8f4e8cc Sep 10 12:19:04.438654 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:04.438629 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc6945cbb-9zpxm"] Sep 10 12:19:04.442660 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:19:04.441972 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eb74df4_2791_4b34_9d46_a2af05e48691.slice/crio-36cc48ceb73cde379bde6c19ab2bacefdddaef4f8dd344d78a88e05058c0fd97 WatchSource:0}: Error finding container 36cc48ceb73cde379bde6c19ab2bacefdddaef4f8dd344d78a88e05058c0fd97: Status 404 returned error can't find the container with id 36cc48ceb73cde379bde6c19ab2bacefdddaef4f8dd344d78a88e05058c0fd97 Sep 10 12:19:04.442966 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:04.442942 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cq4k5"] Sep 10 12:19:04.446678 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:19:04.446653 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6364599_ee05_41d5_a75c_c9fae216b75d.slice/crio-f1f93415c8e68e95047d89a68984c417b6a3545a6d963e409a4ae147bac05af2 WatchSource:0}: Error finding container f1f93415c8e68e95047d89a68984c417b6a3545a6d963e409a4ae147bac05af2: Status 404 returned error can't find the container with id f1f93415c8e68e95047d89a68984c417b6a3545a6d963e409a4ae147bac05af2 Sep 10 12:19:04.805151 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:04.805112 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k9bgv" event={"ID":"1aa57da6-29e7-4701-8b1b-be81cf4ebed6","Type":"ContainerStarted","Data":"6903e1c3d44fe5cf5de39fd851ec6bf638a7bab18f9894c7f4c5ecefd22cc7a5"} Sep 10 12:19:04.805327 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:04.805177 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:19:04.806659 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:04.806633 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dgddq" event={"ID":"6960f97e-f623-4368-b27a-41d81ac1be1f","Type":"ContainerStarted","Data":"58cd5165388949d9672585caf770c1f62053f053c839b8b2dcfec88798fe0744"} Sep 10 12:19:04.807834 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:04.807811 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc6945cbb-9zpxm" event={"ID":"0eb74df4-2791-4b34-9d46-a2af05e48691","Type":"ContainerStarted","Data":"36cc48ceb73cde379bde6c19ab2bacefdddaef4f8dd344d78a88e05058c0fd97"} Sep 10 12:19:04.809140 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:04.809119 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cq4k5" event={"ID":"a6364599-ee05-41d5-a75c-c9fae216b75d","Type":"ContainerStarted","Data":"f1f93415c8e68e95047d89a68984c417b6a3545a6d963e409a4ae147bac05af2"} Sep 10 12:19:04.810298 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:04.810271 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f5f7bdd8d-hx87h" event={"ID":"02bc865a-2994-4d11-8f15-dc17ef64f9c4","Type":"ContainerStarted","Data":"ddaee843be42a9a411a7dcac7a88dbde33e1f8a98cd64d02a2ba611687fc032e"} Sep 10 12:19:04.811470 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:04.811446 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-zscmp" event={"ID":"bfa6a4f9-3186-4067-961d-4b9130110942","Type":"ContainerStarted","Data":"eec6d001d45f6dfce5fae9fb7615c97164fcfe5c87361ad4eb1bda9654be9b2b"} Sep 10 12:19:04.812937 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:04.812917 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" event={"ID":"c0707deb-a9da-4cb3-b67e-38ffbee97f42","Type":"ContainerStarted","Data":"412950394e1a9f06f415ff37edc101172bed255a9ab2dd3ca28ed1aa52863bbd"} Sep 10 12:19:04.813006 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:04.812954 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" event={"ID":"c0707deb-a9da-4cb3-b67e-38ffbee97f42","Type":"ContainerStarted","Data":"f07917d7707ae1a73e5006727aa73bf2196a20482413e4037ca83cdbd8f4e8cc"} Sep 10 12:19:04.813006 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:04.812989 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:04.822806 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:04.822773 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-k9bgv" podStartSLOduration=67.609041899 podStartE2EDuration="1m12.82276139s" totalImagesPullingTime="5.213719491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:17:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:04.82196273 +0000 UTC m=+72.827182190" watchObservedRunningTime="2026-09-10 12:19:04.82276139 +0000 UTC m=+72.827980847" Sep 10 12:19:04.842538 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:04.842489 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" podStartSLOduration=1.842471244 podStartE2EDuration="1.842471244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:19:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:04.842173939 +0000 UTC m=+72.847393400" watchObservedRunningTime="2026-09-10 12:19:04.842471244 +0000 UTC m=+72.847690696" Sep 10 12:19:04.860233 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:04.860188 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dgddq" podStartSLOduration=67.591336586 podStartE2EDuration="1m12.860172707s" totalImagesPullingTime="5.268836121s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:17:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:04.859615768 +0000 UTC m=+72.864835229" watchObservedRunningTime="2026-09-10 12:19:04.860172707 +0000 UTC m=+72.865392160" Sep 10 12:19:10.846608 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:10.845487 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc6945cbb-9zpxm" event={"ID":"0eb74df4-2791-4b34-9d46-a2af05e48691","Type":"ContainerStarted","Data":"c78063c8b01e007d4bcbcfe2f94d25e7c706ae49ed1858acd783ffa1276af96a"} Sep 10 12:19:10.846608 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:10.846504 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc6945cbb-9zpxm" Sep 10 12:19:10.848436 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:10.848411 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc6945cbb-9zpxm" Sep 10 12:19:10.849042 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:10.849017 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cq4k5" event={"ID":"a6364599-ee05-41d5-a75c-c9fae216b75d","Type":"ContainerStarted","Data":"9ce3a8e2a6c5470f09fbb5fe3138d2b8480035e76628098ca914fcb3cb36bb18"} Sep 10 12:19:10.850825 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:10.850804 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f5f7bdd8d-hx87h" event={"ID":"02bc865a-2994-4d11-8f15-dc17ef64f9c4","Type":"ContainerStarted","Data":"86d0e5916a41b2c54fd6e551a4c6c79f35cf59386838caba2f332a80f2fb8639"} Sep 10 12:19:10.866678 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:10.866626 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6bc6945cbb-9zpxm" podStartSLOduration=2.717955204 podStartE2EDuration="8.866607421s" totalImagesPullingTime="6.148652217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:19:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:10.864636905 +0000 UTC m=+78.869856366" watchObservedRunningTime="2026-09-10 12:19:10.866607421 +0000 UTC m=+78.871826883" Sep 10 12:19:10.908408 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:10.908364 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-cq4k5" podStartSLOduration=34.789882003 podStartE2EDuration="40.908350696s" totalImagesPullingTime="6.118468693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:18:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:10.907982613 +0000 UTC m=+78.913202074" watchObservedRunningTime="2026-09-10 12:19:10.908350696 +0000 UTC m=+78.913570155" Sep 10 12:19:10.934454 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:10.934316 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f5f7bdd8d-hx87h" podStartSLOduration=1.78039653 podStartE2EDuration="7.934299863s" totalImagesPullingTime="6.153903333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:19:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:10.933971471 +0000 UTC m=+78.939190930" watchObservedRunningTime="2026-09-10 12:19:10.934299863 +0000 UTC m=+78.939519325" Sep 10 12:19:12.801924 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:12.801894 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dssrq" Sep 10 12:19:13.063254 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:13.063175 2583 patch_prober.go:28] interesting pod/image-registry-bf6588865-dxmvn container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 10 12:19:13.063254 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:13.063226 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-bf6588865-dxmvn" podUID="8d2f46f3-7193-4c2a-be48-f2458d2a0960" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 12:19:13.392793 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:13.392720 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f5f7bdd8d-hx87h" Sep 10 12:19:13.392793 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:13.392779 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f5f7bdd8d-hx87h" Sep 10 12:19:13.394363 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:13.394336 2583 patch_prober.go:28] interesting pod/console-6f5f7bdd8d-hx87h container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.17:8443/health\": dial tcp 10.133.0.17:8443: connect: connection refused" start-of-body= Sep 10 12:19:13.394489 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:13.394378 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-6f5f7bdd8d-hx87h" podUID="02bc865a-2994-4d11-8f15-dc17ef64f9c4" containerName="console" probeResult="failure" output="Get \"https://10.133.0.17:8443/health\": dial tcp 10.133.0.17:8443: connect: connection refused" Sep 10 12:19:21.684834 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.684672 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-c7rdt"] Sep 10 12:19:21.746768 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.746730 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.751115 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.751089 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 10 12:19:21.751288 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.751091 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 10 12:19:21.751800 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.751767 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xpp5p\"" Sep 10 12:19:21.752626 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.752597 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 10 12:19:21.754889 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.754868 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 10 12:19:21.855661 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.855628 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8c893810-45a6-4fe5-a004-32668d671629-node-exporter-textfile\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.855822 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.855691 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z64h4\" (UniqueName: \"kubernetes.io/projected/8c893810-45a6-4fe5-a004-32668d671629-kube-api-access-z64h4\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.855822 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.855725 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8c893810-45a6-4fe5-a004-32668d671629-root\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.855822 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.855764 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8c893810-45a6-4fe5-a004-32668d671629-node-exporter-tls\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.855971 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.855872 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8c893810-45a6-4fe5-a004-32668d671629-node-exporter-wtmp\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.855971 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.855913 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c893810-45a6-4fe5-a004-32668d671629-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.855971 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.855938 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c893810-45a6-4fe5-a004-32668d671629-metrics-client-ca\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.856115 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.855968 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8c893810-45a6-4fe5-a004-32668d671629-node-exporter-accelerators-collector-config\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.856115 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.856056 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c893810-45a6-4fe5-a004-32668d671629-sys\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.957005 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.956929 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z64h4\" (UniqueName: \"kubernetes.io/projected/8c893810-45a6-4fe5-a004-32668d671629-kube-api-access-z64h4\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.957005 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.956969 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8c893810-45a6-4fe5-a004-32668d671629-root\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.957005 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.957000 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8c893810-45a6-4fe5-a004-32668d671629-node-exporter-tls\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.957263 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.957033 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8c893810-45a6-4fe5-a004-32668d671629-node-exporter-wtmp\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.957263 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.957061 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c893810-45a6-4fe5-a004-32668d671629-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.957263 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.957081 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8c893810-45a6-4fe5-a004-32668d671629-root\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.957263 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.957089 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c893810-45a6-4fe5-a004-32668d671629-metrics-client-ca\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.957263 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.957188 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8c893810-45a6-4fe5-a004-32668d671629-node-exporter-accelerators-collector-config\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.957263 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.957210 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8c893810-45a6-4fe5-a004-32668d671629-node-exporter-wtmp\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.957263 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.957241 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c893810-45a6-4fe5-a004-32668d671629-sys\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.957588 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.957279 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8c893810-45a6-4fe5-a004-32668d671629-node-exporter-textfile\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.957588 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.957390 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c893810-45a6-4fe5-a004-32668d671629-sys\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.957588 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.957541 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8c893810-45a6-4fe5-a004-32668d671629-node-exporter-textfile\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.957730 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.957703 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c893810-45a6-4fe5-a004-32668d671629-metrics-client-ca\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.958058 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.958037 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8c893810-45a6-4fe5-a004-32668d671629-node-exporter-accelerators-collector-config\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.960058 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.960038 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8c893810-45a6-4fe5-a004-32668d671629-node-exporter-tls\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.960156 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.960109 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8c893810-45a6-4fe5-a004-32668d671629-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:21.973640 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:21.973616 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z64h4\" (UniqueName: \"kubernetes.io/projected/8c893810-45a6-4fe5-a004-32668d671629-kube-api-access-z64h4\") pod \"node-exporter-c7rdt\" (UID: \"8c893810-45a6-4fe5-a004-32668d671629\") " pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:22.036469 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:22.036433 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f5f7bdd8d-hx87h"] Sep 10 12:19:22.059316 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:22.059290 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c7rdt" Sep 10 12:19:22.539711 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:19:22.539684 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c893810_45a6_4fe5_a004_32668d671629.slice/crio-d0a93d99c4608c30b181f204dc8a509fa93cef15f1624a1fb5bfb1fe221e75ac WatchSource:0}: Error finding container d0a93d99c4608c30b181f204dc8a509fa93cef15f1624a1fb5bfb1fe221e75ac: Status 404 returned error can't find the container with id d0a93d99c4608c30b181f204dc8a509fa93cef15f1624a1fb5bfb1fe221e75ac Sep 10 12:19:22.896377 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:22.896342 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c7rdt" event={"ID":"8c893810-45a6-4fe5-a004-32668d671629","Type":"ContainerStarted","Data":"d0a93d99c4608c30b181f204dc8a509fa93cef15f1624a1fb5bfb1fe221e75ac"} Sep 10 12:19:23.064787 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:23.064751 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:19:23.902992 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:23.902896 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-zscmp" event={"ID":"bfa6a4f9-3186-4067-961d-4b9130110942","Type":"ContainerStarted","Data":"f66c4b31a0a82a9c1628e1420b11052b80a3a8cb04bd5cff061d0b6768c42fb5"} Sep 10 12:19:23.902992 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:23.902971 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-zscmp" Sep 10 12:19:23.905889 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:23.905458 2583 generic.go:356] "Generic (PLEG): container finished" podID="8c893810-45a6-4fe5-a004-32668d671629" containerID="03b2c8d035c7db035acd0c909eaee557379de0bd572b9dd0a885db38f8d471b9" exitCode=0 Sep 10 12:19:23.906033 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:23.905964 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c7rdt" event={"ID":"8c893810-45a6-4fe5-a004-32668d671629","Type":"ContainerDied","Data":"03b2c8d035c7db035acd0c909eaee557379de0bd572b9dd0a885db38f8d471b9"} Sep 10 12:19:23.909713 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:23.909690 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-zscmp" Sep 10 12:19:23.923666 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:23.923628 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-zscmp" podStartSLOduration=2.39830589 podStartE2EDuration="20.923612732s" totalImagesPullingTime="18.525306842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:19:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:23.922690828 +0000 UTC m=+91.927910287" watchObservedRunningTime="2026-09-10 12:19:23.923612732 +0000 UTC m=+91.928832317" Sep 10 12:19:24.914860 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:24.914810 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c7rdt" event={"ID":"8c893810-45a6-4fe5-a004-32668d671629","Type":"ContainerStarted","Data":"66aee25c203be47e37e4716a0a6971f9e9a2784bb83e7c473a71c4d9d1d2447c"} Sep 10 12:19:24.915280 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:24.914888 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c7rdt" event={"ID":"8c893810-45a6-4fe5-a004-32668d671629","Type":"ContainerStarted","Data":"805d1c3f08144a16385c924538d126d07ac3e377e5deb3e4862f142ac85ac9d2"} Sep 10 12:19:24.935652 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:24.935593 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-c7rdt" podStartSLOduration=2.820161612 podStartE2EDuration="3.935572061s" totalImagesPullingTime="1.042206795s" totalInitContainerRuntime="73.203654ms" isStatefulPod=true podCreationTimestamp="2026-09-10 12:19:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:24.934464462 +0000 UTC m=+92.939683926" watchObservedRunningTime="2026-09-10 12:19:24.935572061 +0000 UTC m=+92.940791524" Sep 10 12:19:25.825718 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:25.825683 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-cbd876cd-gdc7j" Sep 10 12:19:26.412316 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.412282 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cbff746c-k9wrl"] Sep 10 12:19:26.441146 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.441116 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:26.449305 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.449275 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cbff746c-k9wrl"] Sep 10 12:19:26.452406 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.452230 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 10 12:19:26.598054 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.598016 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84dcab2c-9a2a-496d-9dec-fd2916c137d3-trusted-ca-bundle\") pod \"console-7cbff746c-k9wrl\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:26.598054 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.598065 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84dcab2c-9a2a-496d-9dec-fd2916c137d3-console-oauth-config\") pod \"console-7cbff746c-k9wrl\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:26.598292 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.598085 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cmsc\" (UniqueName: \"kubernetes.io/projected/84dcab2c-9a2a-496d-9dec-fd2916c137d3-kube-api-access-9cmsc\") pod \"console-7cbff746c-k9wrl\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:26.598292 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.598209 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84dcab2c-9a2a-496d-9dec-fd2916c137d3-service-ca\") pod \"console-7cbff746c-k9wrl\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:26.598292 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.598245 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84dcab2c-9a2a-496d-9dec-fd2916c137d3-console-serving-cert\") pod \"console-7cbff746c-k9wrl\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:26.598292 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.598284 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84dcab2c-9a2a-496d-9dec-fd2916c137d3-oauth-serving-cert\") pod \"console-7cbff746c-k9wrl\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:26.598483 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.598312 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84dcab2c-9a2a-496d-9dec-fd2916c137d3-console-config\") pod \"console-7cbff746c-k9wrl\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:26.699727 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.699642 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84dcab2c-9a2a-496d-9dec-fd2916c137d3-trusted-ca-bundle\") pod \"console-7cbff746c-k9wrl\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:26.699727 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.699703 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84dcab2c-9a2a-496d-9dec-fd2916c137d3-console-oauth-config\") pod \"console-7cbff746c-k9wrl\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:26.700001 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.699735 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cmsc\" (UniqueName: \"kubernetes.io/projected/84dcab2c-9a2a-496d-9dec-fd2916c137d3-kube-api-access-9cmsc\") pod \"console-7cbff746c-k9wrl\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:26.700001 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.699771 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84dcab2c-9a2a-496d-9dec-fd2916c137d3-service-ca\") pod \"console-7cbff746c-k9wrl\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:26.700001 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.699805 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84dcab2c-9a2a-496d-9dec-fd2916c137d3-console-serving-cert\") pod \"console-7cbff746c-k9wrl\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:26.700001 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.699861 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84dcab2c-9a2a-496d-9dec-fd2916c137d3-oauth-serving-cert\") pod \"console-7cbff746c-k9wrl\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:26.700001 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.699891 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84dcab2c-9a2a-496d-9dec-fd2916c137d3-console-config\") pod \"console-7cbff746c-k9wrl\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:26.700709 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.700655 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84dcab2c-9a2a-496d-9dec-fd2916c137d3-console-config\") pod \"console-7cbff746c-k9wrl\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:26.700860 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.700785 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84dcab2c-9a2a-496d-9dec-fd2916c137d3-service-ca\") pod \"console-7cbff746c-k9wrl\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:26.701293 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.701264 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84dcab2c-9a2a-496d-9dec-fd2916c137d3-oauth-serving-cert\") pod \"console-7cbff746c-k9wrl\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:26.701764 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.701745 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84dcab2c-9a2a-496d-9dec-fd2916c137d3-trusted-ca-bundle\") pod \"console-7cbff746c-k9wrl\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:26.702917 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.702890 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84dcab2c-9a2a-496d-9dec-fd2916c137d3-console-oauth-config\") pod \"console-7cbff746c-k9wrl\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:26.703117 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.703088 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84dcab2c-9a2a-496d-9dec-fd2916c137d3-console-serving-cert\") pod \"console-7cbff746c-k9wrl\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:26.709647 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.709619 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cmsc\" (UniqueName: \"kubernetes.io/projected/84dcab2c-9a2a-496d-9dec-fd2916c137d3-kube-api-access-9cmsc\") pod \"console-7cbff746c-k9wrl\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:26.759633 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:26.759606 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:27.072468 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:27.072279 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cbff746c-k9wrl"] Sep 10 12:19:27.074972 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:19:27.074943 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84dcab2c_9a2a_496d_9dec_fd2916c137d3.slice/crio-50ebd57a17eee1624b522decd56e728f86e34712a7043724100774b94a06af4f WatchSource:0}: Error finding container 50ebd57a17eee1624b522decd56e728f86e34712a7043724100774b94a06af4f: Status 404 returned error can't find the container with id 50ebd57a17eee1624b522decd56e728f86e34712a7043724100774b94a06af4f Sep 10 12:19:27.931733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:27.931691 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cbff746c-k9wrl" event={"ID":"84dcab2c-9a2a-496d-9dec-fd2916c137d3","Type":"ContainerStarted","Data":"a569853047a140926e95c3b20a385fbc83ae4649b78b1343099b4d985b05851c"} Sep 10 12:19:27.932288 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:27.931753 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cbff746c-k9wrl" event={"ID":"84dcab2c-9a2a-496d-9dec-fd2916c137d3","Type":"ContainerStarted","Data":"50ebd57a17eee1624b522decd56e728f86e34712a7043724100774b94a06af4f"} Sep 10 12:19:27.957430 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:27.957384 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cbff746c-k9wrl" podStartSLOduration=1.957365496 podStartE2EDuration="1.957365496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:19:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:19:27.957162064 +0000 UTC m=+95.962381525" watchObservedRunningTime="2026-09-10 12:19:27.957365496 +0000 UTC m=+95.962584958" Sep 10 12:19:28.083083 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.083011 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-bf6588865-dxmvn" podUID="8d2f46f3-7193-4c2a-be48-f2458d2a0960" containerName="registry" containerID="cri-o://371fe2547bf91ab216b903a908b2e01f1f1e76f7f2eed511a0173ecaafa67a18" gracePeriod=30 Sep 10 12:19:28.379821 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.379787 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:19:28.515291 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.515220 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-tls\") pod \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " Sep 10 12:19:28.515291 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.515271 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d2f46f3-7193-4c2a-be48-f2458d2a0960-trusted-ca\") pod \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " Sep 10 12:19:28.515505 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.515309 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d2f46f3-7193-4c2a-be48-f2458d2a0960-installation-pull-secrets\") pod \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " Sep 10 12:19:28.515505 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.515343 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d2f46f3-7193-4c2a-be48-f2458d2a0960-image-registry-private-configuration\") pod \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " Sep 10 12:19:28.515505 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.515377 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwwd9\" (UniqueName: \"kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-kube-api-access-hwwd9\") pod \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " Sep 10 12:19:28.515651 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.515511 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d2f46f3-7193-4c2a-be48-f2458d2a0960-ca-trust-extracted\") pod \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " Sep 10 12:19:28.515651 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.515561 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-certificates\") pod \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " Sep 10 12:19:28.515651 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.515619 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-bound-sa-token\") pod \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\" (UID: \"8d2f46f3-7193-4c2a-be48-f2458d2a0960\") " Sep 10 12:19:28.515815 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.515739 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d2f46f3-7193-4c2a-be48-f2458d2a0960-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8d2f46f3-7193-4c2a-be48-f2458d2a0960" (UID: "8d2f46f3-7193-4c2a-be48-f2458d2a0960"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:19:28.515988 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.515964 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d2f46f3-7193-4c2a-be48-f2458d2a0960-trusted-ca\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:19:28.516329 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.516303 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8d2f46f3-7193-4c2a-be48-f2458d2a0960" (UID: "8d2f46f3-7193-4c2a-be48-f2458d2a0960"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:19:28.518868 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.518121 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d2f46f3-7193-4c2a-be48-f2458d2a0960-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "8d2f46f3-7193-4c2a-be48-f2458d2a0960" (UID: "8d2f46f3-7193-4c2a-be48-f2458d2a0960"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:19:28.518868 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.518148 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-kube-api-access-hwwd9" (OuterVolumeSpecName: "kube-api-access-hwwd9") pod "8d2f46f3-7193-4c2a-be48-f2458d2a0960" (UID: "8d2f46f3-7193-4c2a-be48-f2458d2a0960"). InnerVolumeSpecName "kube-api-access-hwwd9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:19:28.522562 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.522481 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d2f46f3-7193-4c2a-be48-f2458d2a0960-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8d2f46f3-7193-4c2a-be48-f2458d2a0960" (UID: "8d2f46f3-7193-4c2a-be48-f2458d2a0960"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:19:28.522562 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.522513 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8d2f46f3-7193-4c2a-be48-f2458d2a0960" (UID: "8d2f46f3-7193-4c2a-be48-f2458d2a0960"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:19:28.522886 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.522602 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8d2f46f3-7193-4c2a-be48-f2458d2a0960" (UID: "8d2f46f3-7193-4c2a-be48-f2458d2a0960"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:19:28.528303 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.528253 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d2f46f3-7193-4c2a-be48-f2458d2a0960-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8d2f46f3-7193-4c2a-be48-f2458d2a0960" (UID: "8d2f46f3-7193-4c2a-be48-f2458d2a0960"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:19:28.617029 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.616996 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwwd9\" (UniqueName: \"kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-kube-api-access-hwwd9\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:19:28.617029 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.617026 2583 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d2f46f3-7193-4c2a-be48-f2458d2a0960-ca-trust-extracted\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:19:28.617202 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.617040 2583 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-certificates\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:19:28.617202 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.617052 2583 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-bound-sa-token\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:19:28.617202 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.617065 2583 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d2f46f3-7193-4c2a-be48-f2458d2a0960-registry-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:19:28.617202 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.617075 2583 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d2f46f3-7193-4c2a-be48-f2458d2a0960-installation-pull-secrets\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:19:28.617202 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.617089 2583 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d2f46f3-7193-4c2a-be48-f2458d2a0960-image-registry-private-configuration\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:19:28.937707 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.937674 2583 generic.go:356] "Generic (PLEG): container finished" podID="8d2f46f3-7193-4c2a-be48-f2458d2a0960" containerID="371fe2547bf91ab216b903a908b2e01f1f1e76f7f2eed511a0173ecaafa67a18" exitCode=0 Sep 10 12:19:28.938197 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.937727 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bf6588865-dxmvn" Sep 10 12:19:28.938197 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.937767 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bf6588865-dxmvn" event={"ID":"8d2f46f3-7193-4c2a-be48-f2458d2a0960","Type":"ContainerDied","Data":"371fe2547bf91ab216b903a908b2e01f1f1e76f7f2eed511a0173ecaafa67a18"} Sep 10 12:19:28.938197 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.937832 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bf6588865-dxmvn" event={"ID":"8d2f46f3-7193-4c2a-be48-f2458d2a0960","Type":"ContainerDied","Data":"0eaf70ddad9d154080b5aec06ba9f4fe6603da8f43c2e09bc918df568d763d07"} Sep 10 12:19:28.938197 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.937876 2583 scope.go:117] "RemoveContainer" containerID="371fe2547bf91ab216b903a908b2e01f1f1e76f7f2eed511a0173ecaafa67a18" Sep 10 12:19:28.948398 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.948381 2583 scope.go:117] "RemoveContainer" containerID="371fe2547bf91ab216b903a908b2e01f1f1e76f7f2eed511a0173ecaafa67a18" Sep 10 12:19:28.948694 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:19:28.948673 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"371fe2547bf91ab216b903a908b2e01f1f1e76f7f2eed511a0173ecaafa67a18\": container with ID starting with 371fe2547bf91ab216b903a908b2e01f1f1e76f7f2eed511a0173ecaafa67a18 not found: ID does not exist" containerID="371fe2547bf91ab216b903a908b2e01f1f1e76f7f2eed511a0173ecaafa67a18" Sep 10 12:19:28.948780 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.948700 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"371fe2547bf91ab216b903a908b2e01f1f1e76f7f2eed511a0173ecaafa67a18"} err="failed to get container status \"371fe2547bf91ab216b903a908b2e01f1f1e76f7f2eed511a0173ecaafa67a18\": rpc error: code = NotFound desc = could not find container \"371fe2547bf91ab216b903a908b2e01f1f1e76f7f2eed511a0173ecaafa67a18\": container with ID starting with 371fe2547bf91ab216b903a908b2e01f1f1e76f7f2eed511a0173ecaafa67a18 not found: ID does not exist" Sep 10 12:19:28.962946 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.962919 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-bf6588865-dxmvn"] Sep 10 12:19:28.969048 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:28.969026 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-bf6588865-dxmvn"] Sep 10 12:19:30.470707 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:30.470620 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d2f46f3-7193-4c2a-be48-f2458d2a0960" path="/var/lib/kubelet/pods/8d2f46f3-7193-4c2a-be48-f2458d2a0960/volumes" Sep 10 12:19:35.822883 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:35.822832 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-k9bgv" Sep 10 12:19:36.759995 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:36.759965 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:36.760167 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:36.760017 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:36.764829 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:36.764802 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:36.978328 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:36.978299 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:19:47.058132 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:47.058090 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6f5f7bdd8d-hx87h" podUID="02bc865a-2994-4d11-8f15-dc17ef64f9c4" containerName="console" containerID="cri-o://86d0e5916a41b2c54fd6e551a4c6c79f35cf59386838caba2f332a80f2fb8639" gracePeriod=15 Sep 10 12:19:47.331371 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:47.331312 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f5f7bdd8d-hx87h" Sep 10 12:19:47.359796 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:47.359770 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/02bc865a-2994-4d11-8f15-dc17ef64f9c4-console-serving-cert\") pod \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\" (UID: \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\") " Sep 10 12:19:47.359943 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:47.359833 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/02bc865a-2994-4d11-8f15-dc17ef64f9c4-console-oauth-config\") pod \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\" (UID: \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\") " Sep 10 12:19:47.360007 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:47.359943 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/02bc865a-2994-4d11-8f15-dc17ef64f9c4-service-ca\") pod \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\" (UID: \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\") " Sep 10 12:19:47.360007 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:47.359994 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/02bc865a-2994-4d11-8f15-dc17ef64f9c4-oauth-serving-cert\") pod \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\" (UID: \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\") " Sep 10 12:19:47.360119 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:47.360059 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wt8wj\" (UniqueName: \"kubernetes.io/projected/02bc865a-2994-4d11-8f15-dc17ef64f9c4-kube-api-access-wt8wj\") pod \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\" (UID: \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\") " Sep 10 12:19:47.360119 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:47.360101 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/02bc865a-2994-4d11-8f15-dc17ef64f9c4-console-config\") pod \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\" (UID: \"02bc865a-2994-4d11-8f15-dc17ef64f9c4\") " Sep 10 12:19:47.360370 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:47.360323 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02bc865a-2994-4d11-8f15-dc17ef64f9c4-service-ca" (OuterVolumeSpecName: "service-ca") pod "02bc865a-2994-4d11-8f15-dc17ef64f9c4" (UID: "02bc865a-2994-4d11-8f15-dc17ef64f9c4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:19:47.360562 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:47.360369 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02bc865a-2994-4d11-8f15-dc17ef64f9c4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "02bc865a-2994-4d11-8f15-dc17ef64f9c4" (UID: "02bc865a-2994-4d11-8f15-dc17ef64f9c4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:19:47.360642 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:47.360589 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02bc865a-2994-4d11-8f15-dc17ef64f9c4-console-config" (OuterVolumeSpecName: "console-config") pod "02bc865a-2994-4d11-8f15-dc17ef64f9c4" (UID: "02bc865a-2994-4d11-8f15-dc17ef64f9c4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:19:47.362722 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:47.362682 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02bc865a-2994-4d11-8f15-dc17ef64f9c4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "02bc865a-2994-4d11-8f15-dc17ef64f9c4" (UID: "02bc865a-2994-4d11-8f15-dc17ef64f9c4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:19:47.362816 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:47.362757 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02bc865a-2994-4d11-8f15-dc17ef64f9c4-kube-api-access-wt8wj" (OuterVolumeSpecName: "kube-api-access-wt8wj") pod "02bc865a-2994-4d11-8f15-dc17ef64f9c4" (UID: "02bc865a-2994-4d11-8f15-dc17ef64f9c4"). InnerVolumeSpecName "kube-api-access-wt8wj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:19:47.362974 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:47.362953 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02bc865a-2994-4d11-8f15-dc17ef64f9c4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "02bc865a-2994-4d11-8f15-dc17ef64f9c4" (UID: "02bc865a-2994-4d11-8f15-dc17ef64f9c4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:19:47.461601 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:47.461562 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/02bc865a-2994-4d11-8f15-dc17ef64f9c4-console-serving-cert\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:19:47.461601 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:47.461592 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/02bc865a-2994-4d11-8f15-dc17ef64f9c4-console-oauth-config\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:19:47.461805 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:47.461610 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/02bc865a-2994-4d11-8f15-dc17ef64f9c4-service-ca\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:19:47.461805 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:47.461622 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/02bc865a-2994-4d11-8f15-dc17ef64f9c4-oauth-serving-cert\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:19:47.461805 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:47.461634 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wt8wj\" (UniqueName: \"kubernetes.io/projected/02bc865a-2994-4d11-8f15-dc17ef64f9c4-kube-api-access-wt8wj\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:19:47.461805 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:47.461646 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/02bc865a-2994-4d11-8f15-dc17ef64f9c4-console-config\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:19:48.016078 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:48.016045 2583 generic.go:356] "Generic (PLEG): container finished" podID="02bc865a-2994-4d11-8f15-dc17ef64f9c4" containerID="86d0e5916a41b2c54fd6e551a4c6c79f35cf59386838caba2f332a80f2fb8639" exitCode=0 Sep 10 12:19:48.016236 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:48.016083 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f5f7bdd8d-hx87h" event={"ID":"02bc865a-2994-4d11-8f15-dc17ef64f9c4","Type":"ContainerDied","Data":"86d0e5916a41b2c54fd6e551a4c6c79f35cf59386838caba2f332a80f2fb8639"} Sep 10 12:19:48.016236 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:48.016090 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f5f7bdd8d-hx87h" Sep 10 12:19:48.016236 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:48.016122 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f5f7bdd8d-hx87h" event={"ID":"02bc865a-2994-4d11-8f15-dc17ef64f9c4","Type":"ContainerDied","Data":"ddaee843be42a9a411a7dcac7a88dbde33e1f8a98cd64d02a2ba611687fc032e"} Sep 10 12:19:48.016236 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:48.016141 2583 scope.go:117] "RemoveContainer" containerID="86d0e5916a41b2c54fd6e551a4c6c79f35cf59386838caba2f332a80f2fb8639" Sep 10 12:19:48.024405 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:48.024373 2583 scope.go:117] "RemoveContainer" containerID="86d0e5916a41b2c54fd6e551a4c6c79f35cf59386838caba2f332a80f2fb8639" Sep 10 12:19:48.024682 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:19:48.024657 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86d0e5916a41b2c54fd6e551a4c6c79f35cf59386838caba2f332a80f2fb8639\": container with ID starting with 86d0e5916a41b2c54fd6e551a4c6c79f35cf59386838caba2f332a80f2fb8639 not found: ID does not exist" containerID="86d0e5916a41b2c54fd6e551a4c6c79f35cf59386838caba2f332a80f2fb8639" Sep 10 12:19:48.024773 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:48.024690 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86d0e5916a41b2c54fd6e551a4c6c79f35cf59386838caba2f332a80f2fb8639"} err="failed to get container status \"86d0e5916a41b2c54fd6e551a4c6c79f35cf59386838caba2f332a80f2fb8639\": rpc error: code = NotFound desc = could not find container \"86d0e5916a41b2c54fd6e551a4c6c79f35cf59386838caba2f332a80f2fb8639\": container with ID starting with 86d0e5916a41b2c54fd6e551a4c6c79f35cf59386838caba2f332a80f2fb8639 not found: ID does not exist" Sep 10 12:19:48.039952 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:48.039932 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f5f7bdd8d-hx87h"] Sep 10 12:19:48.043765 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:48.043745 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f5f7bdd8d-hx87h"] Sep 10 12:19:48.469391 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:19:48.469362 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02bc865a-2994-4d11-8f15-dc17ef64f9c4" path="/var/lib/kubelet/pods/02bc865a-2994-4d11-8f15-dc17ef64f9c4/volumes" Sep 10 12:20:05.075302 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:05.075267 2583 generic.go:356] "Generic (PLEG): container finished" podID="5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2" containerID="25eedb09e7488aad200199fb6d751d39198c97ce95f9a41dab0a6299a2fe9c99" exitCode=0 Sep 10 12:20:05.075706 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:05.075341 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-kmrbv" event={"ID":"5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2","Type":"ContainerDied","Data":"25eedb09e7488aad200199fb6d751d39198c97ce95f9a41dab0a6299a2fe9c99"} Sep 10 12:20:05.075706 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:05.075633 2583 scope.go:117] "RemoveContainer" containerID="25eedb09e7488aad200199fb6d751d39198c97ce95f9a41dab0a6299a2fe9c99" Sep 10 12:20:06.081224 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:06.081189 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-kmrbv" event={"ID":"5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2","Type":"ContainerStarted","Data":"8a272d13591d27662c5e1c3b0255ffee8283d7b766ce26fc1bc2748de9e569f2"} Sep 10 12:20:49.552289 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.552206 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64c5b7b49c-tgsd9"] Sep 10 12:20:49.552874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.552576 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d2f46f3-7193-4c2a-be48-f2458d2a0960" containerName="registry" Sep 10 12:20:49.552874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.552595 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d2f46f3-7193-4c2a-be48-f2458d2a0960" containerName="registry" Sep 10 12:20:49.552874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.552608 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02bc865a-2994-4d11-8f15-dc17ef64f9c4" containerName="console" Sep 10 12:20:49.552874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.552617 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="02bc865a-2994-4d11-8f15-dc17ef64f9c4" containerName="console" Sep 10 12:20:49.552874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.552706 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d2f46f3-7193-4c2a-be48-f2458d2a0960" containerName="registry" Sep 10 12:20:49.552874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.552720 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="02bc865a-2994-4d11-8f15-dc17ef64f9c4" containerName="console" Sep 10 12:20:49.554493 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.554477 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:20:49.571265 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.571243 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64c5b7b49c-tgsd9"] Sep 10 12:20:49.707395 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.707352 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6g4b\" (UniqueName: \"kubernetes.io/projected/fa4a3663-9778-41f6-a299-7c24907bd296-kube-api-access-d6g4b\") pod \"console-64c5b7b49c-tgsd9\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:20:49.707557 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.707417 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa4a3663-9778-41f6-a299-7c24907bd296-service-ca\") pod \"console-64c5b7b49c-tgsd9\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:20:49.707557 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.707462 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa4a3663-9778-41f6-a299-7c24907bd296-oauth-serving-cert\") pod \"console-64c5b7b49c-tgsd9\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:20:49.707557 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.707500 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa4a3663-9778-41f6-a299-7c24907bd296-console-serving-cert\") pod \"console-64c5b7b49c-tgsd9\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:20:49.707557 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.707527 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa4a3663-9778-41f6-a299-7c24907bd296-trusted-ca-bundle\") pod \"console-64c5b7b49c-tgsd9\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:20:49.707708 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.707563 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa4a3663-9778-41f6-a299-7c24907bd296-console-oauth-config\") pod \"console-64c5b7b49c-tgsd9\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:20:49.707708 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.707621 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa4a3663-9778-41f6-a299-7c24907bd296-console-config\") pod \"console-64c5b7b49c-tgsd9\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:20:49.808386 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.808318 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa4a3663-9778-41f6-a299-7c24907bd296-service-ca\") pod \"console-64c5b7b49c-tgsd9\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:20:49.808386 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.808364 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa4a3663-9778-41f6-a299-7c24907bd296-oauth-serving-cert\") pod \"console-64c5b7b49c-tgsd9\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:20:49.808551 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.808395 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa4a3663-9778-41f6-a299-7c24907bd296-console-serving-cert\") pod \"console-64c5b7b49c-tgsd9\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:20:49.808551 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.808422 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa4a3663-9778-41f6-a299-7c24907bd296-trusted-ca-bundle\") pod \"console-64c5b7b49c-tgsd9\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:20:49.808551 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.808456 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa4a3663-9778-41f6-a299-7c24907bd296-console-oauth-config\") pod \"console-64c5b7b49c-tgsd9\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:20:49.808551 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.808490 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa4a3663-9778-41f6-a299-7c24907bd296-console-config\") pod \"console-64c5b7b49c-tgsd9\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:20:49.808796 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.808765 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6g4b\" (UniqueName: \"kubernetes.io/projected/fa4a3663-9778-41f6-a299-7c24907bd296-kube-api-access-d6g4b\") pod \"console-64c5b7b49c-tgsd9\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:20:49.809143 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.809115 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa4a3663-9778-41f6-a299-7c24907bd296-service-ca\") pod \"console-64c5b7b49c-tgsd9\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:20:49.809143 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.809132 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa4a3663-9778-41f6-a299-7c24907bd296-oauth-serving-cert\") pod \"console-64c5b7b49c-tgsd9\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:20:49.809313 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.809126 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa4a3663-9778-41f6-a299-7c24907bd296-console-config\") pod \"console-64c5b7b49c-tgsd9\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:20:49.809424 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.809401 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa4a3663-9778-41f6-a299-7c24907bd296-trusted-ca-bundle\") pod \"console-64c5b7b49c-tgsd9\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:20:49.810718 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.810693 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa4a3663-9778-41f6-a299-7c24907bd296-console-serving-cert\") pod \"console-64c5b7b49c-tgsd9\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:20:49.810899 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.810878 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa4a3663-9778-41f6-a299-7c24907bd296-console-oauth-config\") pod \"console-64c5b7b49c-tgsd9\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:20:49.817904 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.817883 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6g4b\" (UniqueName: \"kubernetes.io/projected/fa4a3663-9778-41f6-a299-7c24907bd296-kube-api-access-d6g4b\") pod \"console-64c5b7b49c-tgsd9\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:20:49.863613 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:49.863587 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:20:50.136359 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:50.136176 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64c5b7b49c-tgsd9"] Sep 10 12:20:50.138931 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:20:50.138904 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa4a3663_9778_41f6_a299_7c24907bd296.slice/crio-0ee026358b35287b87eb10b71d9b054a8b432ff1e299462aec71d9bbab123dab WatchSource:0}: Error finding container 0ee026358b35287b87eb10b71d9b054a8b432ff1e299462aec71d9bbab123dab: Status 404 returned error can't find the container with id 0ee026358b35287b87eb10b71d9b054a8b432ff1e299462aec71d9bbab123dab Sep 10 12:20:50.238341 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:50.238306 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64c5b7b49c-tgsd9" event={"ID":"fa4a3663-9778-41f6-a299-7c24907bd296","Type":"ContainerStarted","Data":"77989397512b1a802c7d5d1551cbf7cb2fc5dba05f2e8515f4638b5bf32b6157"} Sep 10 12:20:50.238500 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:50.238365 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64c5b7b49c-tgsd9" event={"ID":"fa4a3663-9778-41f6-a299-7c24907bd296","Type":"ContainerStarted","Data":"0ee026358b35287b87eb10b71d9b054a8b432ff1e299462aec71d9bbab123dab"} Sep 10 12:20:50.263938 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:50.263883 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64c5b7b49c-tgsd9" podStartSLOduration=1.2638669359999999 podStartE2EDuration="1.263866936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:20:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:20:50.263294982 +0000 UTC m=+178.268514446" watchObservedRunningTime="2026-09-10 12:20:50.263866936 +0000 UTC m=+178.269086392" Sep 10 12:20:56.694994 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.694957 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64c5b7b49c-tgsd9"] Sep 10 12:20:56.729147 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.729115 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d54cf98fc-8g8p2"] Sep 10 12:20:56.732881 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.732279 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:20:56.744024 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.744000 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d54cf98fc-8g8p2"] Sep 10 12:20:56.760191 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.760165 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-trusted-ca-bundle\") pod \"console-d54cf98fc-8g8p2\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:20:56.760311 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.760210 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-console-config\") pod \"console-d54cf98fc-8g8p2\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:20:56.760311 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.760235 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-service-ca\") pod \"console-d54cf98fc-8g8p2\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:20:56.760311 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.760262 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmgtj\" (UniqueName: \"kubernetes.io/projected/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-kube-api-access-gmgtj\") pod \"console-d54cf98fc-8g8p2\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:20:56.760311 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.760292 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-oauth-serving-cert\") pod \"console-d54cf98fc-8g8p2\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:20:56.760482 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.760322 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-console-oauth-config\") pod \"console-d54cf98fc-8g8p2\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:20:56.760482 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.760367 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-console-serving-cert\") pod \"console-d54cf98fc-8g8p2\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:20:56.860982 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.860960 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-trusted-ca-bundle\") pod \"console-d54cf98fc-8g8p2\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:20:56.861099 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.860990 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-console-config\") pod \"console-d54cf98fc-8g8p2\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:20:56.861099 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.861006 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-service-ca\") pod \"console-d54cf98fc-8g8p2\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:20:56.861099 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.861022 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmgtj\" (UniqueName: \"kubernetes.io/projected/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-kube-api-access-gmgtj\") pod \"console-d54cf98fc-8g8p2\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:20:56.861099 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.861042 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-oauth-serving-cert\") pod \"console-d54cf98fc-8g8p2\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:20:56.861099 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.861070 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-console-oauth-config\") pod \"console-d54cf98fc-8g8p2\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:20:56.861099 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.861095 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-console-serving-cert\") pod \"console-d54cf98fc-8g8p2\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:20:56.861732 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.861706 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-console-config\") pod \"console-d54cf98fc-8g8p2\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:20:56.861892 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.861867 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-oauth-serving-cert\") pod \"console-d54cf98fc-8g8p2\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:20:56.861972 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.861949 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-service-ca\") pod \"console-d54cf98fc-8g8p2\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:20:56.862034 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.862013 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-trusted-ca-bundle\") pod \"console-d54cf98fc-8g8p2\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:20:56.863635 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.863614 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-console-serving-cert\") pod \"console-d54cf98fc-8g8p2\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:20:56.863635 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.863625 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-console-oauth-config\") pod \"console-d54cf98fc-8g8p2\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:20:56.870208 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:56.870181 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmgtj\" (UniqueName: \"kubernetes.io/projected/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-kube-api-access-gmgtj\") pod \"console-d54cf98fc-8g8p2\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:20:57.042200 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:57.042175 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:20:57.321331 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:57.321252 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d54cf98fc-8g8p2"] Sep 10 12:20:57.324149 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:20:57.324119 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b081c33_5dfa_461a_b71f_46c7cc73a4b8.slice/crio-6cc0e54430499c81d06fa03f1c030b01986569a73d8aa288a44bef1e7337aa9c WatchSource:0}: Error finding container 6cc0e54430499c81d06fa03f1c030b01986569a73d8aa288a44bef1e7337aa9c: Status 404 returned error can't find the container with id 6cc0e54430499c81d06fa03f1c030b01986569a73d8aa288a44bef1e7337aa9c Sep 10 12:20:58.269037 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:58.268998 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d54cf98fc-8g8p2" event={"ID":"6b081c33-5dfa-461a-b71f-46c7cc73a4b8","Type":"ContainerStarted","Data":"688acf06b7336554091a6b87106b34e4188ebb528f11353d2e91f4869da08efd"} Sep 10 12:20:58.269408 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:58.269048 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d54cf98fc-8g8p2" event={"ID":"6b081c33-5dfa-461a-b71f-46c7cc73a4b8","Type":"ContainerStarted","Data":"6cc0e54430499c81d06fa03f1c030b01986569a73d8aa288a44bef1e7337aa9c"} Sep 10 12:20:58.312515 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:58.312466 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d54cf98fc-8g8p2" podStartSLOduration=2.312450387 podStartE2EDuration="2.312450387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:20:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:20:58.311289625 +0000 UTC m=+186.316509084" watchObservedRunningTime="2026-09-10 12:20:58.312450387 +0000 UTC m=+186.317669847" Sep 10 12:20:59.864085 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:20:59.864055 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:21:07.042784 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:07.042739 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:21:07.043407 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:07.042810 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:21:07.047596 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:07.047573 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:21:07.305676 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:07.305591 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:21:07.352412 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:07.352376 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cbff746c-k9wrl"] Sep 10 12:21:21.714038 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:21.713999 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-64c5b7b49c-tgsd9" podUID="fa4a3663-9778-41f6-a299-7c24907bd296" containerName="console" containerID="cri-o://77989397512b1a802c7d5d1551cbf7cb2fc5dba05f2e8515f4638b5bf32b6157" gracePeriod=15 Sep 10 12:21:21.946157 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:21.946133 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:21:22.044311 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.044281 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa4a3663-9778-41f6-a299-7c24907bd296-oauth-serving-cert\") pod \"fa4a3663-9778-41f6-a299-7c24907bd296\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " Sep 10 12:21:22.044467 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.044329 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa4a3663-9778-41f6-a299-7c24907bd296-console-oauth-config\") pod \"fa4a3663-9778-41f6-a299-7c24907bd296\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " Sep 10 12:21:22.044467 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.044354 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6g4b\" (UniqueName: \"kubernetes.io/projected/fa4a3663-9778-41f6-a299-7c24907bd296-kube-api-access-d6g4b\") pod \"fa4a3663-9778-41f6-a299-7c24907bd296\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " Sep 10 12:21:22.044467 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.044377 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa4a3663-9778-41f6-a299-7c24907bd296-console-config\") pod \"fa4a3663-9778-41f6-a299-7c24907bd296\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " Sep 10 12:21:22.044467 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.044419 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa4a3663-9778-41f6-a299-7c24907bd296-service-ca\") pod \"fa4a3663-9778-41f6-a299-7c24907bd296\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " Sep 10 12:21:22.044673 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.044478 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa4a3663-9778-41f6-a299-7c24907bd296-console-serving-cert\") pod \"fa4a3663-9778-41f6-a299-7c24907bd296\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " Sep 10 12:21:22.044673 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.044527 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa4a3663-9778-41f6-a299-7c24907bd296-trusted-ca-bundle\") pod \"fa4a3663-9778-41f6-a299-7c24907bd296\" (UID: \"fa4a3663-9778-41f6-a299-7c24907bd296\") " Sep 10 12:21:22.044801 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.044771 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa4a3663-9778-41f6-a299-7c24907bd296-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fa4a3663-9778-41f6-a299-7c24907bd296" (UID: "fa4a3663-9778-41f6-a299-7c24907bd296"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:21:22.044890 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.044827 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa4a3663-9778-41f6-a299-7c24907bd296-console-config" (OuterVolumeSpecName: "console-config") pod "fa4a3663-9778-41f6-a299-7c24907bd296" (UID: "fa4a3663-9778-41f6-a299-7c24907bd296"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:21:22.044951 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.044884 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa4a3663-9778-41f6-a299-7c24907bd296-service-ca" (OuterVolumeSpecName: "service-ca") pod "fa4a3663-9778-41f6-a299-7c24907bd296" (UID: "fa4a3663-9778-41f6-a299-7c24907bd296"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:21:22.045111 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.045070 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa4a3663-9778-41f6-a299-7c24907bd296-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fa4a3663-9778-41f6-a299-7c24907bd296" (UID: "fa4a3663-9778-41f6-a299-7c24907bd296"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:21:22.046488 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.046464 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa4a3663-9778-41f6-a299-7c24907bd296-kube-api-access-d6g4b" (OuterVolumeSpecName: "kube-api-access-d6g4b") pod "fa4a3663-9778-41f6-a299-7c24907bd296" (UID: "fa4a3663-9778-41f6-a299-7c24907bd296"). InnerVolumeSpecName "kube-api-access-d6g4b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:21:22.046884 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.046862 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa4a3663-9778-41f6-a299-7c24907bd296-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fa4a3663-9778-41f6-a299-7c24907bd296" (UID: "fa4a3663-9778-41f6-a299-7c24907bd296"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:21:22.046983 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.046881 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa4a3663-9778-41f6-a299-7c24907bd296-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fa4a3663-9778-41f6-a299-7c24907bd296" (UID: "fa4a3663-9778-41f6-a299-7c24907bd296"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:21:22.145370 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.145340 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fa4a3663-9778-41f6-a299-7c24907bd296-oauth-serving-cert\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:21:22.145370 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.145364 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fa4a3663-9778-41f6-a299-7c24907bd296-console-oauth-config\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:21:22.145370 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.145374 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6g4b\" (UniqueName: \"kubernetes.io/projected/fa4a3663-9778-41f6-a299-7c24907bd296-kube-api-access-d6g4b\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:21:22.145540 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.145385 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fa4a3663-9778-41f6-a299-7c24907bd296-console-config\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:21:22.145540 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.145394 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fa4a3663-9778-41f6-a299-7c24907bd296-service-ca\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:21:22.145540 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.145402 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fa4a3663-9778-41f6-a299-7c24907bd296-console-serving-cert\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:21:22.145540 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.145410 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa4a3663-9778-41f6-a299-7c24907bd296-trusted-ca-bundle\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:21:22.357744 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.357673 2583 generic.go:356] "Generic (PLEG): container finished" podID="fa4a3663-9778-41f6-a299-7c24907bd296" containerID="77989397512b1a802c7d5d1551cbf7cb2fc5dba05f2e8515f4638b5bf32b6157" exitCode=0 Sep 10 12:21:22.357744 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.357723 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64c5b7b49c-tgsd9" Sep 10 12:21:22.357947 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.357758 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64c5b7b49c-tgsd9" event={"ID":"fa4a3663-9778-41f6-a299-7c24907bd296","Type":"ContainerDied","Data":"77989397512b1a802c7d5d1551cbf7cb2fc5dba05f2e8515f4638b5bf32b6157"} Sep 10 12:21:22.357947 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.357810 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64c5b7b49c-tgsd9" event={"ID":"fa4a3663-9778-41f6-a299-7c24907bd296","Type":"ContainerDied","Data":"0ee026358b35287b87eb10b71d9b054a8b432ff1e299462aec71d9bbab123dab"} Sep 10 12:21:22.357947 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.357870 2583 scope.go:117] "RemoveContainer" containerID="77989397512b1a802c7d5d1551cbf7cb2fc5dba05f2e8515f4638b5bf32b6157" Sep 10 12:21:22.366256 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.366236 2583 scope.go:117] "RemoveContainer" containerID="77989397512b1a802c7d5d1551cbf7cb2fc5dba05f2e8515f4638b5bf32b6157" Sep 10 12:21:22.366539 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:21:22.366516 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77989397512b1a802c7d5d1551cbf7cb2fc5dba05f2e8515f4638b5bf32b6157\": container with ID starting with 77989397512b1a802c7d5d1551cbf7cb2fc5dba05f2e8515f4638b5bf32b6157 not found: ID does not exist" containerID="77989397512b1a802c7d5d1551cbf7cb2fc5dba05f2e8515f4638b5bf32b6157" Sep 10 12:21:22.366624 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.366546 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77989397512b1a802c7d5d1551cbf7cb2fc5dba05f2e8515f4638b5bf32b6157"} err="failed to get container status \"77989397512b1a802c7d5d1551cbf7cb2fc5dba05f2e8515f4638b5bf32b6157\": rpc error: code = NotFound desc = could not find container \"77989397512b1a802c7d5d1551cbf7cb2fc5dba05f2e8515f4638b5bf32b6157\": container with ID starting with 77989397512b1a802c7d5d1551cbf7cb2fc5dba05f2e8515f4638b5bf32b6157 not found: ID does not exist" Sep 10 12:21:22.378879 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.378855 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64c5b7b49c-tgsd9"] Sep 10 12:21:22.382311 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.382290 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64c5b7b49c-tgsd9"] Sep 10 12:21:22.468338 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:22.468311 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa4a3663-9778-41f6-a299-7c24907bd296" path="/var/lib/kubelet/pods/fa4a3663-9778-41f6-a299-7c24907bd296/volumes" Sep 10 12:21:32.378989 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:32.378927 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7cbff746c-k9wrl" podUID="84dcab2c-9a2a-496d-9dec-fd2916c137d3" containerName="console" containerID="cri-o://a569853047a140926e95c3b20a385fbc83ae4649b78b1343099b4d985b05851c" gracePeriod=15 Sep 10 12:21:32.621221 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:32.621196 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:21:32.724630 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:32.724531 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9cmsc\" (UniqueName: \"kubernetes.io/projected/84dcab2c-9a2a-496d-9dec-fd2916c137d3-kube-api-access-9cmsc\") pod \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " Sep 10 12:21:32.724630 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:32.724607 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84dcab2c-9a2a-496d-9dec-fd2916c137d3-console-serving-cert\") pod \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " Sep 10 12:21:32.724883 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:32.724725 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84dcab2c-9a2a-496d-9dec-fd2916c137d3-oauth-serving-cert\") pod \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " Sep 10 12:21:32.724883 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:32.724770 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84dcab2c-9a2a-496d-9dec-fd2916c137d3-service-ca\") pod \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " Sep 10 12:21:32.724883 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:32.724797 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84dcab2c-9a2a-496d-9dec-fd2916c137d3-console-config\") pod \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " Sep 10 12:21:32.724883 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:32.724860 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84dcab2c-9a2a-496d-9dec-fd2916c137d3-console-oauth-config\") pod \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " Sep 10 12:21:32.725086 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:32.724899 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84dcab2c-9a2a-496d-9dec-fd2916c137d3-trusted-ca-bundle\") pod \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\" (UID: \"84dcab2c-9a2a-496d-9dec-fd2916c137d3\") " Sep 10 12:21:32.725164 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:32.725137 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84dcab2c-9a2a-496d-9dec-fd2916c137d3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "84dcab2c-9a2a-496d-9dec-fd2916c137d3" (UID: "84dcab2c-9a2a-496d-9dec-fd2916c137d3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:21:32.725220 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:32.725181 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84dcab2c-9a2a-496d-9dec-fd2916c137d3-service-ca" (OuterVolumeSpecName: "service-ca") pod "84dcab2c-9a2a-496d-9dec-fd2916c137d3" (UID: "84dcab2c-9a2a-496d-9dec-fd2916c137d3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:21:32.725220 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:32.725194 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84dcab2c-9a2a-496d-9dec-fd2916c137d3-console-config" (OuterVolumeSpecName: "console-config") pod "84dcab2c-9a2a-496d-9dec-fd2916c137d3" (UID: "84dcab2c-9a2a-496d-9dec-fd2916c137d3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:21:32.725472 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:32.725451 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84dcab2c-9a2a-496d-9dec-fd2916c137d3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "84dcab2c-9a2a-496d-9dec-fd2916c137d3" (UID: "84dcab2c-9a2a-496d-9dec-fd2916c137d3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:21:32.726957 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:32.726933 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84dcab2c-9a2a-496d-9dec-fd2916c137d3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "84dcab2c-9a2a-496d-9dec-fd2916c137d3" (UID: "84dcab2c-9a2a-496d-9dec-fd2916c137d3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:21:32.727237 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:32.727211 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84dcab2c-9a2a-496d-9dec-fd2916c137d3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "84dcab2c-9a2a-496d-9dec-fd2916c137d3" (UID: "84dcab2c-9a2a-496d-9dec-fd2916c137d3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:21:32.727237 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:32.727224 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84dcab2c-9a2a-496d-9dec-fd2916c137d3-kube-api-access-9cmsc" (OuterVolumeSpecName: "kube-api-access-9cmsc") pod "84dcab2c-9a2a-496d-9dec-fd2916c137d3" (UID: "84dcab2c-9a2a-496d-9dec-fd2916c137d3"). InnerVolumeSpecName "kube-api-access-9cmsc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:21:32.826292 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:32.826261 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84dcab2c-9a2a-496d-9dec-fd2916c137d3-console-oauth-config\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:21:32.826292 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:32.826286 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84dcab2c-9a2a-496d-9dec-fd2916c137d3-trusted-ca-bundle\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:21:32.826292 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:32.826297 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9cmsc\" (UniqueName: \"kubernetes.io/projected/84dcab2c-9a2a-496d-9dec-fd2916c137d3-kube-api-access-9cmsc\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:21:32.826497 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:32.826306 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84dcab2c-9a2a-496d-9dec-fd2916c137d3-console-serving-cert\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:21:32.826497 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:32.826315 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84dcab2c-9a2a-496d-9dec-fd2916c137d3-oauth-serving-cert\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:21:32.826497 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:32.826324 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84dcab2c-9a2a-496d-9dec-fd2916c137d3-service-ca\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:21:32.826497 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:32.826332 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84dcab2c-9a2a-496d-9dec-fd2916c137d3-console-config\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:21:33.397333 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:33.397294 2583 generic.go:356] "Generic (PLEG): container finished" podID="84dcab2c-9a2a-496d-9dec-fd2916c137d3" containerID="a569853047a140926e95c3b20a385fbc83ae4649b78b1343099b4d985b05851c" exitCode=0 Sep 10 12:21:33.397722 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:33.397341 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cbff746c-k9wrl" Sep 10 12:21:33.397722 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:33.397379 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cbff746c-k9wrl" event={"ID":"84dcab2c-9a2a-496d-9dec-fd2916c137d3","Type":"ContainerDied","Data":"a569853047a140926e95c3b20a385fbc83ae4649b78b1343099b4d985b05851c"} Sep 10 12:21:33.397722 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:33.397433 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cbff746c-k9wrl" event={"ID":"84dcab2c-9a2a-496d-9dec-fd2916c137d3","Type":"ContainerDied","Data":"50ebd57a17eee1624b522decd56e728f86e34712a7043724100774b94a06af4f"} Sep 10 12:21:33.397722 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:33.397454 2583 scope.go:117] "RemoveContainer" containerID="a569853047a140926e95c3b20a385fbc83ae4649b78b1343099b4d985b05851c" Sep 10 12:21:33.406224 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:33.406204 2583 scope.go:117] "RemoveContainer" containerID="a569853047a140926e95c3b20a385fbc83ae4649b78b1343099b4d985b05851c" Sep 10 12:21:33.406477 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:21:33.406459 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a569853047a140926e95c3b20a385fbc83ae4649b78b1343099b4d985b05851c\": container with ID starting with a569853047a140926e95c3b20a385fbc83ae4649b78b1343099b4d985b05851c not found: ID does not exist" containerID="a569853047a140926e95c3b20a385fbc83ae4649b78b1343099b4d985b05851c" Sep 10 12:21:33.406525 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:33.406485 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a569853047a140926e95c3b20a385fbc83ae4649b78b1343099b4d985b05851c"} err="failed to get container status \"a569853047a140926e95c3b20a385fbc83ae4649b78b1343099b4d985b05851c\": rpc error: code = NotFound desc = could not find container \"a569853047a140926e95c3b20a385fbc83ae4649b78b1343099b4d985b05851c\": container with ID starting with a569853047a140926e95c3b20a385fbc83ae4649b78b1343099b4d985b05851c not found: ID does not exist" Sep 10 12:21:33.418419 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:33.418394 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cbff746c-k9wrl"] Sep 10 12:21:33.421116 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:33.421097 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cbff746c-k9wrl"] Sep 10 12:21:34.468762 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:21:34.468728 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84dcab2c-9a2a-496d-9dec-fd2916c137d3" path="/var/lib/kubelet/pods/84dcab2c-9a2a-496d-9dec-fd2916c137d3/volumes" Sep 10 12:22:52.355711 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:22:52.355681 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/ovn-acl-logging/0.log" Sep 10 12:22:52.356482 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:22:52.356448 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/ovn-acl-logging/0.log" Sep 10 12:22:52.361569 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:22:52.361554 2583 kubelet.go:1648] "Image garbage collection succeeded" Sep 10 12:23:01.834203 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:23:01.834169 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5dc8cf9588-qnkfh"] Sep 10 12:23:01.836498 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:23:01.834410 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84dcab2c-9a2a-496d-9dec-fd2916c137d3" containerName="console" Sep 10 12:23:01.836498 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:23:01.834421 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="84dcab2c-9a2a-496d-9dec-fd2916c137d3" containerName="console" Sep 10 12:23:01.836498 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:23:01.834438 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa4a3663-9778-41f6-a299-7c24907bd296" containerName="console" Sep 10 12:23:01.836498 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:23:01.834443 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa4a3663-9778-41f6-a299-7c24907bd296" containerName="console" Sep 10 12:23:01.836498 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:23:01.834488 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa4a3663-9778-41f6-a299-7c24907bd296" containerName="console" Sep 10 12:23:01.836498 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:23:01.834497 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="84dcab2c-9a2a-496d-9dec-fd2916c137d3" containerName="console" Sep 10 12:23:01.837367 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:23:01.837352 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5dc8cf9588-qnkfh" Sep 10 12:23:01.842486 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:23:01.842462 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 10 12:23:01.842593 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:23:01.842494 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 10 12:23:01.842593 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:23:01.842506 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-m62z9\"" Sep 10 12:23:01.851630 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:23:01.851588 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5dc8cf9588-qnkfh"] Sep 10 12:23:01.937644 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:23:01.937617 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnll6\" (UniqueName: \"kubernetes.io/projected/b6217731-d67f-45ad-b7dc-dbff6aca1f8e-kube-api-access-nnll6\") pod \"feast-operator-controller-manager-5dc8cf9588-qnkfh\" (UID: \"b6217731-d67f-45ad-b7dc-dbff6aca1f8e\") " pod="feast-operator-system/feast-operator-controller-manager-5dc8cf9588-qnkfh" Sep 10 12:23:02.037893 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:23:02.037867 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnll6\" (UniqueName: \"kubernetes.io/projected/b6217731-d67f-45ad-b7dc-dbff6aca1f8e-kube-api-access-nnll6\") pod \"feast-operator-controller-manager-5dc8cf9588-qnkfh\" (UID: \"b6217731-d67f-45ad-b7dc-dbff6aca1f8e\") " pod="feast-operator-system/feast-operator-controller-manager-5dc8cf9588-qnkfh" Sep 10 12:23:02.047144 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:23:02.047122 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnll6\" (UniqueName: \"kubernetes.io/projected/b6217731-d67f-45ad-b7dc-dbff6aca1f8e-kube-api-access-nnll6\") pod \"feast-operator-controller-manager-5dc8cf9588-qnkfh\" (UID: \"b6217731-d67f-45ad-b7dc-dbff6aca1f8e\") " pod="feast-operator-system/feast-operator-controller-manager-5dc8cf9588-qnkfh" Sep 10 12:23:02.147198 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:23:02.147126 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5dc8cf9588-qnkfh" Sep 10 12:23:02.426236 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:23:02.426154 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5dc8cf9588-qnkfh"] Sep 10 12:23:02.429083 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:23:02.429044 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6217731_d67f_45ad_b7dc_dbff6aca1f8e.slice/crio-f2b122ad81a1f7ab040cb2e831547b52f5bb57ac6fde08dcacd568fc6c57d9ee WatchSource:0}: Error finding container f2b122ad81a1f7ab040cb2e831547b52f5bb57ac6fde08dcacd568fc6c57d9ee: Status 404 returned error can't find the container with id f2b122ad81a1f7ab040cb2e831547b52f5bb57ac6fde08dcacd568fc6c57d9ee Sep 10 12:23:02.431030 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:23:02.431009 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 12:23:02.690705 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:23:02.690613 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5dc8cf9588-qnkfh" event={"ID":"b6217731-d67f-45ad-b7dc-dbff6aca1f8e","Type":"ContainerStarted","Data":"f2b122ad81a1f7ab040cb2e831547b52f5bb57ac6fde08dcacd568fc6c57d9ee"} Sep 10 12:23:05.702462 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:23:05.702421 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5dc8cf9588-qnkfh" event={"ID":"b6217731-d67f-45ad-b7dc-dbff6aca1f8e","Type":"ContainerStarted","Data":"ed5354db96ea12bd0475f4f68744c5d45f56623010a84fc1cadbf36874737f6d"} Sep 10 12:23:05.702918 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:23:05.702502 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-5dc8cf9588-qnkfh" Sep 10 12:23:05.719621 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:23:05.719566 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-5dc8cf9588-qnkfh" podStartSLOduration=2.395680988 podStartE2EDuration="4.71955337s" totalImagesPullingTime="2.323872382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:23:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:23:05.718757622 +0000 UTC m=+313.723977083" watchObservedRunningTime="2026-09-10 12:23:05.71955337 +0000 UTC m=+313.724772830" Sep 10 12:23:16.709347 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:23:16.709310 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-5dc8cf9588-qnkfh" Sep 10 12:24:38.353920 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.353884 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86d4f87cf6-5gs99"] Sep 10 12:24:38.356982 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.356958 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:38.375911 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.375888 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86d4f87cf6-5gs99"] Sep 10 12:24:38.414946 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.414916 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c82f3625-b939-4036-9c71-5bc21b83f685-oauth-serving-cert\") pod \"console-86d4f87cf6-5gs99\" (UID: \"c82f3625-b939-4036-9c71-5bc21b83f685\") " pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:38.415047 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.414954 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c82f3625-b939-4036-9c71-5bc21b83f685-service-ca\") pod \"console-86d4f87cf6-5gs99\" (UID: \"c82f3625-b939-4036-9c71-5bc21b83f685\") " pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:38.415047 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.414990 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c82f3625-b939-4036-9c71-5bc21b83f685-trusted-ca-bundle\") pod \"console-86d4f87cf6-5gs99\" (UID: \"c82f3625-b939-4036-9c71-5bc21b83f685\") " pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:38.415130 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.415042 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgd9v\" (UniqueName: \"kubernetes.io/projected/c82f3625-b939-4036-9c71-5bc21b83f685-kube-api-access-vgd9v\") pod \"console-86d4f87cf6-5gs99\" (UID: \"c82f3625-b939-4036-9c71-5bc21b83f685\") " pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:38.415130 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.415100 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c82f3625-b939-4036-9c71-5bc21b83f685-console-config\") pod \"console-86d4f87cf6-5gs99\" (UID: \"c82f3625-b939-4036-9c71-5bc21b83f685\") " pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:38.415192 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.415128 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c82f3625-b939-4036-9c71-5bc21b83f685-console-serving-cert\") pod \"console-86d4f87cf6-5gs99\" (UID: \"c82f3625-b939-4036-9c71-5bc21b83f685\") " pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:38.415192 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.415167 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c82f3625-b939-4036-9c71-5bc21b83f685-console-oauth-config\") pod \"console-86d4f87cf6-5gs99\" (UID: \"c82f3625-b939-4036-9c71-5bc21b83f685\") " pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:38.516071 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.516046 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c82f3625-b939-4036-9c71-5bc21b83f685-console-config\") pod \"console-86d4f87cf6-5gs99\" (UID: \"c82f3625-b939-4036-9c71-5bc21b83f685\") " pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:38.516176 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.516077 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c82f3625-b939-4036-9c71-5bc21b83f685-console-serving-cert\") pod \"console-86d4f87cf6-5gs99\" (UID: \"c82f3625-b939-4036-9c71-5bc21b83f685\") " pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:38.516176 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.516099 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c82f3625-b939-4036-9c71-5bc21b83f685-console-oauth-config\") pod \"console-86d4f87cf6-5gs99\" (UID: \"c82f3625-b939-4036-9c71-5bc21b83f685\") " pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:38.516176 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.516120 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c82f3625-b939-4036-9c71-5bc21b83f685-oauth-serving-cert\") pod \"console-86d4f87cf6-5gs99\" (UID: \"c82f3625-b939-4036-9c71-5bc21b83f685\") " pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:38.516336 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.516228 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c82f3625-b939-4036-9c71-5bc21b83f685-service-ca\") pod \"console-86d4f87cf6-5gs99\" (UID: \"c82f3625-b939-4036-9c71-5bc21b83f685\") " pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:38.516336 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.516282 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c82f3625-b939-4036-9c71-5bc21b83f685-trusted-ca-bundle\") pod \"console-86d4f87cf6-5gs99\" (UID: \"c82f3625-b939-4036-9c71-5bc21b83f685\") " pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:38.516336 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.516317 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgd9v\" (UniqueName: \"kubernetes.io/projected/c82f3625-b939-4036-9c71-5bc21b83f685-kube-api-access-vgd9v\") pod \"console-86d4f87cf6-5gs99\" (UID: \"c82f3625-b939-4036-9c71-5bc21b83f685\") " pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:38.516835 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.516810 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c82f3625-b939-4036-9c71-5bc21b83f685-oauth-serving-cert\") pod \"console-86d4f87cf6-5gs99\" (UID: \"c82f3625-b939-4036-9c71-5bc21b83f685\") " pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:38.516968 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.516930 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c82f3625-b939-4036-9c71-5bc21b83f685-service-ca\") pod \"console-86d4f87cf6-5gs99\" (UID: \"c82f3625-b939-4036-9c71-5bc21b83f685\") " pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:38.517122 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.517100 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c82f3625-b939-4036-9c71-5bc21b83f685-console-config\") pod \"console-86d4f87cf6-5gs99\" (UID: \"c82f3625-b939-4036-9c71-5bc21b83f685\") " pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:38.517187 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.517118 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c82f3625-b939-4036-9c71-5bc21b83f685-trusted-ca-bundle\") pod \"console-86d4f87cf6-5gs99\" (UID: \"c82f3625-b939-4036-9c71-5bc21b83f685\") " pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:38.518541 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.518511 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c82f3625-b939-4036-9c71-5bc21b83f685-console-serving-cert\") pod \"console-86d4f87cf6-5gs99\" (UID: \"c82f3625-b939-4036-9c71-5bc21b83f685\") " pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:38.518671 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.518649 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c82f3625-b939-4036-9c71-5bc21b83f685-console-oauth-config\") pod \"console-86d4f87cf6-5gs99\" (UID: \"c82f3625-b939-4036-9c71-5bc21b83f685\") " pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:38.525936 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.525914 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgd9v\" (UniqueName: \"kubernetes.io/projected/c82f3625-b939-4036-9c71-5bc21b83f685-kube-api-access-vgd9v\") pod \"console-86d4f87cf6-5gs99\" (UID: \"c82f3625-b939-4036-9c71-5bc21b83f685\") " pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:38.666164 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.666110 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:38.940601 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:38.940524 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86d4f87cf6-5gs99"] Sep 10 12:24:38.943401 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:24:38.943377 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc82f3625_b939_4036_9c71_5bc21b83f685.slice/crio-4da5ac0a7286213fe7f46dff04671d909c17fa70f2f46e40eec29988d0930e45 WatchSource:0}: Error finding container 4da5ac0a7286213fe7f46dff04671d909c17fa70f2f46e40eec29988d0930e45: Status 404 returned error can't find the container with id 4da5ac0a7286213fe7f46dff04671d909c17fa70f2f46e40eec29988d0930e45 Sep 10 12:24:39.029302 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:39.029273 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86d4f87cf6-5gs99" event={"ID":"c82f3625-b939-4036-9c71-5bc21b83f685","Type":"ContainerStarted","Data":"5cc7af33f9aeca9b7d57a2e0de9ee2fafa561238c0697288bb7b5a0555523b12"} Sep 10 12:24:39.029416 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:39.029319 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86d4f87cf6-5gs99" event={"ID":"c82f3625-b939-4036-9c71-5bc21b83f685","Type":"ContainerStarted","Data":"4da5ac0a7286213fe7f46dff04671d909c17fa70f2f46e40eec29988d0930e45"} Sep 10 12:24:39.048161 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:39.048113 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86d4f87cf6-5gs99" podStartSLOduration=1.04809541 podStartE2EDuration="1.04809541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:24:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:24:39.047678181 +0000 UTC m=+407.052897642" watchObservedRunningTime="2026-09-10 12:24:39.04809541 +0000 UTC m=+407.053314870" Sep 10 12:24:48.667130 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:48.667098 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:48.667499 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:48.667182 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:48.671885 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:48.671860 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:49.066273 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:49.066249 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-86d4f87cf6-5gs99" Sep 10 12:24:49.115381 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:24:49.115339 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d54cf98fc-8g8p2"] Sep 10 12:25:14.136424 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:14.136375 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-d54cf98fc-8g8p2" podUID="6b081c33-5dfa-461a-b71f-46c7cc73a4b8" containerName="console" containerID="cri-o://688acf06b7336554091a6b87106b34e4188ebb528f11353d2e91f4869da08efd" gracePeriod=15 Sep 10 12:25:14.379708 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:14.379686 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:25:14.473036 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:14.472962 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-service-ca\") pod \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " Sep 10 12:25:14.473251 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:14.473234 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-console-serving-cert\") pod \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " Sep 10 12:25:14.473362 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:14.473349 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-trusted-ca-bundle\") pod \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " Sep 10 12:25:14.473544 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:14.473511 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-oauth-serving-cert\") pod \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " Sep 10 12:25:14.473620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:14.473571 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-console-config\") pod \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " Sep 10 12:25:14.473620 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:14.473611 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmgtj\" (UniqueName: \"kubernetes.io/projected/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-kube-api-access-gmgtj\") pod \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " Sep 10 12:25:14.474058 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:14.473684 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-console-oauth-config\") pod \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\" (UID: \"6b081c33-5dfa-461a-b71f-46c7cc73a4b8\") " Sep 10 12:25:14.474175 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:14.474106 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-service-ca" (OuterVolumeSpecName: "service-ca") pod "6b081c33-5dfa-461a-b71f-46c7cc73a4b8" (UID: "6b081c33-5dfa-461a-b71f-46c7cc73a4b8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:25:14.474236 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:14.474216 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6b081c33-5dfa-461a-b71f-46c7cc73a4b8" (UID: "6b081c33-5dfa-461a-b71f-46c7cc73a4b8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:25:14.474366 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:14.474335 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6b081c33-5dfa-461a-b71f-46c7cc73a4b8" (UID: "6b081c33-5dfa-461a-b71f-46c7cc73a4b8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:25:14.474366 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:14.474341 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-console-config" (OuterVolumeSpecName: "console-config") pod "6b081c33-5dfa-461a-b71f-46c7cc73a4b8" (UID: "6b081c33-5dfa-461a-b71f-46c7cc73a4b8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:25:14.474513 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:14.474404 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-service-ca\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:25:14.474513 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:14.474425 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-trusted-ca-bundle\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:25:14.475942 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:14.475910 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6b081c33-5dfa-461a-b71f-46c7cc73a4b8" (UID: "6b081c33-5dfa-461a-b71f-46c7cc73a4b8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:25:14.476049 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:14.476023 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6b081c33-5dfa-461a-b71f-46c7cc73a4b8" (UID: "6b081c33-5dfa-461a-b71f-46c7cc73a4b8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:25:14.476201 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:14.476180 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-kube-api-access-gmgtj" (OuterVolumeSpecName: "kube-api-access-gmgtj") pod "6b081c33-5dfa-461a-b71f-46c7cc73a4b8" (UID: "6b081c33-5dfa-461a-b71f-46c7cc73a4b8"). InnerVolumeSpecName "kube-api-access-gmgtj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:25:14.575615 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:14.575586 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-console-oauth-config\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:25:14.575615 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:14.575611 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-console-serving-cert\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:25:14.575773 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:14.575626 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-oauth-serving-cert\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:25:14.575773 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:14.575640 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-console-config\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:25:14.575773 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:14.575655 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmgtj\" (UniqueName: \"kubernetes.io/projected/6b081c33-5dfa-461a-b71f-46c7cc73a4b8-kube-api-access-gmgtj\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:25:15.158678 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:15.158644 2583 generic.go:356] "Generic (PLEG): container finished" podID="6b081c33-5dfa-461a-b71f-46c7cc73a4b8" containerID="688acf06b7336554091a6b87106b34e4188ebb528f11353d2e91f4869da08efd" exitCode=0 Sep 10 12:25:15.159080 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:15.158707 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d54cf98fc-8g8p2" Sep 10 12:25:15.159080 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:15.158733 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d54cf98fc-8g8p2" event={"ID":"6b081c33-5dfa-461a-b71f-46c7cc73a4b8","Type":"ContainerDied","Data":"688acf06b7336554091a6b87106b34e4188ebb528f11353d2e91f4869da08efd"} Sep 10 12:25:15.159080 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:15.158788 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d54cf98fc-8g8p2" event={"ID":"6b081c33-5dfa-461a-b71f-46c7cc73a4b8","Type":"ContainerDied","Data":"6cc0e54430499c81d06fa03f1c030b01986569a73d8aa288a44bef1e7337aa9c"} Sep 10 12:25:15.159080 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:15.158814 2583 scope.go:117] "RemoveContainer" containerID="688acf06b7336554091a6b87106b34e4188ebb528f11353d2e91f4869da08efd" Sep 10 12:25:15.167478 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:15.167459 2583 scope.go:117] "RemoveContainer" containerID="688acf06b7336554091a6b87106b34e4188ebb528f11353d2e91f4869da08efd" Sep 10 12:25:15.167742 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:25:15.167722 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"688acf06b7336554091a6b87106b34e4188ebb528f11353d2e91f4869da08efd\": container with ID starting with 688acf06b7336554091a6b87106b34e4188ebb528f11353d2e91f4869da08efd not found: ID does not exist" containerID="688acf06b7336554091a6b87106b34e4188ebb528f11353d2e91f4869da08efd" Sep 10 12:25:15.167793 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:15.167749 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"688acf06b7336554091a6b87106b34e4188ebb528f11353d2e91f4869da08efd"} err="failed to get container status \"688acf06b7336554091a6b87106b34e4188ebb528f11353d2e91f4869da08efd\": rpc error: code = NotFound desc = could not find container \"688acf06b7336554091a6b87106b34e4188ebb528f11353d2e91f4869da08efd\": container with ID starting with 688acf06b7336554091a6b87106b34e4188ebb528f11353d2e91f4869da08efd not found: ID does not exist" Sep 10 12:25:15.180391 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:15.180367 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d54cf98fc-8g8p2"] Sep 10 12:25:15.184661 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:15.184641 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d54cf98fc-8g8p2"] Sep 10 12:25:16.469077 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:25:16.469040 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b081c33-5dfa-461a-b71f-46c7cc73a4b8" path="/var/lib/kubelet/pods/6b081c33-5dfa-461a-b71f-46c7cc73a4b8/volumes" Sep 10 12:27:52.377442 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:27:52.377410 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/ovn-acl-logging/0.log" Sep 10 12:27:52.378945 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:27:52.378923 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/ovn-acl-logging/0.log" Sep 10 12:29:23.769760 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:29:23.769719 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-69bbd798b6-bh6dg"] Sep 10 12:29:23.770226 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:29:23.770155 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b081c33-5dfa-461a-b71f-46c7cc73a4b8" containerName="console" Sep 10 12:29:23.770226 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:29:23.770175 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b081c33-5dfa-461a-b71f-46c7cc73a4b8" containerName="console" Sep 10 12:29:23.770295 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:29:23.770251 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b081c33-5dfa-461a-b71f-46c7cc73a4b8" containerName="console" Sep 10 12:29:23.773063 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:29:23.773043 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-bh6dg" Sep 10 12:29:23.775367 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:29:23.775348 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 10 12:29:23.776147 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:29:23.776127 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-h64cm\"" Sep 10 12:29:23.776495 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:29:23.776478 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 10 12:29:23.778037 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:29:23.778015 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-bh6dg"] Sep 10 12:29:23.862139 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:29:23.862111 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fnfr\" (UniqueName: \"kubernetes.io/projected/f86e4987-d66c-4168-b477-a9cb8fe3403d-kube-api-access-7fnfr\") pod \"redis-69bbd798b6-bh6dg\" (UID: \"f86e4987-d66c-4168-b477-a9cb8fe3403d\") " pod="test-ns-feast/redis-69bbd798b6-bh6dg" Sep 10 12:29:23.962429 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:29:23.962399 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fnfr\" (UniqueName: \"kubernetes.io/projected/f86e4987-d66c-4168-b477-a9cb8fe3403d-kube-api-access-7fnfr\") pod \"redis-69bbd798b6-bh6dg\" (UID: \"f86e4987-d66c-4168-b477-a9cb8fe3403d\") " pod="test-ns-feast/redis-69bbd798b6-bh6dg" Sep 10 12:29:23.971785 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:29:23.971752 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fnfr\" (UniqueName: \"kubernetes.io/projected/f86e4987-d66c-4168-b477-a9cb8fe3403d-kube-api-access-7fnfr\") pod \"redis-69bbd798b6-bh6dg\" (UID: \"f86e4987-d66c-4168-b477-a9cb8fe3403d\") " pod="test-ns-feast/redis-69bbd798b6-bh6dg" Sep 10 12:29:24.090783 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:29:24.090760 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-bh6dg" Sep 10 12:29:24.357849 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:29:24.357779 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-bh6dg"] Sep 10 12:29:24.360296 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:29:24.360268 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf86e4987_d66c_4168_b477_a9cb8fe3403d.slice/crio-5a3124398c5916e5250fa7ee7a78c8b4d46018ea4adf1d21f7ed547656795f60 WatchSource:0}: Error finding container 5a3124398c5916e5250fa7ee7a78c8b4d46018ea4adf1d21f7ed547656795f60: Status 404 returned error can't find the container with id 5a3124398c5916e5250fa7ee7a78c8b4d46018ea4adf1d21f7ed547656795f60 Sep 10 12:29:24.362049 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:29:24.362032 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 12:29:25.023519 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:29:25.023483 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-bh6dg" event={"ID":"f86e4987-d66c-4168-b477-a9cb8fe3403d","Type":"ContainerStarted","Data":"5a3124398c5916e5250fa7ee7a78c8b4d46018ea4adf1d21f7ed547656795f60"} Sep 10 12:29:26.028292 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:29:26.028209 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-bh6dg" event={"ID":"f86e4987-d66c-4168-b477-a9cb8fe3403d","Type":"ContainerStarted","Data":"f98fa74f6fa16c1ebb293a3ff51c84fd6c4cbf8e3d30dc9bdcf401e9198c5b00"} Sep 10 12:29:26.043728 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:29:26.043677 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/redis-69bbd798b6-bh6dg" podStartSLOduration=1.690508945 podStartE2EDuration="3.043662356s" totalImagesPullingTime="1.353153411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:29:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:29:26.042288338 +0000 UTC m=+694.047507797" watchObservedRunningTime="2026-09-10 12:29:26.043662356 +0000 UTC m=+694.048881816" Sep 10 12:31:00.691763 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:31:00.691729 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-bh6dg"] Sep 10 12:31:00.692276 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:31:00.691924 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/redis-69bbd798b6-bh6dg" podUID="f86e4987-d66c-4168-b477-a9cb8fe3403d" containerName="redis" containerID="cri-o://f98fa74f6fa16c1ebb293a3ff51c84fd6c4cbf8e3d30dc9bdcf401e9198c5b00" gracePeriod=30 Sep 10 12:31:00.927728 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:31:00.927705 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-bh6dg" Sep 10 12:31:01.047802 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:31:01.047769 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fnfr\" (UniqueName: \"kubernetes.io/projected/f86e4987-d66c-4168-b477-a9cb8fe3403d-kube-api-access-7fnfr\") pod \"f86e4987-d66c-4168-b477-a9cb8fe3403d\" (UID: \"f86e4987-d66c-4168-b477-a9cb8fe3403d\") " Sep 10 12:31:01.049803 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:31:01.049774 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f86e4987-d66c-4168-b477-a9cb8fe3403d-kube-api-access-7fnfr" (OuterVolumeSpecName: "kube-api-access-7fnfr") pod "f86e4987-d66c-4168-b477-a9cb8fe3403d" (UID: "f86e4987-d66c-4168-b477-a9cb8fe3403d"). InnerVolumeSpecName "kube-api-access-7fnfr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:31:01.148323 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:31:01.148296 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fnfr\" (UniqueName: \"kubernetes.io/projected/f86e4987-d66c-4168-b477-a9cb8fe3403d-kube-api-access-7fnfr\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:31:01.361538 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:31:01.361462 2583 generic.go:356] "Generic (PLEG): container finished" podID="f86e4987-d66c-4168-b477-a9cb8fe3403d" containerID="f98fa74f6fa16c1ebb293a3ff51c84fd6c4cbf8e3d30dc9bdcf401e9198c5b00" exitCode=0 Sep 10 12:31:01.361538 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:31:01.361504 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-bh6dg" Sep 10 12:31:01.361703 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:31:01.361550 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-bh6dg" event={"ID":"f86e4987-d66c-4168-b477-a9cb8fe3403d","Type":"ContainerDied","Data":"f98fa74f6fa16c1ebb293a3ff51c84fd6c4cbf8e3d30dc9bdcf401e9198c5b00"} Sep 10 12:31:01.361703 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:31:01.361593 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-bh6dg" event={"ID":"f86e4987-d66c-4168-b477-a9cb8fe3403d","Type":"ContainerDied","Data":"5a3124398c5916e5250fa7ee7a78c8b4d46018ea4adf1d21f7ed547656795f60"} Sep 10 12:31:01.361703 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:31:01.361615 2583 scope.go:117] "RemoveContainer" containerID="f98fa74f6fa16c1ebb293a3ff51c84fd6c4cbf8e3d30dc9bdcf401e9198c5b00" Sep 10 12:31:01.369510 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:31:01.369492 2583 scope.go:117] "RemoveContainer" containerID="f98fa74f6fa16c1ebb293a3ff51c84fd6c4cbf8e3d30dc9bdcf401e9198c5b00" Sep 10 12:31:01.369760 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:31:01.369738 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f98fa74f6fa16c1ebb293a3ff51c84fd6c4cbf8e3d30dc9bdcf401e9198c5b00\": container with ID starting with f98fa74f6fa16c1ebb293a3ff51c84fd6c4cbf8e3d30dc9bdcf401e9198c5b00 not found: ID does not exist" containerID="f98fa74f6fa16c1ebb293a3ff51c84fd6c4cbf8e3d30dc9bdcf401e9198c5b00" Sep 10 12:31:01.369822 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:31:01.369773 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f98fa74f6fa16c1ebb293a3ff51c84fd6c4cbf8e3d30dc9bdcf401e9198c5b00"} err="failed to get container status \"f98fa74f6fa16c1ebb293a3ff51c84fd6c4cbf8e3d30dc9bdcf401e9198c5b00\": rpc error: code = NotFound desc = could not find container \"f98fa74f6fa16c1ebb293a3ff51c84fd6c4cbf8e3d30dc9bdcf401e9198c5b00\": container with ID starting with f98fa74f6fa16c1ebb293a3ff51c84fd6c4cbf8e3d30dc9bdcf401e9198c5b00 not found: ID does not exist" Sep 10 12:31:01.380227 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:31:01.380207 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-bh6dg"] Sep 10 12:31:01.384233 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:31:01.384215 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-69bbd798b6-bh6dg"] Sep 10 12:31:02.468077 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:31:02.468046 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f86e4987-d66c-4168-b477-a9cb8fe3403d" path="/var/lib/kubelet/pods/f86e4987-d66c-4168-b477-a9cb8fe3403d/volumes" Sep 10 12:32:52.398882 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:32:52.398856 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/ovn-acl-logging/0.log" Sep 10 12:32:52.402561 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:32:52.402543 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/ovn-acl-logging/0.log" Sep 10 12:35:52.825453 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:35:52.825419 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5dc8cf9588-qnkfh"] Sep 10 12:35:52.825950 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:35:52.825624 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-5dc8cf9588-qnkfh" podUID="b6217731-d67f-45ad-b7dc-dbff6aca1f8e" containerName="manager" containerID="cri-o://ed5354db96ea12bd0475f4f68744c5d45f56623010a84fc1cadbf36874737f6d" gracePeriod=10 Sep 10 12:35:53.087617 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:35:53.087555 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5dc8cf9588-qnkfh" Sep 10 12:35:53.193257 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:35:53.193220 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnll6\" (UniqueName: \"kubernetes.io/projected/b6217731-d67f-45ad-b7dc-dbff6aca1f8e-kube-api-access-nnll6\") pod \"b6217731-d67f-45ad-b7dc-dbff6aca1f8e\" (UID: \"b6217731-d67f-45ad-b7dc-dbff6aca1f8e\") " Sep 10 12:35:53.195361 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:35:53.195337 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6217731-d67f-45ad-b7dc-dbff6aca1f8e-kube-api-access-nnll6" (OuterVolumeSpecName: "kube-api-access-nnll6") pod "b6217731-d67f-45ad-b7dc-dbff6aca1f8e" (UID: "b6217731-d67f-45ad-b7dc-dbff6aca1f8e"). InnerVolumeSpecName "kube-api-access-nnll6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:35:53.294684 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:35:53.294650 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnll6\" (UniqueName: \"kubernetes.io/projected/b6217731-d67f-45ad-b7dc-dbff6aca1f8e-kube-api-access-nnll6\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:35:53.376008 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:35:53.375926 2583 generic.go:356] "Generic (PLEG): container finished" podID="b6217731-d67f-45ad-b7dc-dbff6aca1f8e" containerID="ed5354db96ea12bd0475f4f68744c5d45f56623010a84fc1cadbf36874737f6d" exitCode=0 Sep 10 12:35:53.376008 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:35:53.375966 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5dc8cf9588-qnkfh" Sep 10 12:35:53.376008 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:35:53.375997 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5dc8cf9588-qnkfh" event={"ID":"b6217731-d67f-45ad-b7dc-dbff6aca1f8e","Type":"ContainerDied","Data":"ed5354db96ea12bd0475f4f68744c5d45f56623010a84fc1cadbf36874737f6d"} Sep 10 12:35:53.376212 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:35:53.376041 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5dc8cf9588-qnkfh" event={"ID":"b6217731-d67f-45ad-b7dc-dbff6aca1f8e","Type":"ContainerDied","Data":"f2b122ad81a1f7ab040cb2e831547b52f5bb57ac6fde08dcacd568fc6c57d9ee"} Sep 10 12:35:53.376212 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:35:53.376063 2583 scope.go:117] "RemoveContainer" containerID="ed5354db96ea12bd0475f4f68744c5d45f56623010a84fc1cadbf36874737f6d" Sep 10 12:35:53.389904 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:35:53.389881 2583 scope.go:117] "RemoveContainer" containerID="ed5354db96ea12bd0475f4f68744c5d45f56623010a84fc1cadbf36874737f6d" Sep 10 12:35:53.390170 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:35:53.390151 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed5354db96ea12bd0475f4f68744c5d45f56623010a84fc1cadbf36874737f6d\": container with ID starting with ed5354db96ea12bd0475f4f68744c5d45f56623010a84fc1cadbf36874737f6d not found: ID does not exist" containerID="ed5354db96ea12bd0475f4f68744c5d45f56623010a84fc1cadbf36874737f6d" Sep 10 12:35:53.390227 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:35:53.390178 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed5354db96ea12bd0475f4f68744c5d45f56623010a84fc1cadbf36874737f6d"} err="failed to get container status \"ed5354db96ea12bd0475f4f68744c5d45f56623010a84fc1cadbf36874737f6d\": rpc error: code = NotFound desc = could not find container \"ed5354db96ea12bd0475f4f68744c5d45f56623010a84fc1cadbf36874737f6d\": container with ID starting with ed5354db96ea12bd0475f4f68744c5d45f56623010a84fc1cadbf36874737f6d not found: ID does not exist" Sep 10 12:35:53.399636 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:35:53.399612 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5dc8cf9588-qnkfh"] Sep 10 12:35:53.401095 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:35:53.401075 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5dc8cf9588-qnkfh"] Sep 10 12:35:54.468943 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:35:54.468903 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6217731-d67f-45ad-b7dc-dbff6aca1f8e" path="/var/lib/kubelet/pods/b6217731-d67f-45ad-b7dc-dbff6aca1f8e/volumes" Sep 10 12:36:04.067203 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:04.067171 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-kzn5b"] Sep 10 12:36:04.067623 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:04.067417 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f86e4987-d66c-4168-b477-a9cb8fe3403d" containerName="redis" Sep 10 12:36:04.067623 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:04.067427 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f86e4987-d66c-4168-b477-a9cb8fe3403d" containerName="redis" Sep 10 12:36:04.067623 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:04.067442 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6217731-d67f-45ad-b7dc-dbff6aca1f8e" containerName="manager" Sep 10 12:36:04.067623 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:04.067448 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6217731-d67f-45ad-b7dc-dbff6aca1f8e" containerName="manager" Sep 10 12:36:04.067623 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:04.067486 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f86e4987-d66c-4168-b477-a9cb8fe3403d" containerName="redis" Sep 10 12:36:04.067623 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:04.067494 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6217731-d67f-45ad-b7dc-dbff6aca1f8e" containerName="manager" Sep 10 12:36:04.071684 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:04.071668 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kzn5b" Sep 10 12:36:04.076024 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:04.075996 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 10 12:36:04.076024 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:04.076011 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-bmgmk\"" Sep 10 12:36:04.076206 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:04.076004 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 10 12:36:04.080051 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:04.080030 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-kzn5b"] Sep 10 12:36:04.175524 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:04.175485 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fr9k\" (UniqueName: \"kubernetes.io/projected/0e31c99a-d958-4e92-b12a-6521b45e778c-kube-api-access-8fr9k\") pod \"feast-operator-controller-manager-689b877b4b-kzn5b\" (UID: \"0e31c99a-d958-4e92-b12a-6521b45e778c\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kzn5b" Sep 10 12:36:04.276642 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:04.276604 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fr9k\" (UniqueName: \"kubernetes.io/projected/0e31c99a-d958-4e92-b12a-6521b45e778c-kube-api-access-8fr9k\") pod \"feast-operator-controller-manager-689b877b4b-kzn5b\" (UID: \"0e31c99a-d958-4e92-b12a-6521b45e778c\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kzn5b" Sep 10 12:36:04.287551 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:04.287521 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fr9k\" (UniqueName: \"kubernetes.io/projected/0e31c99a-d958-4e92-b12a-6521b45e778c-kube-api-access-8fr9k\") pod \"feast-operator-controller-manager-689b877b4b-kzn5b\" (UID: \"0e31c99a-d958-4e92-b12a-6521b45e778c\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kzn5b" Sep 10 12:36:04.381381 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:04.381316 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kzn5b" Sep 10 12:36:04.650984 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:04.650961 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-kzn5b"] Sep 10 12:36:04.653195 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:36:04.653155 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e31c99a_d958_4e92_b12a_6521b45e778c.slice/crio-aa65230a4ee315b8e1da65d906a4d943e1e11d9562dc1a1a395f79dff3acf61c WatchSource:0}: Error finding container aa65230a4ee315b8e1da65d906a4d943e1e11d9562dc1a1a395f79dff3acf61c: Status 404 returned error can't find the container with id aa65230a4ee315b8e1da65d906a4d943e1e11d9562dc1a1a395f79dff3acf61c Sep 10 12:36:04.654914 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:04.654897 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 12:36:05.419483 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:05.419448 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kzn5b" event={"ID":"0e31c99a-d958-4e92-b12a-6521b45e778c","Type":"ContainerStarted","Data":"aa65230a4ee315b8e1da65d906a4d943e1e11d9562dc1a1a395f79dff3acf61c"} Sep 10 12:36:07.427479 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:07.427443 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kzn5b" event={"ID":"0e31c99a-d958-4e92-b12a-6521b45e778c","Type":"ContainerStarted","Data":"1ba454369d5dee4d3c7d66788c7d191685291bb78c604fdc6b56e826dc724b7e"} Sep 10 12:36:07.427907 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:07.427514 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kzn5b" Sep 10 12:36:07.443549 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:07.443410 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kzn5b" podStartSLOduration=1.102536519 podStartE2EDuration="3.443389631s" totalImagesPullingTime="2.340853112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:36:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:36:07.442299817 +0000 UTC m=+1095.447519277" watchObservedRunningTime="2026-09-10 12:36:07.443389631 +0000 UTC m=+1095.448609091" Sep 10 12:36:18.433931 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:18.433897 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kzn5b" Sep 10 12:36:20.965993 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:20.965951 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj"] Sep 10 12:36:20.970128 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:20.970105 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:20.972858 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:20.972816 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 10 12:36:20.973083 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:20.973064 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 10 12:36:20.973420 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:20.972820 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 10 12:36:20.973572 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:20.972891 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 10 12:36:20.973678 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:20.972904 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-9w5mx\"" Sep 10 12:36:20.973828 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:20.973074 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 10 12:36:20.973961 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:20.972816 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 10 12:36:20.983334 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:20.983313 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj"] Sep 10 12:36:21.110965 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:21.110931 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vphrj\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:21.111112 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:21.110983 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vphrj\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:21.111112 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:21.111069 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vphrj\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:21.111227 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:21.111127 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9cb2\" (UniqueName: \"kubernetes.io/projected/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-kube-api-access-h9cb2\") pod \"feast-simple-feast-setup-57576b8fcf-vphrj\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:21.111227 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:21.111164 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vphrj\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:21.111227 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:21.111193 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-vphrj\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:21.212057 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:21.212032 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9cb2\" (UniqueName: \"kubernetes.io/projected/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-kube-api-access-h9cb2\") pod \"feast-simple-feast-setup-57576b8fcf-vphrj\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:21.212255 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:21.212064 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vphrj\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:21.212255 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:21.212089 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-vphrj\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:21.212255 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:21.212113 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vphrj\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:21.212255 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:21.212141 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vphrj\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:21.212255 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:21.212171 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vphrj\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:21.212255 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:36:21.212195 2583 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 10 12:36:21.212585 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:36:21.212265 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-offline-tls podName:9e0ea2aa-87ac-4fb5-85d4-ad26382aa448 nodeName:}" failed. No retries permitted until 2026-09-10 12:36:21.712245283 +0000 UTC m=+1109.717464734 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-offline-tls") pod "feast-simple-feast-setup-57576b8fcf-vphrj" (UID: "9e0ea2aa-87ac-4fb5-85d4-ad26382aa448") : secret "feast-simple-feast-setup-offline-tls" not found Sep 10 12:36:21.212585 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:21.212471 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-vphrj\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:21.212696 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:36:21.212637 2583 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Sep 10 12:36:21.212747 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:36:21.212703 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-online-tls podName:9e0ea2aa-87ac-4fb5-85d4-ad26382aa448 nodeName:}" failed. No retries permitted until 2026-09-10 12:36:21.712687495 +0000 UTC m=+1109.717906936 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-online-tls") pod "feast-simple-feast-setup-57576b8fcf-vphrj" (UID: "9e0ea2aa-87ac-4fb5-85d4-ad26382aa448") : secret "feast-simple-feast-setup-online-tls" not found Sep 10 12:36:21.214662 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:21.214637 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vphrj\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:21.214794 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:21.214756 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vphrj\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:21.221057 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:21.220996 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9cb2\" (UniqueName: \"kubernetes.io/projected/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-kube-api-access-h9cb2\") pod \"feast-simple-feast-setup-57576b8fcf-vphrj\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:21.716855 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:21.716804 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vphrj\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:21.717022 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:21.716887 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vphrj\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:21.719224 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:21.719185 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vphrj\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:21.719358 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:21.719271 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-vphrj\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:21.888693 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:21.888643 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:22.169554 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:22.169509 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj"] Sep 10 12:36:22.172415 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:36:22.172387 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e0ea2aa_87ac_4fb5_85d4_ad26382aa448.slice/crio-90997f83f595aa9cbc1bdd7b65fb3eb8014cbc32a07928b2a71acb6f3db878e5 WatchSource:0}: Error finding container 90997f83f595aa9cbc1bdd7b65fb3eb8014cbc32a07928b2a71acb6f3db878e5: Status 404 returned error can't find the container with id 90997f83f595aa9cbc1bdd7b65fb3eb8014cbc32a07928b2a71acb6f3db878e5 Sep 10 12:36:22.491805 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:22.491707 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" event={"ID":"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448","Type":"ContainerStarted","Data":"90997f83f595aa9cbc1bdd7b65fb3eb8014cbc32a07928b2a71acb6f3db878e5"} Sep 10 12:36:35.549697 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:35.549658 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" event={"ID":"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448","Type":"ContainerStarted","Data":"43fc517f0f46b4574180650672ab6bdf5c9af5fcec6c5ae9be0c0a56fada93e7"} Sep 10 12:36:40.568628 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:40.568592 2583 generic.go:356] "Generic (PLEG): container finished" podID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerID="43fc517f0f46b4574180650672ab6bdf5c9af5fcec6c5ae9be0c0a56fada93e7" exitCode=0 Sep 10 12:36:40.569090 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:40.568666 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" event={"ID":"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448","Type":"ContainerDied","Data":"43fc517f0f46b4574180650672ab6bdf5c9af5fcec6c5ae9be0c0a56fada93e7"} Sep 10 12:36:41.574887 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:41.574851 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" event={"ID":"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448","Type":"ContainerStarted","Data":"c85b8087ceeb1987456c8b7861c3e4bfc319580ded0e9193c2d34a12cd1dcb4a"} Sep 10 12:36:46.596057 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:46.596022 2583 generic.go:356] "Generic (PLEG): container finished" podID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerID="c85b8087ceeb1987456c8b7861c3e4bfc319580ded0e9193c2d34a12cd1dcb4a" exitCode=0 Sep 10 12:36:46.596387 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:46.596085 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" event={"ID":"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448","Type":"ContainerDied","Data":"c85b8087ceeb1987456c8b7861c3e4bfc319580ded0e9193c2d34a12cd1dcb4a"} Sep 10 12:36:47.605378 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:47.605330 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" event={"ID":"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448","Type":"ContainerStarted","Data":"8c48f7aae6251b9c6aa5e6adba96981b728a6a3c4d5f47082275bf398ad5763a"} Sep 10 12:36:47.605873 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:47.605392 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" event={"ID":"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448","Type":"ContainerStarted","Data":"0d1a35fda49eb1a8ab798ac3e3a796669265f29bd7e4098aa1f59992b4ed8032"} Sep 10 12:36:47.605873 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:47.605413 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" event={"ID":"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448","Type":"ContainerStarted","Data":"227813a99cec654273f7ca68bd9e7e0ce9fd8a57a7d7831fd1fa86c675319908"} Sep 10 12:36:47.605873 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:47.605431 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" event={"ID":"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448","Type":"ContainerStarted","Data":"8dc1f8643772a9fdaa1100085a9ed6998d7ea6d0665b9e403f5e32136208f8cf"} Sep 10 12:36:47.629714 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:47.629648 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" podStartSLOduration=15.062243234 podStartE2EDuration="27.62962521s" totalImagesPullingTime="12.567381976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:36:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:36:47.628057181 +0000 UTC m=+1135.633276646" watchObservedRunningTime="2026-09-10 12:36:47.62962521 +0000 UTC m=+1135.634844671" Sep 10 12:36:48.889793 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:48.889666 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:48.889793 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:48.889731 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:48.889793 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:48.889751 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:48.890407 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:48.889889 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:48.892133 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:48.892093 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused" Sep 10 12:36:48.892273 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:48.892130 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.26:8816: connect: connection refused" Sep 10 12:36:48.892273 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:48.892176 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.26:6571: connect: connection refused" Sep 10 12:36:48.892273 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:48.892137 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 10 12:36:51.889218 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:51.889161 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:51.889700 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:51.889241 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:51.889700 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:51.889417 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused" Sep 10 12:36:51.889700 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:51.889490 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.26:8816: connect: connection refused" Sep 10 12:36:51.889700 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:51.889542 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 10 12:36:51.889700 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:51.889584 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:51.889700 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:51.889628 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:51.890059 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:51.889753 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.26:6571: connect: connection refused" Sep 10 12:36:54.889495 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:54.889443 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused" Sep 10 12:36:54.890117 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:54.890096 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:54.890593 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:54.890570 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:54.890970 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:54.890952 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:54.895590 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:54.895568 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:55.650979 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:55.650947 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:55.653896 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:55.653876 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:57.889925 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:57.889898 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:36:58.661263 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:36:58.661234 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:37:52.421128 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:37:52.421100 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/ovn-acl-logging/0.log" Sep 10 12:37:52.425283 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:37:52.425262 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/ovn-acl-logging/0.log" Sep 10 12:38:01.376064 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.376029 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj"] Sep 10 12:38:01.376528 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.376274 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="registry" containerID="cri-o://8dc1f8643772a9fdaa1100085a9ed6998d7ea6d0665b9e403f5e32136208f8cf" gracePeriod=30 Sep 10 12:38:01.376528 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.376340 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="online" containerID="cri-o://227813a99cec654273f7ca68bd9e7e0ce9fd8a57a7d7831fd1fa86c675319908" gracePeriod=30 Sep 10 12:38:01.376528 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.376339 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="ui" containerID="cri-o://8c48f7aae6251b9c6aa5e6adba96981b728a6a3c4d5f47082275bf398ad5763a" gracePeriod=30 Sep 10 12:38:01.376692 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.376362 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="offline" containerID="cri-o://0d1a35fda49eb1a8ab798ac3e3a796669265f29bd7e4098aa1f59992b4ed8032" gracePeriod=30 Sep 10 12:38:01.799343 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.799286 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f"] Sep 10 12:38:01.803752 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.803720 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:01.806469 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.806277 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-p5prj\"" Sep 10 12:38:01.816025 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.815914 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f"] Sep 10 12:38:01.877727 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.877689 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7dd0116-d7f3-42c3-8150-335c8f95a150-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-ppn2f\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:01.877982 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.877961 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ppn2f\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:01.878133 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.878119 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ppn2f\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:01.878243 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.878229 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ppn2f\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:01.878345 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.878331 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ppn2f\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:01.878446 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.878433 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr6mj\" (UniqueName: \"kubernetes.io/projected/e7dd0116-d7f3-42c3-8150-335c8f95a150-kube-api-access-gr6mj\") pod \"feast-simple-feast-setup-57576b8fcf-ppn2f\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:01.979351 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.979308 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7dd0116-d7f3-42c3-8150-335c8f95a150-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-ppn2f\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:01.979351 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.979353 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ppn2f\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:01.979606 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.979400 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ppn2f\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:01.979606 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.979424 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ppn2f\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:01.979606 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.979451 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ppn2f\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:01.979761 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.979477 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr6mj\" (UniqueName: \"kubernetes.io/projected/e7dd0116-d7f3-42c3-8150-335c8f95a150-kube-api-access-gr6mj\") pod \"feast-simple-feast-setup-57576b8fcf-ppn2f\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:01.979920 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.979893 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7dd0116-d7f3-42c3-8150-335c8f95a150-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-ppn2f\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:01.980108 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:38:01.979922 2583 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 10 12:38:01.980270 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:38:01.980091 2583 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 10 12:38:01.980382 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:38:01.980279 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-registry-tls podName:e7dd0116-d7f3-42c3-8150-335c8f95a150 nodeName:}" failed. No retries permitted until 2026-09-10 12:38:02.480239395 +0000 UTC m=+1210.485458847 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-registry-tls") pod "feast-simple-feast-setup-57576b8fcf-ppn2f" (UID: "e7dd0116-d7f3-42c3-8150-335c8f95a150") : secret "feast-simple-feast-setup-registry-tls" not found Sep 10 12:38:01.980382 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:38:01.980331 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-ui-tls podName:e7dd0116-d7f3-42c3-8150-335c8f95a150 nodeName:}" failed. No retries permitted until 2026-09-10 12:38:02.480296354 +0000 UTC m=+1210.485515795 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-ui-tls") pod "feast-simple-feast-setup-57576b8fcf-ppn2f" (UID: "e7dd0116-d7f3-42c3-8150-335c8f95a150") : secret "feast-simple-feast-setup-ui-tls" not found Sep 10 12:38:01.982337 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.982293 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ppn2f\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:01.982776 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.982753 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ppn2f\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:01.990899 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:01.990824 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr6mj\" (UniqueName: \"kubernetes.io/projected/e7dd0116-d7f3-42c3-8150-335c8f95a150-kube-api-access-gr6mj\") pod \"feast-simple-feast-setup-57576b8fcf-ppn2f\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:02.482863 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:02.482765 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ppn2f\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:02.482863 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:02.482860 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ppn2f\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:02.485319 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:02.485291 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ppn2f\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:02.485420 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:02.485299 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-ppn2f\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:02.720943 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:02.720916 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:02.894901 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:02.894869 2583 generic.go:356] "Generic (PLEG): container finished" podID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerID="8c48f7aae6251b9c6aa5e6adba96981b728a6a3c4d5f47082275bf398ad5763a" exitCode=0 Sep 10 12:38:02.894901 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:02.894897 2583 generic.go:356] "Generic (PLEG): container finished" podID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerID="227813a99cec654273f7ca68bd9e7e0ce9fd8a57a7d7831fd1fa86c675319908" exitCode=0 Sep 10 12:38:02.895075 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:02.894945 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" event={"ID":"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448","Type":"ContainerDied","Data":"8c48f7aae6251b9c6aa5e6adba96981b728a6a3c4d5f47082275bf398ad5763a"} Sep 10 12:38:02.895075 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:02.894987 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" event={"ID":"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448","Type":"ContainerDied","Data":"227813a99cec654273f7ca68bd9e7e0ce9fd8a57a7d7831fd1fa86c675319908"} Sep 10 12:38:02.994742 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:02.994716 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f"] Sep 10 12:38:02.995829 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:38:02.995804 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7dd0116_d7f3_42c3_8150_335c8f95a150.slice/crio-1330e78f2b8d0e1c0b1bbcf7d2269fa5fb7497265908912b69418092b9cbc182 WatchSource:0}: Error finding container 1330e78f2b8d0e1c0b1bbcf7d2269fa5fb7497265908912b69418092b9cbc182: Status 404 returned error can't find the container with id 1330e78f2b8d0e1c0b1bbcf7d2269fa5fb7497265908912b69418092b9cbc182 Sep 10 12:38:03.901276 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:03.901239 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" event={"ID":"e7dd0116-d7f3-42c3-8150-335c8f95a150","Type":"ContainerStarted","Data":"29acf929a9f642461f9d2b037c843d2248318f92ad72e6d7bce8f0eb020f4bf5"} Sep 10 12:38:03.901276 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:03.901281 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" event={"ID":"e7dd0116-d7f3-42c3-8150-335c8f95a150","Type":"ContainerStarted","Data":"1330e78f2b8d0e1c0b1bbcf7d2269fa5fb7497265908912b69418092b9cbc182"} Sep 10 12:38:05.651250 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:05.651202 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 10 12:38:07.917983 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:07.917952 2583 generic.go:356] "Generic (PLEG): container finished" podID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerID="29acf929a9f642461f9d2b037c843d2248318f92ad72e6d7bce8f0eb020f4bf5" exitCode=0 Sep 10 12:38:07.918336 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:07.918000 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" event={"ID":"e7dd0116-d7f3-42c3-8150-335c8f95a150","Type":"ContainerDied","Data":"29acf929a9f642461f9d2b037c843d2248318f92ad72e6d7bce8f0eb020f4bf5"} Sep 10 12:38:08.661365 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:08.661319 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused" Sep 10 12:38:08.923992 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:08.923905 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" event={"ID":"e7dd0116-d7f3-42c3-8150-335c8f95a150","Type":"ContainerStarted","Data":"74c6ab84625709944dba6b12e7ffa7d35de9856fdd5b6fa525628e6c12e90e0c"} Sep 10 12:38:13.946190 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:13.946118 2583 generic.go:356] "Generic (PLEG): container finished" podID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerID="74c6ab84625709944dba6b12e7ffa7d35de9856fdd5b6fa525628e6c12e90e0c" exitCode=0 Sep 10 12:38:13.946504 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:13.946189 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" event={"ID":"e7dd0116-d7f3-42c3-8150-335c8f95a150","Type":"ContainerDied","Data":"74c6ab84625709944dba6b12e7ffa7d35de9856fdd5b6fa525628e6c12e90e0c"} Sep 10 12:38:14.956639 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:14.956595 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" event={"ID":"e7dd0116-d7f3-42c3-8150-335c8f95a150","Type":"ContainerStarted","Data":"7262df39e01736bc27bf66d66d08d04bf95a716192e4dbb8da38cfb1790b85c0"} Sep 10 12:38:14.957146 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:14.956661 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" event={"ID":"e7dd0116-d7f3-42c3-8150-335c8f95a150","Type":"ContainerStarted","Data":"f5c3bef02da8b094b79e97309f9323a1063eba3b275fb5e0cbd5f67c7fa20cde"} Sep 10 12:38:14.957146 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:14.956682 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" event={"ID":"e7dd0116-d7f3-42c3-8150-335c8f95a150","Type":"ContainerStarted","Data":"dd6c53f7296c85f431eb7ccaee08bc257f0d8b75eafab821713b54f11b569321"} Sep 10 12:38:14.957146 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:14.956702 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" event={"ID":"e7dd0116-d7f3-42c3-8150-335c8f95a150","Type":"ContainerStarted","Data":"8b93f9ea0513b184de1ec96a93445f1c7c6034ab93968b7ef4398f9e0b9e90b7"} Sep 10 12:38:14.980554 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:14.980496 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" podStartSLOduration=13.980475479999999 podStartE2EDuration="13.98047548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:38:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:38:14.97894375 +0000 UTC m=+1222.984163208" watchObservedRunningTime="2026-09-10 12:38:14.98047548 +0000 UTC m=+1222.985694941" Sep 10 12:38:15.651615 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:15.651562 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 10 12:38:17.721905 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:17.721860 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:17.722388 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:17.721926 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:17.722388 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:17.721948 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:17.722388 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:17.721967 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:17.724248 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:17.724195 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.27:6571: connect: connection refused" Sep 10 12:38:17.724393 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:17.724217 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused" Sep 10 12:38:17.724393 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:17.724278 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused" Sep 10 12:38:17.724393 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:17.724221 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.27:8816: connect: connection refused" Sep 10 12:38:18.661115 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:18.661061 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused" Sep 10 12:38:20.721369 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:20.721324 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.27:6571: connect: connection refused" Sep 10 12:38:20.722046 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:20.721325 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused" Sep 10 12:38:20.722046 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:20.721325 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.27:8816: connect: connection refused" Sep 10 12:38:20.722254 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:20.721353 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused" Sep 10 12:38:22.722008 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:22.721968 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:22.722370 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:22.722025 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:22.722370 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:22.722037 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:22.722370 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:22.722052 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:23.722457 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:23.722421 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:23.723032 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:23.722690 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:23.723183 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:23.723155 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:23.723283 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:23.723238 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:23.723322 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:23.723294 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:23.728229 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:23.728207 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:24.001503 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:24.001422 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:24.009062 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:24.004742 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:38:25.650955 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:25.650910 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 10 12:38:25.651410 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:25.651043 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:38:28.661539 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:28.661500 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused" Sep 10 12:38:28.661924 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:28.661615 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:38:32.023567 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.023545 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:38:32.033323 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.033300 2583 generic.go:356] "Generic (PLEG): container finished" podID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerID="0d1a35fda49eb1a8ab798ac3e3a796669265f29bd7e4098aa1f59992b4ed8032" exitCode=137 Sep 10 12:38:32.033323 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.033320 2583 generic.go:356] "Generic (PLEG): container finished" podID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerID="8dc1f8643772a9fdaa1100085a9ed6998d7ea6d0665b9e403f5e32136208f8cf" exitCode=137 Sep 10 12:38:32.033461 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.033359 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" Sep 10 12:38:32.033461 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.033381 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" event={"ID":"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448","Type":"ContainerDied","Data":"0d1a35fda49eb1a8ab798ac3e3a796669265f29bd7e4098aa1f59992b4ed8032"} Sep 10 12:38:32.033461 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.033428 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" event={"ID":"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448","Type":"ContainerDied","Data":"8dc1f8643772a9fdaa1100085a9ed6998d7ea6d0665b9e403f5e32136208f8cf"} Sep 10 12:38:32.033461 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.033449 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj" event={"ID":"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448","Type":"ContainerDied","Data":"90997f83f595aa9cbc1bdd7b65fb3eb8014cbc32a07928b2a71acb6f3db878e5"} Sep 10 12:38:32.033584 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.033474 2583 scope.go:117] "RemoveContainer" containerID="8c48f7aae6251b9c6aa5e6adba96981b728a6a3c4d5f47082275bf398ad5763a" Sep 10 12:38:32.040917 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.040900 2583 scope.go:117] "RemoveContainer" containerID="0d1a35fda49eb1a8ab798ac3e3a796669265f29bd7e4098aa1f59992b4ed8032" Sep 10 12:38:32.048890 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.048873 2583 scope.go:117] "RemoveContainer" containerID="227813a99cec654273f7ca68bd9e7e0ce9fd8a57a7d7831fd1fa86c675319908" Sep 10 12:38:32.056747 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.056727 2583 scope.go:117] "RemoveContainer" containerID="8dc1f8643772a9fdaa1100085a9ed6998d7ea6d0665b9e403f5e32136208f8cf" Sep 10 12:38:32.063888 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.063868 2583 scope.go:117] "RemoveContainer" containerID="c85b8087ceeb1987456c8b7861c3e4bfc319580ded0e9193c2d34a12cd1dcb4a" Sep 10 12:38:32.070861 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.070827 2583 scope.go:117] "RemoveContainer" containerID="43fc517f0f46b4574180650672ab6bdf5c9af5fcec6c5ae9be0c0a56fada93e7" Sep 10 12:38:32.077374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.077358 2583 scope.go:117] "RemoveContainer" containerID="8c48f7aae6251b9c6aa5e6adba96981b728a6a3c4d5f47082275bf398ad5763a" Sep 10 12:38:32.077645 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:38:32.077621 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c48f7aae6251b9c6aa5e6adba96981b728a6a3c4d5f47082275bf398ad5763a\": container with ID starting with 8c48f7aae6251b9c6aa5e6adba96981b728a6a3c4d5f47082275bf398ad5763a not found: ID does not exist" containerID="8c48f7aae6251b9c6aa5e6adba96981b728a6a3c4d5f47082275bf398ad5763a" Sep 10 12:38:32.077693 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.077646 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c48f7aae6251b9c6aa5e6adba96981b728a6a3c4d5f47082275bf398ad5763a"} err="failed to get container status \"8c48f7aae6251b9c6aa5e6adba96981b728a6a3c4d5f47082275bf398ad5763a\": rpc error: code = NotFound desc = could not find container \"8c48f7aae6251b9c6aa5e6adba96981b728a6a3c4d5f47082275bf398ad5763a\": container with ID starting with 8c48f7aae6251b9c6aa5e6adba96981b728a6a3c4d5f47082275bf398ad5763a not found: ID does not exist" Sep 10 12:38:32.077693 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.077663 2583 scope.go:117] "RemoveContainer" containerID="0d1a35fda49eb1a8ab798ac3e3a796669265f29bd7e4098aa1f59992b4ed8032" Sep 10 12:38:32.077943 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:38:32.077912 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d1a35fda49eb1a8ab798ac3e3a796669265f29bd7e4098aa1f59992b4ed8032\": container with ID starting with 0d1a35fda49eb1a8ab798ac3e3a796669265f29bd7e4098aa1f59992b4ed8032 not found: ID does not exist" containerID="0d1a35fda49eb1a8ab798ac3e3a796669265f29bd7e4098aa1f59992b4ed8032" Sep 10 12:38:32.078032 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.077946 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d1a35fda49eb1a8ab798ac3e3a796669265f29bd7e4098aa1f59992b4ed8032"} err="failed to get container status \"0d1a35fda49eb1a8ab798ac3e3a796669265f29bd7e4098aa1f59992b4ed8032\": rpc error: code = NotFound desc = could not find container \"0d1a35fda49eb1a8ab798ac3e3a796669265f29bd7e4098aa1f59992b4ed8032\": container with ID starting with 0d1a35fda49eb1a8ab798ac3e3a796669265f29bd7e4098aa1f59992b4ed8032 not found: ID does not exist" Sep 10 12:38:32.078032 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.077961 2583 scope.go:117] "RemoveContainer" containerID="227813a99cec654273f7ca68bd9e7e0ce9fd8a57a7d7831fd1fa86c675319908" Sep 10 12:38:32.078173 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:38:32.078158 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"227813a99cec654273f7ca68bd9e7e0ce9fd8a57a7d7831fd1fa86c675319908\": container with ID starting with 227813a99cec654273f7ca68bd9e7e0ce9fd8a57a7d7831fd1fa86c675319908 not found: ID does not exist" containerID="227813a99cec654273f7ca68bd9e7e0ce9fd8a57a7d7831fd1fa86c675319908" Sep 10 12:38:32.078212 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.078176 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"227813a99cec654273f7ca68bd9e7e0ce9fd8a57a7d7831fd1fa86c675319908"} err="failed to get container status \"227813a99cec654273f7ca68bd9e7e0ce9fd8a57a7d7831fd1fa86c675319908\": rpc error: code = NotFound desc = could not find container \"227813a99cec654273f7ca68bd9e7e0ce9fd8a57a7d7831fd1fa86c675319908\": container with ID starting with 227813a99cec654273f7ca68bd9e7e0ce9fd8a57a7d7831fd1fa86c675319908 not found: ID does not exist" Sep 10 12:38:32.078212 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.078189 2583 scope.go:117] "RemoveContainer" containerID="8dc1f8643772a9fdaa1100085a9ed6998d7ea6d0665b9e403f5e32136208f8cf" Sep 10 12:38:32.078402 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:38:32.078377 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dc1f8643772a9fdaa1100085a9ed6998d7ea6d0665b9e403f5e32136208f8cf\": container with ID starting with 8dc1f8643772a9fdaa1100085a9ed6998d7ea6d0665b9e403f5e32136208f8cf not found: ID does not exist" containerID="8dc1f8643772a9fdaa1100085a9ed6998d7ea6d0665b9e403f5e32136208f8cf" Sep 10 12:38:32.078448 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.078408 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc1f8643772a9fdaa1100085a9ed6998d7ea6d0665b9e403f5e32136208f8cf"} err="failed to get container status \"8dc1f8643772a9fdaa1100085a9ed6998d7ea6d0665b9e403f5e32136208f8cf\": rpc error: code = NotFound desc = could not find container \"8dc1f8643772a9fdaa1100085a9ed6998d7ea6d0665b9e403f5e32136208f8cf\": container with ID starting with 8dc1f8643772a9fdaa1100085a9ed6998d7ea6d0665b9e403f5e32136208f8cf not found: ID does not exist" Sep 10 12:38:32.078448 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.078423 2583 scope.go:117] "RemoveContainer" containerID="c85b8087ceeb1987456c8b7861c3e4bfc319580ded0e9193c2d34a12cd1dcb4a" Sep 10 12:38:32.078665 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:38:32.078650 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c85b8087ceeb1987456c8b7861c3e4bfc319580ded0e9193c2d34a12cd1dcb4a\": container with ID starting with c85b8087ceeb1987456c8b7861c3e4bfc319580ded0e9193c2d34a12cd1dcb4a not found: ID does not exist" containerID="c85b8087ceeb1987456c8b7861c3e4bfc319580ded0e9193c2d34a12cd1dcb4a" Sep 10 12:38:32.078709 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.078666 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c85b8087ceeb1987456c8b7861c3e4bfc319580ded0e9193c2d34a12cd1dcb4a"} err="failed to get container status \"c85b8087ceeb1987456c8b7861c3e4bfc319580ded0e9193c2d34a12cd1dcb4a\": rpc error: code = NotFound desc = could not find container \"c85b8087ceeb1987456c8b7861c3e4bfc319580ded0e9193c2d34a12cd1dcb4a\": container with ID starting with c85b8087ceeb1987456c8b7861c3e4bfc319580ded0e9193c2d34a12cd1dcb4a not found: ID does not exist" Sep 10 12:38:32.078709 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.078678 2583 scope.go:117] "RemoveContainer" containerID="43fc517f0f46b4574180650672ab6bdf5c9af5fcec6c5ae9be0c0a56fada93e7" Sep 10 12:38:32.078945 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:38:32.078925 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43fc517f0f46b4574180650672ab6bdf5c9af5fcec6c5ae9be0c0a56fada93e7\": container with ID starting with 43fc517f0f46b4574180650672ab6bdf5c9af5fcec6c5ae9be0c0a56fada93e7 not found: ID does not exist" containerID="43fc517f0f46b4574180650672ab6bdf5c9af5fcec6c5ae9be0c0a56fada93e7" Sep 10 12:38:32.079012 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.078948 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43fc517f0f46b4574180650672ab6bdf5c9af5fcec6c5ae9be0c0a56fada93e7"} err="failed to get container status \"43fc517f0f46b4574180650672ab6bdf5c9af5fcec6c5ae9be0c0a56fada93e7\": rpc error: code = NotFound desc = could not find container \"43fc517f0f46b4574180650672ab6bdf5c9af5fcec6c5ae9be0c0a56fada93e7\": container with ID starting with 43fc517f0f46b4574180650672ab6bdf5c9af5fcec6c5ae9be0c0a56fada93e7 not found: ID does not exist" Sep 10 12:38:32.079012 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.078963 2583 scope.go:117] "RemoveContainer" containerID="8c48f7aae6251b9c6aa5e6adba96981b728a6a3c4d5f47082275bf398ad5763a" Sep 10 12:38:32.079187 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.079172 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c48f7aae6251b9c6aa5e6adba96981b728a6a3c4d5f47082275bf398ad5763a"} err="failed to get container status \"8c48f7aae6251b9c6aa5e6adba96981b728a6a3c4d5f47082275bf398ad5763a\": rpc error: code = NotFound desc = could not find container \"8c48f7aae6251b9c6aa5e6adba96981b728a6a3c4d5f47082275bf398ad5763a\": container with ID starting with 8c48f7aae6251b9c6aa5e6adba96981b728a6a3c4d5f47082275bf398ad5763a not found: ID does not exist" Sep 10 12:38:32.079229 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.079186 2583 scope.go:117] "RemoveContainer" containerID="0d1a35fda49eb1a8ab798ac3e3a796669265f29bd7e4098aa1f59992b4ed8032" Sep 10 12:38:32.079420 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.079404 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d1a35fda49eb1a8ab798ac3e3a796669265f29bd7e4098aa1f59992b4ed8032"} err="failed to get container status \"0d1a35fda49eb1a8ab798ac3e3a796669265f29bd7e4098aa1f59992b4ed8032\": rpc error: code = NotFound desc = could not find container \"0d1a35fda49eb1a8ab798ac3e3a796669265f29bd7e4098aa1f59992b4ed8032\": container with ID starting with 0d1a35fda49eb1a8ab798ac3e3a796669265f29bd7e4098aa1f59992b4ed8032 not found: ID does not exist" Sep 10 12:38:32.079461 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.079419 2583 scope.go:117] "RemoveContainer" containerID="227813a99cec654273f7ca68bd9e7e0ce9fd8a57a7d7831fd1fa86c675319908" Sep 10 12:38:32.079638 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.079619 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"227813a99cec654273f7ca68bd9e7e0ce9fd8a57a7d7831fd1fa86c675319908"} err="failed to get container status \"227813a99cec654273f7ca68bd9e7e0ce9fd8a57a7d7831fd1fa86c675319908\": rpc error: code = NotFound desc = could not find container \"227813a99cec654273f7ca68bd9e7e0ce9fd8a57a7d7831fd1fa86c675319908\": container with ID starting with 227813a99cec654273f7ca68bd9e7e0ce9fd8a57a7d7831fd1fa86c675319908 not found: ID does not exist" Sep 10 12:38:32.079678 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.079639 2583 scope.go:117] "RemoveContainer" containerID="8dc1f8643772a9fdaa1100085a9ed6998d7ea6d0665b9e403f5e32136208f8cf" Sep 10 12:38:32.079892 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.079874 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc1f8643772a9fdaa1100085a9ed6998d7ea6d0665b9e403f5e32136208f8cf"} err="failed to get container status \"8dc1f8643772a9fdaa1100085a9ed6998d7ea6d0665b9e403f5e32136208f8cf\": rpc error: code = NotFound desc = could not find container \"8dc1f8643772a9fdaa1100085a9ed6998d7ea6d0665b9e403f5e32136208f8cf\": container with ID starting with 8dc1f8643772a9fdaa1100085a9ed6998d7ea6d0665b9e403f5e32136208f8cf not found: ID does not exist" Sep 10 12:38:32.079958 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.079894 2583 scope.go:117] "RemoveContainer" containerID="c85b8087ceeb1987456c8b7861c3e4bfc319580ded0e9193c2d34a12cd1dcb4a" Sep 10 12:38:32.080129 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.080113 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c85b8087ceeb1987456c8b7861c3e4bfc319580ded0e9193c2d34a12cd1dcb4a"} err="failed to get container status \"c85b8087ceeb1987456c8b7861c3e4bfc319580ded0e9193c2d34a12cd1dcb4a\": rpc error: code = NotFound desc = could not find container \"c85b8087ceeb1987456c8b7861c3e4bfc319580ded0e9193c2d34a12cd1dcb4a\": container with ID starting with c85b8087ceeb1987456c8b7861c3e4bfc319580ded0e9193c2d34a12cd1dcb4a not found: ID does not exist" Sep 10 12:38:32.080129 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.080129 2583 scope.go:117] "RemoveContainer" containerID="43fc517f0f46b4574180650672ab6bdf5c9af5fcec6c5ae9be0c0a56fada93e7" Sep 10 12:38:32.080342 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.080325 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43fc517f0f46b4574180650672ab6bdf5c9af5fcec6c5ae9be0c0a56fada93e7"} err="failed to get container status \"43fc517f0f46b4574180650672ab6bdf5c9af5fcec6c5ae9be0c0a56fada93e7\": rpc error: code = NotFound desc = could not find container \"43fc517f0f46b4574180650672ab6bdf5c9af5fcec6c5ae9be0c0a56fada93e7\": container with ID starting with 43fc517f0f46b4574180650672ab6bdf5c9af5fcec6c5ae9be0c0a56fada93e7 not found: ID does not exist" Sep 10 12:38:32.120977 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.120955 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-registry-tls\") pod \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " Sep 10 12:38:32.121051 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.120999 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-feast-data\") pod \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " Sep 10 12:38:32.121051 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.121036 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9cb2\" (UniqueName: \"kubernetes.io/projected/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-kube-api-access-h9cb2\") pod \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " Sep 10 12:38:32.121167 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.121150 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-offline-tls\") pod \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " Sep 10 12:38:32.121219 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.121201 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-ui-tls\") pod \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " Sep 10 12:38:32.121579 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.121305 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-online-tls\") pod \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\" (UID: \"9e0ea2aa-87ac-4fb5-85d4-ad26382aa448\") " Sep 10 12:38:32.121579 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.121555 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-feast-data" (OuterVolumeSpecName: "feast-data") pod "9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" (UID: "9e0ea2aa-87ac-4fb5-85d4-ad26382aa448"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:38:32.123297 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.123261 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" (UID: "9e0ea2aa-87ac-4fb5-85d4-ad26382aa448"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:38:32.123297 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.123280 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" (UID: "9e0ea2aa-87ac-4fb5-85d4-ad26382aa448"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:38:32.123433 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.123289 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-kube-api-access-h9cb2" (OuterVolumeSpecName: "kube-api-access-h9cb2") pod "9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" (UID: "9e0ea2aa-87ac-4fb5-85d4-ad26382aa448"). InnerVolumeSpecName "kube-api-access-h9cb2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:38:32.123433 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.123411 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" (UID: "9e0ea2aa-87ac-4fb5-85d4-ad26382aa448"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:38:32.123502 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.123467 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-online-tls" (OuterVolumeSpecName: "online-tls") pod "9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" (UID: "9e0ea2aa-87ac-4fb5-85d4-ad26382aa448"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:38:32.222583 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.222535 2583 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-feast-data\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:38:32.222583 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.222554 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9cb2\" (UniqueName: \"kubernetes.io/projected/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-kube-api-access-h9cb2\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:38:32.222583 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.222565 2583 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-offline-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:38:32.222583 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.222574 2583 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-ui-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:38:32.222583 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.222582 2583 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-online-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:38:32.222770 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.222589 2583 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448-registry-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:38:32.356613 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.356584 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj"] Sep 10 12:38:32.361437 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.361408 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-vphrj"] Sep 10 12:38:32.469153 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:38:32.469115 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" path="/var/lib/kubelet/pods/9e0ea2aa-87ac-4fb5-85d4-ad26382aa448/volumes" Sep 10 12:39:26.750004 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.749964 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c"] Sep 10 12:39:26.750558 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.750337 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="feast-apply" Sep 10 12:39:26.750558 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.750352 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="feast-apply" Sep 10 12:39:26.750558 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.750370 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="online" Sep 10 12:39:26.750558 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.750380 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="online" Sep 10 12:39:26.750558 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.750398 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="offline" Sep 10 12:39:26.750558 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.750407 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="offline" Sep 10 12:39:26.750558 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.750417 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="registry" Sep 10 12:39:26.750558 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.750425 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="registry" Sep 10 12:39:26.750558 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.750440 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="feast-init" Sep 10 12:39:26.750558 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.750447 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="feast-init" Sep 10 12:39:26.750558 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.750457 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="ui" Sep 10 12:39:26.750558 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.750465 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="ui" Sep 10 12:39:26.750558 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.750527 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="registry" Sep 10 12:39:26.750558 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.750537 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="offline" Sep 10 12:39:26.750558 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.750547 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="online" Sep 10 12:39:26.750558 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.750555 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e0ea2aa-87ac-4fb5-85d4-ad26382aa448" containerName="ui" Sep 10 12:39:26.754174 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.754150 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:26.756570 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.756547 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 10 12:39:26.756697 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.756534 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 10 12:39:26.756795 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.756774 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 10 12:39:26.757317 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.757295 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 10 12:39:26.757447 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.757333 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-zlkgb\"" Sep 10 12:39:26.757542 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.757348 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 10 12:39:26.757542 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.757354 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 10 12:39:26.767913 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.767893 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c"] Sep 10 12:39:26.910361 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.910319 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9a4b3055-a26e-4480-bea4-54f88452d6c5-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wvt5c\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:26.910562 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.910366 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9a4b3055-a26e-4480-bea4-54f88452d6c5-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wvt5c\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:26.910562 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.910398 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztdfm\" (UniqueName: \"kubernetes.io/projected/9a4b3055-a26e-4480-bea4-54f88452d6c5-kube-api-access-ztdfm\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wvt5c\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:26.910562 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.910454 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9a4b3055-a26e-4480-bea4-54f88452d6c5-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wvt5c\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:26.910562 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.910489 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9a4b3055-a26e-4480-bea4-54f88452d6c5-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wvt5c\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:26.910562 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:26.910540 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9a4b3055-a26e-4480-bea4-54f88452d6c5-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wvt5c\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:27.011662 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:27.011573 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9a4b3055-a26e-4480-bea4-54f88452d6c5-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wvt5c\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:27.011662 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:27.011613 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9a4b3055-a26e-4480-bea4-54f88452d6c5-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wvt5c\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:27.011662 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:27.011643 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztdfm\" (UniqueName: \"kubernetes.io/projected/9a4b3055-a26e-4480-bea4-54f88452d6c5-kube-api-access-ztdfm\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wvt5c\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:27.012005 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:27.011675 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9a4b3055-a26e-4480-bea4-54f88452d6c5-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wvt5c\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:27.012005 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:27.011707 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9a4b3055-a26e-4480-bea4-54f88452d6c5-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wvt5c\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:27.012005 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:39:27.011740 2583 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Sep 10 12:39:27.012005 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:27.011760 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9a4b3055-a26e-4480-bea4-54f88452d6c5-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wvt5c\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:27.012005 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:39:27.011825 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a4b3055-a26e-4480-bea4-54f88452d6c5-online-tls podName:9a4b3055-a26e-4480-bea4-54f88452d6c5 nodeName:}" failed. No retries permitted until 2026-09-10 12:39:27.511802834 +0000 UTC m=+1295.517022280 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/9a4b3055-a26e-4480-bea4-54f88452d6c5-online-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" (UID: "9a4b3055-a26e-4480-bea4-54f88452d6c5") : secret "feast-simple-feast-remote-setup-online-tls" not found Sep 10 12:39:27.012267 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:27.012220 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9a4b3055-a26e-4480-bea4-54f88452d6c5-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wvt5c\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:27.012385 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:39:27.012365 2583 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 10 12:39:27.012452 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:39:27.012440 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a4b3055-a26e-4480-bea4-54f88452d6c5-ui-tls podName:9a4b3055-a26e-4480-bea4-54f88452d6c5 nodeName:}" failed. No retries permitted until 2026-09-10 12:39:27.512423703 +0000 UTC m=+1295.517643157 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/9a4b3055-a26e-4480-bea4-54f88452d6c5-ui-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" (UID: "9a4b3055-a26e-4480-bea4-54f88452d6c5") : secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 10 12:39:27.012521 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:27.012499 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9a4b3055-a26e-4480-bea4-54f88452d6c5-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wvt5c\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:27.014182 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:27.014161 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9a4b3055-a26e-4480-bea4-54f88452d6c5-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wvt5c\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:27.021194 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:27.021172 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztdfm\" (UniqueName: \"kubernetes.io/projected/9a4b3055-a26e-4480-bea4-54f88452d6c5-kube-api-access-ztdfm\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wvt5c\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:27.515400 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:27.515359 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9a4b3055-a26e-4480-bea4-54f88452d6c5-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wvt5c\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:27.515400 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:27.515404 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9a4b3055-a26e-4480-bea4-54f88452d6c5-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wvt5c\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:27.517785 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:27.517759 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9a4b3055-a26e-4480-bea4-54f88452d6c5-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wvt5c\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:27.517921 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:27.517789 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9a4b3055-a26e-4480-bea4-54f88452d6c5-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-wvt5c\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:27.663816 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:27.663778 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:27.938527 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:27.938502 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c"] Sep 10 12:39:27.940155 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:39:27.940131 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a4b3055_a26e_4480_bea4_54f88452d6c5.slice/crio-5772af11d94533f4587011174c4111acda7a95bde1d17167c86870df968c5de0 WatchSource:0}: Error finding container 5772af11d94533f4587011174c4111acda7a95bde1d17167c86870df968c5de0: Status 404 returned error can't find the container with id 5772af11d94533f4587011174c4111acda7a95bde1d17167c86870df968c5de0 Sep 10 12:39:28.239776 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:28.239672 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" event={"ID":"9a4b3055-a26e-4480-bea4-54f88452d6c5","Type":"ContainerStarted","Data":"f1a43254b80d8d1772c1abdc82bc530159c2d0007c5c51b3d9c255bc10e2c6e8"} Sep 10 12:39:28.239776 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:28.239750 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" event={"ID":"9a4b3055-a26e-4480-bea4-54f88452d6c5","Type":"ContainerStarted","Data":"5772af11d94533f4587011174c4111acda7a95bde1d17167c86870df968c5de0"} Sep 10 12:39:33.268570 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:33.268536 2583 generic.go:356] "Generic (PLEG): container finished" podID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerID="f1a43254b80d8d1772c1abdc82bc530159c2d0007c5c51b3d9c255bc10e2c6e8" exitCode=0 Sep 10 12:39:33.268978 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:33.268617 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" event={"ID":"9a4b3055-a26e-4480-bea4-54f88452d6c5","Type":"ContainerDied","Data":"f1a43254b80d8d1772c1abdc82bc530159c2d0007c5c51b3d9c255bc10e2c6e8"} Sep 10 12:39:34.274632 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:34.274599 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" event={"ID":"9a4b3055-a26e-4480-bea4-54f88452d6c5","Type":"ContainerStarted","Data":"2febdd0d631f3fcc2dc8d3162a48a41f94c9636e6a7fe830ff13785bd0cf81f8"} Sep 10 12:39:55.355140 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:55.355104 2583 generic.go:356] "Generic (PLEG): container finished" podID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerID="2febdd0d631f3fcc2dc8d3162a48a41f94c9636e6a7fe830ff13785bd0cf81f8" exitCode=0 Sep 10 12:39:55.355600 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:55.355184 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" event={"ID":"9a4b3055-a26e-4480-bea4-54f88452d6c5","Type":"ContainerDied","Data":"2febdd0d631f3fcc2dc8d3162a48a41f94c9636e6a7fe830ff13785bd0cf81f8"} Sep 10 12:39:56.362951 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:56.362912 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" event={"ID":"9a4b3055-a26e-4480-bea4-54f88452d6c5","Type":"ContainerStarted","Data":"f83a05e2304ebb73a653d6529ba968416c2bdd172d65a70099998c7dcf74e6ae"} Sep 10 12:39:56.363436 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:56.362962 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" event={"ID":"9a4b3055-a26e-4480-bea4-54f88452d6c5","Type":"ContainerStarted","Data":"c6d68580d16f823043ceecee630e03e088246e1b6f5d04502ac52bfbeb5b5691"} Sep 10 12:39:56.363436 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:56.362983 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" event={"ID":"9a4b3055-a26e-4480-bea4-54f88452d6c5","Type":"ContainerStarted","Data":"4a7c9ecc99385e55b5ba5f917667f6bed3bd71da1edd7dae4eec36ee1937478e"} Sep 10 12:39:56.388164 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:56.388089 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" podStartSLOduration=30.3880673 podStartE2EDuration="30.3880673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:39:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:39:56.385590777 +0000 UTC m=+1324.390810238" watchObservedRunningTime="2026-09-10 12:39:56.3880673 +0000 UTC m=+1324.393286763" Sep 10 12:39:57.664607 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:57.664571 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:57.665106 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:57.664629 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:57.665106 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:57.664649 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:57.665106 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:57.664667 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:57.665106 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:57.664684 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:57.665106 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:57.664700 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:39:57.666491 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:57.666451 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.28:8443: connect: connection refused" Sep 10 12:39:57.666491 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:57.666477 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.28:8816: connect: connection refused" Sep 10 12:39:57.666684 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:39:57.666500 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused" Sep 10 12:40:00.665012 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:40:00.664957 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.28:8443: connect: connection refused" Sep 10 12:40:00.665401 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:40:00.665024 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused" Sep 10 12:40:00.665401 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:40:00.664962 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.28:8816: connect: connection refused" Sep 10 12:40:03.665072 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:40:03.665041 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:40:03.665603 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:40:03.665238 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:40:03.665985 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:40:03.665962 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:40:03.666251 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:40:03.666235 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:40:04.664464 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:40:04.664423 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Sep 10 12:40:07.295107 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:40:07.295075 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:40:07.925016 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:40:07.924978 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:40:28.668049 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:40:28.667954 2583 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 10 12:41:09.300031 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:09.299993 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c"] Sep 10 12:41:09.300547 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:09.300326 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="online" containerID="cri-o://4a7c9ecc99385e55b5ba5f917667f6bed3bd71da1edd7dae4eec36ee1937478e" gracePeriod=30 Sep 10 12:41:09.300547 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:09.300388 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="ui" containerID="cri-o://f83a05e2304ebb73a653d6529ba968416c2bdd172d65a70099998c7dcf74e6ae" gracePeriod=30 Sep 10 12:41:09.300547 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:09.300507 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="offline" containerID="cri-o://c6d68580d16f823043ceecee630e03e088246e1b6f5d04502ac52bfbeb5b5691" gracePeriod=30 Sep 10 12:41:09.450946 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:09.450908 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f"] Sep 10 12:41:09.451472 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:09.451208 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="registry" containerID="cri-o://8b93f9ea0513b184de1ec96a93445f1c7c6034ab93968b7ef4398f9e0b9e90b7" gracePeriod=30 Sep 10 12:41:09.451472 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:09.451319 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="ui" containerID="cri-o://7262df39e01736bc27bf66d66d08d04bf95a716192e4dbb8da38cfb1790b85c0" gracePeriod=30 Sep 10 12:41:09.451472 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:09.451326 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="offline" containerID="cri-o://f5c3bef02da8b094b79e97309f9323a1063eba3b275fb5e0cbd5f67c7fa20cde" gracePeriod=30 Sep 10 12:41:09.451472 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:09.451387 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="online" containerID="cri-o://dd6c53f7296c85f431eb7ccaee08bc257f0d8b75eafab821713b54f11b569321" gracePeriod=30 Sep 10 12:41:10.657652 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:10.657617 2583 generic.go:356] "Generic (PLEG): container finished" podID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerID="f83a05e2304ebb73a653d6529ba968416c2bdd172d65a70099998c7dcf74e6ae" exitCode=0 Sep 10 12:41:10.657652 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:10.657645 2583 generic.go:356] "Generic (PLEG): container finished" podID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerID="4a7c9ecc99385e55b5ba5f917667f6bed3bd71da1edd7dae4eec36ee1937478e" exitCode=0 Sep 10 12:41:10.658108 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:10.657696 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" event={"ID":"9a4b3055-a26e-4480-bea4-54f88452d6c5","Type":"ContainerDied","Data":"f83a05e2304ebb73a653d6529ba968416c2bdd172d65a70099998c7dcf74e6ae"} Sep 10 12:41:10.658108 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:10.657735 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" event={"ID":"9a4b3055-a26e-4480-bea4-54f88452d6c5","Type":"ContainerDied","Data":"4a7c9ecc99385e55b5ba5f917667f6bed3bd71da1edd7dae4eec36ee1937478e"} Sep 10 12:41:10.660562 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:10.660541 2583 generic.go:356] "Generic (PLEG): container finished" podID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerID="7262df39e01736bc27bf66d66d08d04bf95a716192e4dbb8da38cfb1790b85c0" exitCode=0 Sep 10 12:41:10.660562 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:10.660558 2583 generic.go:356] "Generic (PLEG): container finished" podID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerID="dd6c53f7296c85f431eb7ccaee08bc257f0d8b75eafab821713b54f11b569321" exitCode=0 Sep 10 12:41:10.660702 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:10.660578 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" event={"ID":"e7dd0116-d7f3-42c3-8150-335c8f95a150","Type":"ContainerDied","Data":"7262df39e01736bc27bf66d66d08d04bf95a716192e4dbb8da38cfb1790b85c0"} Sep 10 12:41:10.660702 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:10.660604 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" event={"ID":"e7dd0116-d7f3-42c3-8150-335c8f95a150","Type":"ContainerDied","Data":"dd6c53f7296c85f431eb7ccaee08bc257f0d8b75eafab821713b54f11b569321"} Sep 10 12:41:13.666204 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:13.666155 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.28:8443: connect: connection refused" Sep 10 12:41:13.722954 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:13.722910 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused" Sep 10 12:41:14.001498 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:14.001396 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused" Sep 10 12:41:17.296389 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:17.296339 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused" Sep 10 12:41:23.666205 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:23.666149 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.28:8443: connect: connection refused" Sep 10 12:41:23.723979 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:23.723938 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused" Sep 10 12:41:24.002077 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:24.001961 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused" Sep 10 12:41:27.295772 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:27.295722 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused" Sep 10 12:41:33.666097 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:33.666049 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.28:8443: connect: connection refused" Sep 10 12:41:33.666515 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:33.666167 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:41:33.723254 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:33.723207 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused" Sep 10 12:41:33.723446 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:33.723360 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:41:34.002114 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:34.002009 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused" Sep 10 12:41:34.002261 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:34.002138 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:41:37.296588 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:37.296544 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused" Sep 10 12:41:37.296973 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:37.296679 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:41:39.768224 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:39.768141 2583 generic.go:356] "Generic (PLEG): container finished" podID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerID="f5c3bef02da8b094b79e97309f9323a1063eba3b275fb5e0cbd5f67c7fa20cde" exitCode=137 Sep 10 12:41:39.768224 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:39.768168 2583 generic.go:356] "Generic (PLEG): container finished" podID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerID="8b93f9ea0513b184de1ec96a93445f1c7c6034ab93968b7ef4398f9e0b9e90b7" exitCode=137 Sep 10 12:41:39.768668 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:39.768215 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" event={"ID":"e7dd0116-d7f3-42c3-8150-335c8f95a150","Type":"ContainerDied","Data":"f5c3bef02da8b094b79e97309f9323a1063eba3b275fb5e0cbd5f67c7fa20cde"} Sep 10 12:41:39.768668 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:39.768266 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" event={"ID":"e7dd0116-d7f3-42c3-8150-335c8f95a150","Type":"ContainerDied","Data":"8b93f9ea0513b184de1ec96a93445f1c7c6034ab93968b7ef4398f9e0b9e90b7"} Sep 10 12:41:39.770834 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:39.770813 2583 generic.go:356] "Generic (PLEG): container finished" podID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerID="c6d68580d16f823043ceecee630e03e088246e1b6f5d04502ac52bfbeb5b5691" exitCode=137 Sep 10 12:41:39.770963 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:39.770889 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" event={"ID":"9a4b3055-a26e-4480-bea4-54f88452d6c5","Type":"ContainerDied","Data":"c6d68580d16f823043ceecee630e03e088246e1b6f5d04502ac52bfbeb5b5691"} Sep 10 12:41:39.949257 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:39.949233 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:41:40.066060 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.066029 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9a4b3055-a26e-4480-bea4-54f88452d6c5-registry-tls\") pod \"9a4b3055-a26e-4480-bea4-54f88452d6c5\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " Sep 10 12:41:40.066230 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.066092 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9a4b3055-a26e-4480-bea4-54f88452d6c5-online-tls\") pod \"9a4b3055-a26e-4480-bea4-54f88452d6c5\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " Sep 10 12:41:40.066230 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.066138 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ztdfm\" (UniqueName: \"kubernetes.io/projected/9a4b3055-a26e-4480-bea4-54f88452d6c5-kube-api-access-ztdfm\") pod \"9a4b3055-a26e-4480-bea4-54f88452d6c5\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " Sep 10 12:41:40.066230 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.066174 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9a4b3055-a26e-4480-bea4-54f88452d6c5-offline-tls\") pod \"9a4b3055-a26e-4480-bea4-54f88452d6c5\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " Sep 10 12:41:40.066230 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.066208 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9a4b3055-a26e-4480-bea4-54f88452d6c5-feast-data\") pod \"9a4b3055-a26e-4480-bea4-54f88452d6c5\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " Sep 10 12:41:40.066408 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.066241 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9a4b3055-a26e-4480-bea4-54f88452d6c5-ui-tls\") pod \"9a4b3055-a26e-4480-bea4-54f88452d6c5\" (UID: \"9a4b3055-a26e-4480-bea4-54f88452d6c5\") " Sep 10 12:41:40.066464 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.066424 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a4b3055-a26e-4480-bea4-54f88452d6c5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9a4b3055-a26e-4480-bea4-54f88452d6c5" (UID: "9a4b3055-a26e-4480-bea4-54f88452d6c5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:41:40.066873 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.066827 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a4b3055-a26e-4480-bea4-54f88452d6c5-feast-data" (OuterVolumeSpecName: "feast-data") pod "9a4b3055-a26e-4480-bea4-54f88452d6c5" (UID: "9a4b3055-a26e-4480-bea4-54f88452d6c5"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:41:40.068558 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.068530 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a4b3055-a26e-4480-bea4-54f88452d6c5-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "9a4b3055-a26e-4480-bea4-54f88452d6c5" (UID: "9a4b3055-a26e-4480-bea4-54f88452d6c5"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:41:40.068740 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.068721 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a4b3055-a26e-4480-bea4-54f88452d6c5-online-tls" (OuterVolumeSpecName: "online-tls") pod "9a4b3055-a26e-4480-bea4-54f88452d6c5" (UID: "9a4b3055-a26e-4480-bea4-54f88452d6c5"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:41:40.068799 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.068733 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a4b3055-a26e-4480-bea4-54f88452d6c5-kube-api-access-ztdfm" (OuterVolumeSpecName: "kube-api-access-ztdfm") pod "9a4b3055-a26e-4480-bea4-54f88452d6c5" (UID: "9a4b3055-a26e-4480-bea4-54f88452d6c5"). InnerVolumeSpecName "kube-api-access-ztdfm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:41:40.068799 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.068759 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a4b3055-a26e-4480-bea4-54f88452d6c5-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "9a4b3055-a26e-4480-bea4-54f88452d6c5" (UID: "9a4b3055-a26e-4480-bea4-54f88452d6c5"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:41:40.105980 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.105956 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:41:40.167546 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.167509 2583 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9a4b3055-a26e-4480-bea4-54f88452d6c5-registry-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:41:40.167546 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.167541 2583 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9a4b3055-a26e-4480-bea4-54f88452d6c5-online-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:41:40.167546 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.167550 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztdfm\" (UniqueName: \"kubernetes.io/projected/9a4b3055-a26e-4480-bea4-54f88452d6c5-kube-api-access-ztdfm\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:41:40.167783 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.167559 2583 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9a4b3055-a26e-4480-bea4-54f88452d6c5-offline-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:41:40.167783 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.167568 2583 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9a4b3055-a26e-4480-bea4-54f88452d6c5-feast-data\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:41:40.167783 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.167575 2583 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9a4b3055-a26e-4480-bea4-54f88452d6c5-ui-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:41:40.267996 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.267962 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-registry-tls\") pod \"e7dd0116-d7f3-42c3-8150-335c8f95a150\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " Sep 10 12:41:40.268168 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.268013 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gr6mj\" (UniqueName: \"kubernetes.io/projected/e7dd0116-d7f3-42c3-8150-335c8f95a150-kube-api-access-gr6mj\") pod \"e7dd0116-d7f3-42c3-8150-335c8f95a150\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " Sep 10 12:41:40.268168 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.268060 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7dd0116-d7f3-42c3-8150-335c8f95a150-feast-data\") pod \"e7dd0116-d7f3-42c3-8150-335c8f95a150\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " Sep 10 12:41:40.268168 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.268092 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-ui-tls\") pod \"e7dd0116-d7f3-42c3-8150-335c8f95a150\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " Sep 10 12:41:40.268168 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.268153 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-offline-tls\") pod \"e7dd0116-d7f3-42c3-8150-335c8f95a150\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " Sep 10 12:41:40.268377 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.268178 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-online-tls\") pod \"e7dd0116-d7f3-42c3-8150-335c8f95a150\" (UID: \"e7dd0116-d7f3-42c3-8150-335c8f95a150\") " Sep 10 12:41:40.268639 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.268608 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7dd0116-d7f3-42c3-8150-335c8f95a150-feast-data" (OuterVolumeSpecName: "feast-data") pod "e7dd0116-d7f3-42c3-8150-335c8f95a150" (UID: "e7dd0116-d7f3-42c3-8150-335c8f95a150"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:41:40.270234 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.270209 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-online-tls" (OuterVolumeSpecName: "online-tls") pod "e7dd0116-d7f3-42c3-8150-335c8f95a150" (UID: "e7dd0116-d7f3-42c3-8150-335c8f95a150"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:41:40.270368 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.270290 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e7dd0116-d7f3-42c3-8150-335c8f95a150" (UID: "e7dd0116-d7f3-42c3-8150-335c8f95a150"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:41:40.270368 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.270297 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7dd0116-d7f3-42c3-8150-335c8f95a150-kube-api-access-gr6mj" (OuterVolumeSpecName: "kube-api-access-gr6mj") pod "e7dd0116-d7f3-42c3-8150-335c8f95a150" (UID: "e7dd0116-d7f3-42c3-8150-335c8f95a150"). InnerVolumeSpecName "kube-api-access-gr6mj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:41:40.270573 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.270551 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "e7dd0116-d7f3-42c3-8150-335c8f95a150" (UID: "e7dd0116-d7f3-42c3-8150-335c8f95a150"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:41:40.270653 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.270570 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "e7dd0116-d7f3-42c3-8150-335c8f95a150" (UID: "e7dd0116-d7f3-42c3-8150-335c8f95a150"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:41:40.369382 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.369346 2583 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7dd0116-d7f3-42c3-8150-335c8f95a150-feast-data\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:41:40.369382 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.369376 2583 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-ui-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:41:40.369382 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.369385 2583 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-offline-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:41:40.369585 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.369395 2583 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-online-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:41:40.369585 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.369402 2583 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7dd0116-d7f3-42c3-8150-335c8f95a150-registry-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:41:40.369585 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.369411 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gr6mj\" (UniqueName: \"kubernetes.io/projected/e7dd0116-d7f3-42c3-8150-335c8f95a150-kube-api-access-gr6mj\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:41:40.777210 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.777175 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" event={"ID":"e7dd0116-d7f3-42c3-8150-335c8f95a150","Type":"ContainerDied","Data":"1330e78f2b8d0e1c0b1bbcf7d2269fa5fb7497265908912b69418092b9cbc182"} Sep 10 12:41:40.777210 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.777201 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f" Sep 10 12:41:40.777671 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.777237 2583 scope.go:117] "RemoveContainer" containerID="7262df39e01736bc27bf66d66d08d04bf95a716192e4dbb8da38cfb1790b85c0" Sep 10 12:41:40.780288 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.780262 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" event={"ID":"9a4b3055-a26e-4480-bea4-54f88452d6c5","Type":"ContainerDied","Data":"5772af11d94533f4587011174c4111acda7a95bde1d17167c86870df968c5de0"} Sep 10 12:41:40.780416 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.780307 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c" Sep 10 12:41:40.786171 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.786152 2583 scope.go:117] "RemoveContainer" containerID="f5c3bef02da8b094b79e97309f9323a1063eba3b275fb5e0cbd5f67c7fa20cde" Sep 10 12:41:40.793591 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.793573 2583 scope.go:117] "RemoveContainer" containerID="dd6c53f7296c85f431eb7ccaee08bc257f0d8b75eafab821713b54f11b569321" Sep 10 12:41:40.798773 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.798748 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f"] Sep 10 12:41:40.801654 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.801633 2583 scope.go:117] "RemoveContainer" containerID="8b93f9ea0513b184de1ec96a93445f1c7c6034ab93968b7ef4398f9e0b9e90b7" Sep 10 12:41:40.803119 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.803095 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-ppn2f"] Sep 10 12:41:40.808987 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.808970 2583 scope.go:117] "RemoveContainer" containerID="74c6ab84625709944dba6b12e7ffa7d35de9856fdd5b6fa525628e6c12e90e0c" Sep 10 12:41:40.813682 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.813662 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c"] Sep 10 12:41:40.817011 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.816912 2583 scope.go:117] "RemoveContainer" containerID="29acf929a9f642461f9d2b037c843d2248318f92ad72e6d7bce8f0eb020f4bf5" Sep 10 12:41:40.817791 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.817767 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-wvt5c"] Sep 10 12:41:40.825233 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.825216 2583 scope.go:117] "RemoveContainer" containerID="f83a05e2304ebb73a653d6529ba968416c2bdd172d65a70099998c7dcf74e6ae" Sep 10 12:41:40.833113 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.833092 2583 scope.go:117] "RemoveContainer" containerID="c6d68580d16f823043ceecee630e03e088246e1b6f5d04502ac52bfbeb5b5691" Sep 10 12:41:40.841129 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.841111 2583 scope.go:117] "RemoveContainer" containerID="4a7c9ecc99385e55b5ba5f917667f6bed3bd71da1edd7dae4eec36ee1937478e" Sep 10 12:41:40.848933 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.848911 2583 scope.go:117] "RemoveContainer" containerID="2febdd0d631f3fcc2dc8d3162a48a41f94c9636e6a7fe830ff13785bd0cf81f8" Sep 10 12:41:40.858273 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:40.858242 2583 scope.go:117] "RemoveContainer" containerID="f1a43254b80d8d1772c1abdc82bc530159c2d0007c5c51b3d9c255bc10e2c6e8" Sep 10 12:41:42.469657 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:42.469617 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" path="/var/lib/kubelet/pods/9a4b3055-a26e-4480-bea4-54f88452d6c5/volumes" Sep 10 12:41:42.470195 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:42.470179 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" path="/var/lib/kubelet/pods/e7dd0116-d7f3-42c3-8150-335c8f95a150/volumes" Sep 10 12:41:58.015217 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:58.015179 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-kzn5b"] Sep 10 12:41:58.015663 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:58.015374 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kzn5b" podUID="0e31c99a-d958-4e92-b12a-6521b45e778c" containerName="manager" containerID="cri-o://1ba454369d5dee4d3c7d66788c7d191685291bb78c604fdc6b56e826dc724b7e" gracePeriod=10 Sep 10 12:41:58.260649 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:58.260625 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kzn5b" Sep 10 12:41:58.301434 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:58.301408 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fr9k\" (UniqueName: \"kubernetes.io/projected/0e31c99a-d958-4e92-b12a-6521b45e778c-kube-api-access-8fr9k\") pod \"0e31c99a-d958-4e92-b12a-6521b45e778c\" (UID: \"0e31c99a-d958-4e92-b12a-6521b45e778c\") " Sep 10 12:41:58.303598 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:58.303573 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e31c99a-d958-4e92-b12a-6521b45e778c-kube-api-access-8fr9k" (OuterVolumeSpecName: "kube-api-access-8fr9k") pod "0e31c99a-d958-4e92-b12a-6521b45e778c" (UID: "0e31c99a-d958-4e92-b12a-6521b45e778c"). InnerVolumeSpecName "kube-api-access-8fr9k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:41:58.402569 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:58.402543 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fr9k\" (UniqueName: \"kubernetes.io/projected/0e31c99a-d958-4e92-b12a-6521b45e778c-kube-api-access-8fr9k\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:41:58.847815 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:58.847774 2583 generic.go:356] "Generic (PLEG): container finished" podID="0e31c99a-d958-4e92-b12a-6521b45e778c" containerID="1ba454369d5dee4d3c7d66788c7d191685291bb78c604fdc6b56e826dc724b7e" exitCode=0 Sep 10 12:41:58.848035 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:58.847856 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kzn5b" Sep 10 12:41:58.848035 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:58.847869 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kzn5b" event={"ID":"0e31c99a-d958-4e92-b12a-6521b45e778c","Type":"ContainerDied","Data":"1ba454369d5dee4d3c7d66788c7d191685291bb78c604fdc6b56e826dc724b7e"} Sep 10 12:41:58.848035 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:58.847924 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kzn5b" event={"ID":"0e31c99a-d958-4e92-b12a-6521b45e778c","Type":"ContainerDied","Data":"aa65230a4ee315b8e1da65d906a4d943e1e11d9562dc1a1a395f79dff3acf61c"} Sep 10 12:41:58.848035 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:58.847948 2583 scope.go:117] "RemoveContainer" containerID="1ba454369d5dee4d3c7d66788c7d191685291bb78c604fdc6b56e826dc724b7e" Sep 10 12:41:58.857346 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:58.857325 2583 scope.go:117] "RemoveContainer" containerID="1ba454369d5dee4d3c7d66788c7d191685291bb78c604fdc6b56e826dc724b7e" Sep 10 12:41:58.857681 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:41:58.857662 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ba454369d5dee4d3c7d66788c7d191685291bb78c604fdc6b56e826dc724b7e\": container with ID starting with 1ba454369d5dee4d3c7d66788c7d191685291bb78c604fdc6b56e826dc724b7e not found: ID does not exist" containerID="1ba454369d5dee4d3c7d66788c7d191685291bb78c604fdc6b56e826dc724b7e" Sep 10 12:41:58.857736 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:58.857690 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ba454369d5dee4d3c7d66788c7d191685291bb78c604fdc6b56e826dc724b7e"} err="failed to get container status \"1ba454369d5dee4d3c7d66788c7d191685291bb78c604fdc6b56e826dc724b7e\": rpc error: code = NotFound desc = could not find container \"1ba454369d5dee4d3c7d66788c7d191685291bb78c604fdc6b56e826dc724b7e\": container with ID starting with 1ba454369d5dee4d3c7d66788c7d191685291bb78c604fdc6b56e826dc724b7e not found: ID does not exist" Sep 10 12:41:58.863252 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:58.863226 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-kzn5b"] Sep 10 12:41:58.866446 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:41:58.866425 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-kzn5b"] Sep 10 12:42:00.468945 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:00.468912 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e31c99a-d958-4e92-b12a-6521b45e778c" path="/var/lib/kubelet/pods/0e31c99a-d958-4e92-b12a-6521b45e778c/volumes" Sep 10 12:42:03.219895 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.219862 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-2cq96"] Sep 10 12:42:03.220256 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220122 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="online" Sep 10 12:42:03.220256 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220136 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="online" Sep 10 12:42:03.220256 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220144 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="feast-init" Sep 10 12:42:03.220256 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220149 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="feast-init" Sep 10 12:42:03.220256 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220159 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="offline" Sep 10 12:42:03.220256 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220164 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="offline" Sep 10 12:42:03.220256 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220174 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="ui" Sep 10 12:42:03.220256 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220178 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="ui" Sep 10 12:42:03.220256 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220185 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="feast-apply" Sep 10 12:42:03.220256 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220193 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="feast-apply" Sep 10 12:42:03.220256 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220207 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e31c99a-d958-4e92-b12a-6521b45e778c" containerName="manager" Sep 10 12:42:03.220256 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220212 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e31c99a-d958-4e92-b12a-6521b45e778c" containerName="manager" Sep 10 12:42:03.220256 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220221 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="offline" Sep 10 12:42:03.220256 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220226 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="offline" Sep 10 12:42:03.220256 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220231 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="ui" Sep 10 12:42:03.220256 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220235 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="ui" Sep 10 12:42:03.220256 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220242 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="registry" Sep 10 12:42:03.220256 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220246 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="registry" Sep 10 12:42:03.220256 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220252 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="feast-apply" Sep 10 12:42:03.220256 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220257 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="feast-apply" Sep 10 12:42:03.220256 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220262 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="online" Sep 10 12:42:03.220866 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220268 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="online" Sep 10 12:42:03.220866 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220279 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="feast-init" Sep 10 12:42:03.220866 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220285 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="feast-init" Sep 10 12:42:03.220866 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220325 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="offline" Sep 10 12:42:03.220866 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220333 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="online" Sep 10 12:42:03.220866 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220339 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="registry" Sep 10 12:42:03.220866 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220347 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="ui" Sep 10 12:42:03.220866 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220357 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e31c99a-d958-4e92-b12a-6521b45e778c" containerName="manager" Sep 10 12:42:03.220866 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220364 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="online" Sep 10 12:42:03.220866 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220371 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7dd0116-d7f3-42c3-8150-335c8f95a150" containerName="offline" Sep 10 12:42:03.220866 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.220377 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a4b3055-a26e-4480-bea4-54f88452d6c5" containerName="ui" Sep 10 12:42:03.224556 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.224538 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-2cq96" Sep 10 12:42:03.227004 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.226975 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 10 12:42:03.227126 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.227094 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 10 12:42:03.227772 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.227756 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-bmgmk\"" Sep 10 12:42:03.231922 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.231902 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-2cq96"] Sep 10 12:42:03.343881 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.343824 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgkg8\" (UniqueName: \"kubernetes.io/projected/fc48a060-91c8-45c6-b7c2-d9db3264cbd8-kube-api-access-pgkg8\") pod \"feast-operator-controller-manager-689b877b4b-2cq96\" (UID: \"fc48a060-91c8-45c6-b7c2-d9db3264cbd8\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-2cq96" Sep 10 12:42:03.444950 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.444911 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgkg8\" (UniqueName: \"kubernetes.io/projected/fc48a060-91c8-45c6-b7c2-d9db3264cbd8-kube-api-access-pgkg8\") pod \"feast-operator-controller-manager-689b877b4b-2cq96\" (UID: \"fc48a060-91c8-45c6-b7c2-d9db3264cbd8\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-2cq96" Sep 10 12:42:03.453891 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.453867 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgkg8\" (UniqueName: \"kubernetes.io/projected/fc48a060-91c8-45c6-b7c2-d9db3264cbd8-kube-api-access-pgkg8\") pod \"feast-operator-controller-manager-689b877b4b-2cq96\" (UID: \"fc48a060-91c8-45c6-b7c2-d9db3264cbd8\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-2cq96" Sep 10 12:42:03.534661 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.534574 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-2cq96" Sep 10 12:42:03.803896 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.803854 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-2cq96"] Sep 10 12:42:03.806610 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:42:03.806583 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc48a060_91c8_45c6_b7c2_d9db3264cbd8.slice/crio-6a4f396948aaab4022bdcdf3664a0d5372bf7bbcbd33b3e1ef9db4daade3e678 WatchSource:0}: Error finding container 6a4f396948aaab4022bdcdf3664a0d5372bf7bbcbd33b3e1ef9db4daade3e678: Status 404 returned error can't find the container with id 6a4f396948aaab4022bdcdf3664a0d5372bf7bbcbd33b3e1ef9db4daade3e678 Sep 10 12:42:03.867513 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.867478 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-2cq96" event={"ID":"fc48a060-91c8-45c6-b7c2-d9db3264cbd8","Type":"ContainerStarted","Data":"078d1fc881fc6e2a04a0ddfc375afee61d9501a671e4d0d2554ca1d89b3d387e"} Sep 10 12:42:03.867635 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.867544 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-2cq96" event={"ID":"fc48a060-91c8-45c6-b7c2-d9db3264cbd8","Type":"ContainerStarted","Data":"6a4f396948aaab4022bdcdf3664a0d5372bf7bbcbd33b3e1ef9db4daade3e678"} Sep 10 12:42:03.867635 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.867573 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-2cq96" Sep 10 12:42:03.882969 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:03.882874 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-2cq96" podStartSLOduration=0.882834336 podStartE2EDuration="882.834336ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:42:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:42:03.882031569 +0000 UTC m=+1451.887251033" watchObservedRunningTime="2026-09-10 12:42:03.882834336 +0000 UTC m=+1451.888053798" Sep 10 12:42:14.872887 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:14.872801 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-2cq96" Sep 10 12:42:20.421699 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.421656 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw"] Sep 10 12:42:20.425932 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.425911 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:20.429254 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.429028 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 10 12:42:20.429254 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.429069 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 10 12:42:20.429254 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.429224 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 10 12:42:20.429507 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.429329 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-g62mc\"" Sep 10 12:42:20.429507 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.429396 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 10 12:42:20.429614 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.429582 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 10 12:42:20.430259 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.429768 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 10 12:42:20.438889 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.438867 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw"] Sep 10 12:42:20.574127 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.574094 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zbnbw\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:20.574300 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.574151 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zbnbw\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:20.574300 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.574197 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zbnbw\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:20.574300 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.574224 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtmtw\" (UniqueName: \"kubernetes.io/projected/baa02eb4-2b22-4131-a891-ccb7e30bc051-kube-api-access-vtmtw\") pod \"feast-simple-feast-setup-57576b8fcf-zbnbw\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:20.574300 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.574244 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zbnbw\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:20.574520 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.574311 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/baa02eb4-2b22-4131-a891-ccb7e30bc051-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-zbnbw\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:20.674701 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.674629 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zbnbw\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:20.674701 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.674660 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zbnbw\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:20.674701 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.674681 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtmtw\" (UniqueName: \"kubernetes.io/projected/baa02eb4-2b22-4131-a891-ccb7e30bc051-kube-api-access-vtmtw\") pod \"feast-simple-feast-setup-57576b8fcf-zbnbw\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:20.674701 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.674698 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zbnbw\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:20.674978 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:42:20.674802 2583 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 10 12:42:20.674978 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.674827 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/baa02eb4-2b22-4131-a891-ccb7e30bc051-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-zbnbw\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:20.674978 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:42:20.674893 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-offline-tls podName:baa02eb4-2b22-4131-a891-ccb7e30bc051 nodeName:}" failed. No retries permitted until 2026-09-10 12:42:21.174872816 +0000 UTC m=+1469.180092258 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-offline-tls") pod "feast-simple-feast-setup-57576b8fcf-zbnbw" (UID: "baa02eb4-2b22-4131-a891-ccb7e30bc051") : secret "feast-simple-feast-setup-offline-tls" not found Sep 10 12:42:20.674978 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.674973 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zbnbw\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:20.675261 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.675238 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/baa02eb4-2b22-4131-a891-ccb7e30bc051-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-zbnbw\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:20.677356 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.677330 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zbnbw\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:20.677464 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.677330 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zbnbw\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:20.677464 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.677341 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zbnbw\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:20.686876 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:20.686855 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtmtw\" (UniqueName: \"kubernetes.io/projected/baa02eb4-2b22-4131-a891-ccb7e30bc051-kube-api-access-vtmtw\") pod \"feast-simple-feast-setup-57576b8fcf-zbnbw\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:21.178941 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:21.178904 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zbnbw\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:21.181397 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:21.181370 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zbnbw\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:21.344901 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:21.344856 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:21.619901 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:21.619880 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw"] Sep 10 12:42:21.621312 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:42:21.621285 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbaa02eb4_2b22_4131_a891_ccb7e30bc051.slice/crio-6697b0c46ae828346e12bc60591b6c8d71e10afbbb645bfe72d1a1db3fddda2a WatchSource:0}: Error finding container 6697b0c46ae828346e12bc60591b6c8d71e10afbbb645bfe72d1a1db3fddda2a: Status 404 returned error can't find the container with id 6697b0c46ae828346e12bc60591b6c8d71e10afbbb645bfe72d1a1db3fddda2a Sep 10 12:42:21.937131 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:21.936997 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" event={"ID":"baa02eb4-2b22-4131-a891-ccb7e30bc051","Type":"ContainerStarted","Data":"f7468a220b9e3adac100870e32b730f437bb0019d4774a56ba1838382c18f866"} Sep 10 12:42:21.937131 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:21.937053 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" event={"ID":"baa02eb4-2b22-4131-a891-ccb7e30bc051","Type":"ContainerStarted","Data":"6697b0c46ae828346e12bc60591b6c8d71e10afbbb645bfe72d1a1db3fddda2a"} Sep 10 12:42:26.962999 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:26.962963 2583 generic.go:356] "Generic (PLEG): container finished" podID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerID="f7468a220b9e3adac100870e32b730f437bb0019d4774a56ba1838382c18f866" exitCode=0 Sep 10 12:42:26.963409 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:26.963007 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" event={"ID":"baa02eb4-2b22-4131-a891-ccb7e30bc051","Type":"ContainerDied","Data":"f7468a220b9e3adac100870e32b730f437bb0019d4774a56ba1838382c18f866"} Sep 10 12:42:27.969274 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:27.969244 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" event={"ID":"baa02eb4-2b22-4131-a891-ccb7e30bc051","Type":"ContainerStarted","Data":"5de47961c119b363012925f362165b64c927b9fe49b4067336ebc9a7cd35bd80"} Sep 10 12:42:32.989507 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:32.989473 2583 generic.go:356] "Generic (PLEG): container finished" podID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerID="5de47961c119b363012925f362165b64c927b9fe49b4067336ebc9a7cd35bd80" exitCode=0 Sep 10 12:42:32.989507 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:32.989519 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" event={"ID":"baa02eb4-2b22-4131-a891-ccb7e30bc051","Type":"ContainerDied","Data":"5de47961c119b363012925f362165b64c927b9fe49b4067336ebc9a7cd35bd80"} Sep 10 12:42:34.003922 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:34.003880 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" event={"ID":"baa02eb4-2b22-4131-a891-ccb7e30bc051","Type":"ContainerStarted","Data":"62ffb7a2e299797de04d67a111b037ef96f9d63acc630da0ec414d5ada4c1eb1"} Sep 10 12:42:34.004410 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:34.003948 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" event={"ID":"baa02eb4-2b22-4131-a891-ccb7e30bc051","Type":"ContainerStarted","Data":"890ccd98eec822cef3fe46c460b37ae38b5cb3e107e40f5ef37f4c55e5c5234d"} Sep 10 12:42:34.004410 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:34.003971 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" event={"ID":"baa02eb4-2b22-4131-a891-ccb7e30bc051","Type":"ContainerStarted","Data":"11cf80db3f965798377a9416181a987eb064d23004a10271e2a1f22487755bbe"} Sep 10 12:42:34.004410 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:34.003990 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" event={"ID":"baa02eb4-2b22-4131-a891-ccb7e30bc051","Type":"ContainerStarted","Data":"9afef2784b360becb8f488a2eff9ca5935978588ec4fe8b64c1e1777ef7f4d15"} Sep 10 12:42:34.030894 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:34.030803 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" podStartSLOduration=14.03077994 podStartE2EDuration="14.03077994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:42:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:42:34.028165481 +0000 UTC m=+1482.033384954" watchObservedRunningTime="2026-09-10 12:42:34.03077994 +0000 UTC m=+1482.035999417" Sep 10 12:42:36.345599 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:36.345437 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:36.345599 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:36.345509 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:36.345599 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:36.345529 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:36.345599 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:36.345546 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:36.347822 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:36.347791 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.30:6571: connect: connection refused" Sep 10 12:42:36.348054 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:36.347824 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.30:8816: connect: connection refused" Sep 10 12:42:36.348140 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:36.347862 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.30:8443: connect: connection refused" Sep 10 12:42:36.348140 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:36.347868 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused" Sep 10 12:42:39.346003 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:39.345878 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.30:6571: connect: connection refused" Sep 10 12:42:39.346003 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:39.345922 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.30:8443: connect: connection refused" Sep 10 12:42:39.346003 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:39.345894 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused" Sep 10 12:42:39.346003 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:39.345896 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.30:8816: connect: connection refused" Sep 10 12:42:41.345399 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:41.345358 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:41.345785 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:41.345419 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:41.345785 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:41.345435 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:41.345785 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:41.345447 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:42.346699 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:42.346665 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:42.347140 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:42.346948 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:42.347480 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:42.347461 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:42.347568 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:42.347513 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:42.347568 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:42.347548 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:42.352489 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:42.352472 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:43.047652 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:43.047621 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:43.050186 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:43.050166 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:42:52.445122 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:52.445096 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/ovn-acl-logging/0.log" Sep 10 12:42:52.448115 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:42:52.448093 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/ovn-acl-logging/0.log" Sep 10 12:43:44.593308 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:44.593275 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw"] Sep 10 12:43:44.596815 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:44.593546 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="registry" containerID="cri-o://9afef2784b360becb8f488a2eff9ca5935978588ec4fe8b64c1e1777ef7f4d15" gracePeriod=30 Sep 10 12:43:44.596815 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:44.593758 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="ui" containerID="cri-o://62ffb7a2e299797de04d67a111b037ef96f9d63acc630da0ec414d5ada4c1eb1" gracePeriod=30 Sep 10 12:43:44.596815 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:44.593833 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="online" containerID="cri-o://11cf80db3f965798377a9416181a987eb064d23004a10271e2a1f22487755bbe" gracePeriod=30 Sep 10 12:43:44.596815 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:44.594112 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="offline" containerID="cri-o://890ccd98eec822cef3fe46c460b37ae38b5cb3e107e40f5ef37f4c55e5c5234d" gracePeriod=30 Sep 10 12:43:44.991265 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:44.991177 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6"] Sep 10 12:43:44.994853 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:44.994819 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:43:44.997565 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:44.997544 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-74qs8\"" Sep 10 12:43:45.010760 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:45.010475 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6"] Sep 10 12:43:45.111520 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:45.111478 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f618c8bf-9256-42fc-8937-49680946bcd5-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8d6r6\" (UID: \"f618c8bf-9256-42fc-8937-49680946bcd5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:43:45.111704 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:45.111547 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f618c8bf-9256-42fc-8937-49680946bcd5-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8d6r6\" (UID: \"f618c8bf-9256-42fc-8937-49680946bcd5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:43:45.111704 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:45.111596 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9dwm\" (UniqueName: \"kubernetes.io/projected/f618c8bf-9256-42fc-8937-49680946bcd5-kube-api-access-b9dwm\") pod \"feast-simple-feast-setup-57576b8fcf-8d6r6\" (UID: \"f618c8bf-9256-42fc-8937-49680946bcd5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:43:45.111704 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:45.111638 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f618c8bf-9256-42fc-8937-49680946bcd5-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-8d6r6\" (UID: \"f618c8bf-9256-42fc-8937-49680946bcd5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:43:45.111704 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:45.111665 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f618c8bf-9256-42fc-8937-49680946bcd5-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8d6r6\" (UID: \"f618c8bf-9256-42fc-8937-49680946bcd5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:43:45.111704 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:45.111695 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f618c8bf-9256-42fc-8937-49680946bcd5-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8d6r6\" (UID: \"f618c8bf-9256-42fc-8937-49680946bcd5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:43:45.212992 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:45.212959 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f618c8bf-9256-42fc-8937-49680946bcd5-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8d6r6\" (UID: \"f618c8bf-9256-42fc-8937-49680946bcd5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:43:45.213130 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:45.213003 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9dwm\" (UniqueName: \"kubernetes.io/projected/f618c8bf-9256-42fc-8937-49680946bcd5-kube-api-access-b9dwm\") pod \"feast-simple-feast-setup-57576b8fcf-8d6r6\" (UID: \"f618c8bf-9256-42fc-8937-49680946bcd5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:43:45.213130 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:45.213022 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f618c8bf-9256-42fc-8937-49680946bcd5-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-8d6r6\" (UID: \"f618c8bf-9256-42fc-8937-49680946bcd5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:43:45.213130 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:45.213048 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f618c8bf-9256-42fc-8937-49680946bcd5-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8d6r6\" (UID: \"f618c8bf-9256-42fc-8937-49680946bcd5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:43:45.213130 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:45.213080 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f618c8bf-9256-42fc-8937-49680946bcd5-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8d6r6\" (UID: \"f618c8bf-9256-42fc-8937-49680946bcd5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:43:45.213342 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:45.213139 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f618c8bf-9256-42fc-8937-49680946bcd5-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8d6r6\" (UID: \"f618c8bf-9256-42fc-8937-49680946bcd5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:43:45.213439 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:45.213416 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f618c8bf-9256-42fc-8937-49680946bcd5-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-8d6r6\" (UID: \"f618c8bf-9256-42fc-8937-49680946bcd5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:43:45.216039 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:45.216014 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f618c8bf-9256-42fc-8937-49680946bcd5-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8d6r6\" (UID: \"f618c8bf-9256-42fc-8937-49680946bcd5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:43:45.216147 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:45.216044 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f618c8bf-9256-42fc-8937-49680946bcd5-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8d6r6\" (UID: \"f618c8bf-9256-42fc-8937-49680946bcd5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:43:45.216147 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:45.216117 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f618c8bf-9256-42fc-8937-49680946bcd5-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8d6r6\" (UID: \"f618c8bf-9256-42fc-8937-49680946bcd5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:43:45.216147 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:45.216136 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f618c8bf-9256-42fc-8937-49680946bcd5-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-8d6r6\" (UID: \"f618c8bf-9256-42fc-8937-49680946bcd5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:43:45.221834 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:45.221812 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9dwm\" (UniqueName: \"kubernetes.io/projected/f618c8bf-9256-42fc-8937-49680946bcd5-kube-api-access-b9dwm\") pod \"feast-simple-feast-setup-57576b8fcf-8d6r6\" (UID: \"f618c8bf-9256-42fc-8937-49680946bcd5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:43:45.286809 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:45.286714 2583 generic.go:356] "Generic (PLEG): container finished" podID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerID="62ffb7a2e299797de04d67a111b037ef96f9d63acc630da0ec414d5ada4c1eb1" exitCode=0 Sep 10 12:43:45.286809 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:45.286761 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" event={"ID":"baa02eb4-2b22-4131-a891-ccb7e30bc051","Type":"ContainerDied","Data":"62ffb7a2e299797de04d67a111b037ef96f9d63acc630da0ec414d5ada4c1eb1"} Sep 10 12:43:45.304990 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:45.304955 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:43:45.588895 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:45.588870 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6"] Sep 10 12:43:45.590801 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:43:45.590777 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf618c8bf_9256_42fc_8937_49680946bcd5.slice/crio-1e184b3de9fa28978af5ec3bbebaeba36d8bd349d2deba868a10399605f20073 WatchSource:0}: Error finding container 1e184b3de9fa28978af5ec3bbebaeba36d8bd349d2deba868a10399605f20073: Status 404 returned error can't find the container with id 1e184b3de9fa28978af5ec3bbebaeba36d8bd349d2deba868a10399605f20073 Sep 10 12:43:46.293698 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:46.293660 2583 generic.go:356] "Generic (PLEG): container finished" podID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerID="11cf80db3f965798377a9416181a987eb064d23004a10271e2a1f22487755bbe" exitCode=0 Sep 10 12:43:46.294101 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:46.293730 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" event={"ID":"baa02eb4-2b22-4131-a891-ccb7e30bc051","Type":"ContainerDied","Data":"11cf80db3f965798377a9416181a987eb064d23004a10271e2a1f22487755bbe"} Sep 10 12:43:46.295116 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:46.295090 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" event={"ID":"f618c8bf-9256-42fc-8937-49680946bcd5","Type":"ContainerStarted","Data":"56145c218b2a28b86c74911881fd5121a9ea236ded0bdea8a27a9601d0bce9e0"} Sep 10 12:43:46.295220 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:46.295132 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" event={"ID":"f618c8bf-9256-42fc-8937-49680946bcd5","Type":"ContainerStarted","Data":"1e184b3de9fa28978af5ec3bbebaeba36d8bd349d2deba868a10399605f20073"} Sep 10 12:43:51.314943 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:51.314912 2583 generic.go:356] "Generic (PLEG): container finished" podID="f618c8bf-9256-42fc-8937-49680946bcd5" containerID="56145c218b2a28b86c74911881fd5121a9ea236ded0bdea8a27a9601d0bce9e0" exitCode=0 Sep 10 12:43:51.315311 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:51.314983 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" event={"ID":"f618c8bf-9256-42fc-8937-49680946bcd5","Type":"ContainerDied","Data":"56145c218b2a28b86c74911881fd5121a9ea236ded0bdea8a27a9601d0bce9e0"} Sep 10 12:43:52.320914 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:52.320880 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" event={"ID":"f618c8bf-9256-42fc-8937-49680946bcd5","Type":"ContainerStarted","Data":"4228ca52ccff209cbf6744901677de6020712d4608157617425cc90e60ad5fc2"} Sep 10 12:43:53.047611 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:53.047566 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused" Sep 10 12:43:53.047777 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:53.047570 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.30:8443: connect: connection refused" Sep 10 12:43:57.341521 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:57.341434 2583 generic.go:356] "Generic (PLEG): container finished" podID="f618c8bf-9256-42fc-8937-49680946bcd5" containerID="4228ca52ccff209cbf6744901677de6020712d4608157617425cc90e60ad5fc2" exitCode=0 Sep 10 12:43:57.341521 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:57.341470 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" event={"ID":"f618c8bf-9256-42fc-8937-49680946bcd5","Type":"ContainerDied","Data":"4228ca52ccff209cbf6744901677de6020712d4608157617425cc90e60ad5fc2"} Sep 10 12:43:58.355920 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:58.355879 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" event={"ID":"f618c8bf-9256-42fc-8937-49680946bcd5","Type":"ContainerStarted","Data":"1b6ea7559f005c4bc4fd0e97e20171a285a6edb9faeaab923447b2b5228a00d6"} Sep 10 12:43:58.356388 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:58.355939 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" event={"ID":"f618c8bf-9256-42fc-8937-49680946bcd5","Type":"ContainerStarted","Data":"7aacdc257fe24e88a9e7bd16bdd4b9a3e53e95217afe0d65c613dc968831b489"} Sep 10 12:43:58.356388 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:58.355962 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" event={"ID":"f618c8bf-9256-42fc-8937-49680946bcd5","Type":"ContainerStarted","Data":"40cf400d85b84b20e9c1268cf3fb48448e8b59e73c0766ed766bb60b6f4c90a5"} Sep 10 12:43:58.356388 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:58.355979 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" event={"ID":"f618c8bf-9256-42fc-8937-49680946bcd5","Type":"ContainerStarted","Data":"f9037a2743e2cc99bb9ee342834036b7edda2ad96323ff506c4c3a56039b03c2"} Sep 10 12:43:58.381483 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:43:58.381411 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" podStartSLOduration=14.381388896 podStartE2EDuration="14.381388896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:43:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:43:58.378896061 +0000 UTC m=+1566.384115524" watchObservedRunningTime="2026-09-10 12:43:58.381388896 +0000 UTC m=+1566.386608360" Sep 10 12:44:00.306332 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:00.306287 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:44:00.306813 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:00.306354 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:44:00.306813 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:00.306375 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:44:00.306813 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:00.306392 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:44:00.308729 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:00.308606 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.31:8443: connect: connection refused" Sep 10 12:44:00.308729 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:00.308625 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.31:8816: connect: connection refused" Sep 10 12:44:00.308729 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:00.308645 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.31:6571: connect: connection refused" Sep 10 12:44:00.308729 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:00.308708 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": dial tcp 10.133.0.31:6567: connect: connection refused" Sep 10 12:44:03.048249 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:03.048194 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.30:8443: connect: connection refused" Sep 10 12:44:03.048733 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:03.048206 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused" Sep 10 12:44:03.306250 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:03.306065 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.31:8443: connect: connection refused" Sep 10 12:44:03.306250 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:03.306131 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.31:8816: connect: connection refused" Sep 10 12:44:03.306250 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:03.306187 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.31:6571: connect: connection refused" Sep 10 12:44:03.306636 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:03.306611 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": dial tcp 10.133.0.31:6567: connect: connection refused" Sep 10 12:44:05.305116 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:05.305083 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:44:05.305461 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:05.305144 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:44:05.305461 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:05.305165 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:44:05.305461 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:05.305184 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:44:06.305761 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:06.305727 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:44:06.306177 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:06.305931 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:44:06.306177 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:06.306095 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:44:06.306796 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:06.306772 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:44:06.306929 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:06.306864 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:44:06.307004 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:06.306985 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:44:06.311306 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:06.311291 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:44:06.399999 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:06.399968 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:44:13.047969 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:13.047925 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.30:8443: connect: connection refused" Sep 10 12:44:13.048336 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:13.048000 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused" Sep 10 12:44:13.048336 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:13.048056 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:44:13.048336 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:13.048222 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:44:15.231991 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.231967 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:44:15.264792 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.264764 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-ui-tls\") pod \"baa02eb4-2b22-4131-a891-ccb7e30bc051\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " Sep 10 12:44:15.264955 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.264805 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/baa02eb4-2b22-4131-a891-ccb7e30bc051-feast-data\") pod \"baa02eb4-2b22-4131-a891-ccb7e30bc051\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " Sep 10 12:44:15.264955 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.264856 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-registry-tls\") pod \"baa02eb4-2b22-4131-a891-ccb7e30bc051\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " Sep 10 12:44:15.265310 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.265287 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-offline-tls\") pod \"baa02eb4-2b22-4131-a891-ccb7e30bc051\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " Sep 10 12:44:15.265428 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.265353 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-online-tls\") pod \"baa02eb4-2b22-4131-a891-ccb7e30bc051\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " Sep 10 12:44:15.265428 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.265375 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baa02eb4-2b22-4131-a891-ccb7e30bc051-feast-data" (OuterVolumeSpecName: "feast-data") pod "baa02eb4-2b22-4131-a891-ccb7e30bc051" (UID: "baa02eb4-2b22-4131-a891-ccb7e30bc051"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:44:15.265428 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.265413 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vtmtw\" (UniqueName: \"kubernetes.io/projected/baa02eb4-2b22-4131-a891-ccb7e30bc051-kube-api-access-vtmtw\") pod \"baa02eb4-2b22-4131-a891-ccb7e30bc051\" (UID: \"baa02eb4-2b22-4131-a891-ccb7e30bc051\") " Sep 10 12:44:15.266231 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.266196 2583 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/baa02eb4-2b22-4131-a891-ccb7e30bc051-feast-data\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:44:15.267666 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.267633 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "baa02eb4-2b22-4131-a891-ccb7e30bc051" (UID: "baa02eb4-2b22-4131-a891-ccb7e30bc051"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:44:15.267666 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.267646 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "baa02eb4-2b22-4131-a891-ccb7e30bc051" (UID: "baa02eb4-2b22-4131-a891-ccb7e30bc051"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:44:15.267874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.267682 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "baa02eb4-2b22-4131-a891-ccb7e30bc051" (UID: "baa02eb4-2b22-4131-a891-ccb7e30bc051"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:44:15.267874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.267860 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baa02eb4-2b22-4131-a891-ccb7e30bc051-kube-api-access-vtmtw" (OuterVolumeSpecName: "kube-api-access-vtmtw") pod "baa02eb4-2b22-4131-a891-ccb7e30bc051" (UID: "baa02eb4-2b22-4131-a891-ccb7e30bc051"). InnerVolumeSpecName "kube-api-access-vtmtw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:44:15.268116 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.268090 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-online-tls" (OuterVolumeSpecName: "online-tls") pod "baa02eb4-2b22-4131-a891-ccb7e30bc051" (UID: "baa02eb4-2b22-4131-a891-ccb7e30bc051"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:44:15.367002 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.366947 2583 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-ui-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:44:15.367002 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.366968 2583 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-registry-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:44:15.367002 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.366977 2583 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-offline-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:44:15.367002 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.367002 2583 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/baa02eb4-2b22-4131-a891-ccb7e30bc051-online-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:44:15.367179 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.367013 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtmtw\" (UniqueName: \"kubernetes.io/projected/baa02eb4-2b22-4131-a891-ccb7e30bc051-kube-api-access-vtmtw\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:44:15.431728 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.431699 2583 generic.go:356] "Generic (PLEG): container finished" podID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerID="890ccd98eec822cef3fe46c460b37ae38b5cb3e107e40f5ef37f4c55e5c5234d" exitCode=137 Sep 10 12:44:15.431728 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.431723 2583 generic.go:356] "Generic (PLEG): container finished" podID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerID="9afef2784b360becb8f488a2eff9ca5935978588ec4fe8b64c1e1777ef7f4d15" exitCode=137 Sep 10 12:44:15.431875 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.431757 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" Sep 10 12:44:15.431875 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.431762 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" event={"ID":"baa02eb4-2b22-4131-a891-ccb7e30bc051","Type":"ContainerDied","Data":"890ccd98eec822cef3fe46c460b37ae38b5cb3e107e40f5ef37f4c55e5c5234d"} Sep 10 12:44:15.431875 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.431804 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" event={"ID":"baa02eb4-2b22-4131-a891-ccb7e30bc051","Type":"ContainerDied","Data":"9afef2784b360becb8f488a2eff9ca5935978588ec4fe8b64c1e1777ef7f4d15"} Sep 10 12:44:15.431875 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.431818 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw" event={"ID":"baa02eb4-2b22-4131-a891-ccb7e30bc051","Type":"ContainerDied","Data":"6697b0c46ae828346e12bc60591b6c8d71e10afbbb645bfe72d1a1db3fddda2a"} Sep 10 12:44:15.431875 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.431855 2583 scope.go:117] "RemoveContainer" containerID="62ffb7a2e299797de04d67a111b037ef96f9d63acc630da0ec414d5ada4c1eb1" Sep 10 12:44:15.440271 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.440040 2583 scope.go:117] "RemoveContainer" containerID="890ccd98eec822cef3fe46c460b37ae38b5cb3e107e40f5ef37f4c55e5c5234d" Sep 10 12:44:15.447236 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.447218 2583 scope.go:117] "RemoveContainer" containerID="11cf80db3f965798377a9416181a987eb064d23004a10271e2a1f22487755bbe" Sep 10 12:44:15.454575 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.454552 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw"] Sep 10 12:44:15.455125 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.455108 2583 scope.go:117] "RemoveContainer" containerID="9afef2784b360becb8f488a2eff9ca5935978588ec4fe8b64c1e1777ef7f4d15" Sep 10 12:44:15.457735 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.457716 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-zbnbw"] Sep 10 12:44:15.462375 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.462349 2583 scope.go:117] "RemoveContainer" containerID="5de47961c119b363012925f362165b64c927b9fe49b4067336ebc9a7cd35bd80" Sep 10 12:44:15.469240 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.469223 2583 scope.go:117] "RemoveContainer" containerID="f7468a220b9e3adac100870e32b730f437bb0019d4774a56ba1838382c18f866" Sep 10 12:44:15.475866 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.475823 2583 scope.go:117] "RemoveContainer" containerID="62ffb7a2e299797de04d67a111b037ef96f9d63acc630da0ec414d5ada4c1eb1" Sep 10 12:44:15.476121 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:44:15.476100 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62ffb7a2e299797de04d67a111b037ef96f9d63acc630da0ec414d5ada4c1eb1\": container with ID starting with 62ffb7a2e299797de04d67a111b037ef96f9d63acc630da0ec414d5ada4c1eb1 not found: ID does not exist" containerID="62ffb7a2e299797de04d67a111b037ef96f9d63acc630da0ec414d5ada4c1eb1" Sep 10 12:44:15.476169 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.476135 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62ffb7a2e299797de04d67a111b037ef96f9d63acc630da0ec414d5ada4c1eb1"} err="failed to get container status \"62ffb7a2e299797de04d67a111b037ef96f9d63acc630da0ec414d5ada4c1eb1\": rpc error: code = NotFound desc = could not find container \"62ffb7a2e299797de04d67a111b037ef96f9d63acc630da0ec414d5ada4c1eb1\": container with ID starting with 62ffb7a2e299797de04d67a111b037ef96f9d63acc630da0ec414d5ada4c1eb1 not found: ID does not exist" Sep 10 12:44:15.476169 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.476154 2583 scope.go:117] "RemoveContainer" containerID="890ccd98eec822cef3fe46c460b37ae38b5cb3e107e40f5ef37f4c55e5c5234d" Sep 10 12:44:15.476416 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:44:15.476398 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"890ccd98eec822cef3fe46c460b37ae38b5cb3e107e40f5ef37f4c55e5c5234d\": container with ID starting with 890ccd98eec822cef3fe46c460b37ae38b5cb3e107e40f5ef37f4c55e5c5234d not found: ID does not exist" containerID="890ccd98eec822cef3fe46c460b37ae38b5cb3e107e40f5ef37f4c55e5c5234d" Sep 10 12:44:15.476467 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.476422 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"890ccd98eec822cef3fe46c460b37ae38b5cb3e107e40f5ef37f4c55e5c5234d"} err="failed to get container status \"890ccd98eec822cef3fe46c460b37ae38b5cb3e107e40f5ef37f4c55e5c5234d\": rpc error: code = NotFound desc = could not find container \"890ccd98eec822cef3fe46c460b37ae38b5cb3e107e40f5ef37f4c55e5c5234d\": container with ID starting with 890ccd98eec822cef3fe46c460b37ae38b5cb3e107e40f5ef37f4c55e5c5234d not found: ID does not exist" Sep 10 12:44:15.476467 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.476437 2583 scope.go:117] "RemoveContainer" containerID="11cf80db3f965798377a9416181a987eb064d23004a10271e2a1f22487755bbe" Sep 10 12:44:15.476695 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:44:15.476678 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11cf80db3f965798377a9416181a987eb064d23004a10271e2a1f22487755bbe\": container with ID starting with 11cf80db3f965798377a9416181a987eb064d23004a10271e2a1f22487755bbe not found: ID does not exist" containerID="11cf80db3f965798377a9416181a987eb064d23004a10271e2a1f22487755bbe" Sep 10 12:44:15.476750 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.476699 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11cf80db3f965798377a9416181a987eb064d23004a10271e2a1f22487755bbe"} err="failed to get container status \"11cf80db3f965798377a9416181a987eb064d23004a10271e2a1f22487755bbe\": rpc error: code = NotFound desc = could not find container \"11cf80db3f965798377a9416181a987eb064d23004a10271e2a1f22487755bbe\": container with ID starting with 11cf80db3f965798377a9416181a987eb064d23004a10271e2a1f22487755bbe not found: ID does not exist" Sep 10 12:44:15.476750 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.476712 2583 scope.go:117] "RemoveContainer" containerID="9afef2784b360becb8f488a2eff9ca5935978588ec4fe8b64c1e1777ef7f4d15" Sep 10 12:44:15.476958 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:44:15.476943 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9afef2784b360becb8f488a2eff9ca5935978588ec4fe8b64c1e1777ef7f4d15\": container with ID starting with 9afef2784b360becb8f488a2eff9ca5935978588ec4fe8b64c1e1777ef7f4d15 not found: ID does not exist" containerID="9afef2784b360becb8f488a2eff9ca5935978588ec4fe8b64c1e1777ef7f4d15" Sep 10 12:44:15.477011 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.476960 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9afef2784b360becb8f488a2eff9ca5935978588ec4fe8b64c1e1777ef7f4d15"} err="failed to get container status \"9afef2784b360becb8f488a2eff9ca5935978588ec4fe8b64c1e1777ef7f4d15\": rpc error: code = NotFound desc = could not find container \"9afef2784b360becb8f488a2eff9ca5935978588ec4fe8b64c1e1777ef7f4d15\": container with ID starting with 9afef2784b360becb8f488a2eff9ca5935978588ec4fe8b64c1e1777ef7f4d15 not found: ID does not exist" Sep 10 12:44:15.477011 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.476973 2583 scope.go:117] "RemoveContainer" containerID="5de47961c119b363012925f362165b64c927b9fe49b4067336ebc9a7cd35bd80" Sep 10 12:44:15.477190 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:44:15.477174 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5de47961c119b363012925f362165b64c927b9fe49b4067336ebc9a7cd35bd80\": container with ID starting with 5de47961c119b363012925f362165b64c927b9fe49b4067336ebc9a7cd35bd80 not found: ID does not exist" containerID="5de47961c119b363012925f362165b64c927b9fe49b4067336ebc9a7cd35bd80" Sep 10 12:44:15.477232 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.477193 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5de47961c119b363012925f362165b64c927b9fe49b4067336ebc9a7cd35bd80"} err="failed to get container status \"5de47961c119b363012925f362165b64c927b9fe49b4067336ebc9a7cd35bd80\": rpc error: code = NotFound desc = could not find container \"5de47961c119b363012925f362165b64c927b9fe49b4067336ebc9a7cd35bd80\": container with ID starting with 5de47961c119b363012925f362165b64c927b9fe49b4067336ebc9a7cd35bd80 not found: ID does not exist" Sep 10 12:44:15.477232 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.477205 2583 scope.go:117] "RemoveContainer" containerID="f7468a220b9e3adac100870e32b730f437bb0019d4774a56ba1838382c18f866" Sep 10 12:44:15.477460 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:44:15.477443 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7468a220b9e3adac100870e32b730f437bb0019d4774a56ba1838382c18f866\": container with ID starting with f7468a220b9e3adac100870e32b730f437bb0019d4774a56ba1838382c18f866 not found: ID does not exist" containerID="f7468a220b9e3adac100870e32b730f437bb0019d4774a56ba1838382c18f866" Sep 10 12:44:15.477504 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.477464 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7468a220b9e3adac100870e32b730f437bb0019d4774a56ba1838382c18f866"} err="failed to get container status \"f7468a220b9e3adac100870e32b730f437bb0019d4774a56ba1838382c18f866\": rpc error: code = NotFound desc = could not find container \"f7468a220b9e3adac100870e32b730f437bb0019d4774a56ba1838382c18f866\": container with ID starting with f7468a220b9e3adac100870e32b730f437bb0019d4774a56ba1838382c18f866 not found: ID does not exist" Sep 10 12:44:15.477504 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.477479 2583 scope.go:117] "RemoveContainer" containerID="62ffb7a2e299797de04d67a111b037ef96f9d63acc630da0ec414d5ada4c1eb1" Sep 10 12:44:15.477704 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.477688 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62ffb7a2e299797de04d67a111b037ef96f9d63acc630da0ec414d5ada4c1eb1"} err="failed to get container status \"62ffb7a2e299797de04d67a111b037ef96f9d63acc630da0ec414d5ada4c1eb1\": rpc error: code = NotFound desc = could not find container \"62ffb7a2e299797de04d67a111b037ef96f9d63acc630da0ec414d5ada4c1eb1\": container with ID starting with 62ffb7a2e299797de04d67a111b037ef96f9d63acc630da0ec414d5ada4c1eb1 not found: ID does not exist" Sep 10 12:44:15.477704 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.477703 2583 scope.go:117] "RemoveContainer" containerID="890ccd98eec822cef3fe46c460b37ae38b5cb3e107e40f5ef37f4c55e5c5234d" Sep 10 12:44:15.477978 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.477962 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"890ccd98eec822cef3fe46c460b37ae38b5cb3e107e40f5ef37f4c55e5c5234d"} err="failed to get container status \"890ccd98eec822cef3fe46c460b37ae38b5cb3e107e40f5ef37f4c55e5c5234d\": rpc error: code = NotFound desc = could not find container \"890ccd98eec822cef3fe46c460b37ae38b5cb3e107e40f5ef37f4c55e5c5234d\": container with ID starting with 890ccd98eec822cef3fe46c460b37ae38b5cb3e107e40f5ef37f4c55e5c5234d not found: ID does not exist" Sep 10 12:44:15.477978 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.477977 2583 scope.go:117] "RemoveContainer" containerID="11cf80db3f965798377a9416181a987eb064d23004a10271e2a1f22487755bbe" Sep 10 12:44:15.478209 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.478192 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11cf80db3f965798377a9416181a987eb064d23004a10271e2a1f22487755bbe"} err="failed to get container status \"11cf80db3f965798377a9416181a987eb064d23004a10271e2a1f22487755bbe\": rpc error: code = NotFound desc = could not find container \"11cf80db3f965798377a9416181a987eb064d23004a10271e2a1f22487755bbe\": container with ID starting with 11cf80db3f965798377a9416181a987eb064d23004a10271e2a1f22487755bbe not found: ID does not exist" Sep 10 12:44:15.478253 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.478209 2583 scope.go:117] "RemoveContainer" containerID="9afef2784b360becb8f488a2eff9ca5935978588ec4fe8b64c1e1777ef7f4d15" Sep 10 12:44:15.478466 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.478447 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9afef2784b360becb8f488a2eff9ca5935978588ec4fe8b64c1e1777ef7f4d15"} err="failed to get container status \"9afef2784b360becb8f488a2eff9ca5935978588ec4fe8b64c1e1777ef7f4d15\": rpc error: code = NotFound desc = could not find container \"9afef2784b360becb8f488a2eff9ca5935978588ec4fe8b64c1e1777ef7f4d15\": container with ID starting with 9afef2784b360becb8f488a2eff9ca5935978588ec4fe8b64c1e1777ef7f4d15 not found: ID does not exist" Sep 10 12:44:15.478519 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.478466 2583 scope.go:117] "RemoveContainer" containerID="5de47961c119b363012925f362165b64c927b9fe49b4067336ebc9a7cd35bd80" Sep 10 12:44:15.478686 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.478669 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5de47961c119b363012925f362165b64c927b9fe49b4067336ebc9a7cd35bd80"} err="failed to get container status \"5de47961c119b363012925f362165b64c927b9fe49b4067336ebc9a7cd35bd80\": rpc error: code = NotFound desc = could not find container \"5de47961c119b363012925f362165b64c927b9fe49b4067336ebc9a7cd35bd80\": container with ID starting with 5de47961c119b363012925f362165b64c927b9fe49b4067336ebc9a7cd35bd80 not found: ID does not exist" Sep 10 12:44:15.478736 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.478687 2583 scope.go:117] "RemoveContainer" containerID="f7468a220b9e3adac100870e32b730f437bb0019d4774a56ba1838382c18f866" Sep 10 12:44:15.478958 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:15.478938 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7468a220b9e3adac100870e32b730f437bb0019d4774a56ba1838382c18f866"} err="failed to get container status \"f7468a220b9e3adac100870e32b730f437bb0019d4774a56ba1838382c18f866\": rpc error: code = NotFound desc = could not find container \"f7468a220b9e3adac100870e32b730f437bb0019d4774a56ba1838382c18f866\": container with ID starting with f7468a220b9e3adac100870e32b730f437bb0019d4774a56ba1838382c18f866 not found: ID does not exist" Sep 10 12:44:16.468812 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:44:16.468782 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" path="/var/lib/kubelet/pods/baa02eb4-2b22-4131-a891-ccb7e30bc051/volumes" Sep 10 12:45:09.565943 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.565906 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l"] Sep 10 12:45:09.566449 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.566323 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="feast-apply" Sep 10 12:45:09.566449 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.566340 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="feast-apply" Sep 10 12:45:09.566449 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.566353 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="online" Sep 10 12:45:09.566449 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.566361 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="online" Sep 10 12:45:09.566449 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.566380 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="ui" Sep 10 12:45:09.566449 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.566389 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="ui" Sep 10 12:45:09.566449 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.566400 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="feast-init" Sep 10 12:45:09.566449 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.566408 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="feast-init" Sep 10 12:45:09.566449 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.566424 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="offline" Sep 10 12:45:09.566449 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.566432 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="offline" Sep 10 12:45:09.566449 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.566452 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="registry" Sep 10 12:45:09.566861 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.566461 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="registry" Sep 10 12:45:09.566861 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.566529 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="online" Sep 10 12:45:09.566861 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.566542 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="ui" Sep 10 12:45:09.566861 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.566552 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="offline" Sep 10 12:45:09.566861 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.566563 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="baa02eb4-2b22-4131-a891-ccb7e30bc051" containerName="registry" Sep 10 12:45:09.570331 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.570308 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:09.572729 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.572703 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 10 12:45:09.572923 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.572900 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 10 12:45:09.573078 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.573054 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 10 12:45:09.573078 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.573008 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 10 12:45:09.573236 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.573110 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 10 12:45:09.573236 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.573067 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-rj4dg\"" Sep 10 12:45:09.573236 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.573054 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 10 12:45:09.582954 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.582928 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l"] Sep 10 12:45:09.671545 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.671508 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d1441adb-8fbd-4593-898c-12106c6a8e30-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-cqx9l\" (UID: \"d1441adb-8fbd-4593-898c-12106c6a8e30\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:09.671741 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.671605 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d1441adb-8fbd-4593-898c-12106c6a8e30-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-cqx9l\" (UID: \"d1441adb-8fbd-4593-898c-12106c6a8e30\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:09.671741 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.671667 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d1441adb-8fbd-4593-898c-12106c6a8e30-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-cqx9l\" (UID: \"d1441adb-8fbd-4593-898c-12106c6a8e30\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:09.671741 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.671701 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4hfz\" (UniqueName: \"kubernetes.io/projected/d1441adb-8fbd-4593-898c-12106c6a8e30-kube-api-access-c4hfz\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-cqx9l\" (UID: \"d1441adb-8fbd-4593-898c-12106c6a8e30\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:09.671741 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.671729 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d1441adb-8fbd-4593-898c-12106c6a8e30-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-cqx9l\" (UID: \"d1441adb-8fbd-4593-898c-12106c6a8e30\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:09.672040 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.671756 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d1441adb-8fbd-4593-898c-12106c6a8e30-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-cqx9l\" (UID: \"d1441adb-8fbd-4593-898c-12106c6a8e30\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:09.773025 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.772984 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d1441adb-8fbd-4593-898c-12106c6a8e30-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-cqx9l\" (UID: \"d1441adb-8fbd-4593-898c-12106c6a8e30\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:09.773025 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.773032 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d1441adb-8fbd-4593-898c-12106c6a8e30-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-cqx9l\" (UID: \"d1441adb-8fbd-4593-898c-12106c6a8e30\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:09.773240 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.773061 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d1441adb-8fbd-4593-898c-12106c6a8e30-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-cqx9l\" (UID: \"d1441adb-8fbd-4593-898c-12106c6a8e30\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:09.773240 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.773082 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4hfz\" (UniqueName: \"kubernetes.io/projected/d1441adb-8fbd-4593-898c-12106c6a8e30-kube-api-access-c4hfz\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-cqx9l\" (UID: \"d1441adb-8fbd-4593-898c-12106c6a8e30\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:09.773240 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.773100 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d1441adb-8fbd-4593-898c-12106c6a8e30-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-cqx9l\" (UID: \"d1441adb-8fbd-4593-898c-12106c6a8e30\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:09.773240 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.773120 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d1441adb-8fbd-4593-898c-12106c6a8e30-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-cqx9l\" (UID: \"d1441adb-8fbd-4593-898c-12106c6a8e30\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:09.773562 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.773521 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d1441adb-8fbd-4593-898c-12106c6a8e30-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-cqx9l\" (UID: \"d1441adb-8fbd-4593-898c-12106c6a8e30\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:09.773905 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.773885 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d1441adb-8fbd-4593-898c-12106c6a8e30-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-cqx9l\" (UID: \"d1441adb-8fbd-4593-898c-12106c6a8e30\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:09.775565 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.775542 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d1441adb-8fbd-4593-898c-12106c6a8e30-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-cqx9l\" (UID: \"d1441adb-8fbd-4593-898c-12106c6a8e30\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:09.775662 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.775583 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d1441adb-8fbd-4593-898c-12106c6a8e30-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-cqx9l\" (UID: \"d1441adb-8fbd-4593-898c-12106c6a8e30\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:09.775662 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.775639 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d1441adb-8fbd-4593-898c-12106c6a8e30-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-cqx9l\" (UID: \"d1441adb-8fbd-4593-898c-12106c6a8e30\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:09.781804 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.781780 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4hfz\" (UniqueName: \"kubernetes.io/projected/d1441adb-8fbd-4593-898c-12106c6a8e30-kube-api-access-c4hfz\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-cqx9l\" (UID: \"d1441adb-8fbd-4593-898c-12106c6a8e30\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:09.881683 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:09.881582 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:10.156711 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:10.156639 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l"] Sep 10 12:45:10.159779 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:45:10.159743 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd1441adb_8fbd_4593_898c_12106c6a8e30.slice/crio-c045cfdd2bdb96e705df6f2e00004bdffde8f1048f43e9c5d3bb98945a7712fb WatchSource:0}: Error finding container c045cfdd2bdb96e705df6f2e00004bdffde8f1048f43e9c5d3bb98945a7712fb: Status 404 returned error can't find the container with id c045cfdd2bdb96e705df6f2e00004bdffde8f1048f43e9c5d3bb98945a7712fb Sep 10 12:45:10.637581 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:10.637543 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" event={"ID":"d1441adb-8fbd-4593-898c-12106c6a8e30","Type":"ContainerStarted","Data":"3b16aad31d9ddf4f85088c39397d794831fae5fa3be6bb70903224d44c5d869e"} Sep 10 12:45:10.638036 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:10.637592 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" event={"ID":"d1441adb-8fbd-4593-898c-12106c6a8e30","Type":"ContainerStarted","Data":"c045cfdd2bdb96e705df6f2e00004bdffde8f1048f43e9c5d3bb98945a7712fb"} Sep 10 12:45:15.657957 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:15.657918 2583 generic.go:356] "Generic (PLEG): container finished" podID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerID="3b16aad31d9ddf4f85088c39397d794831fae5fa3be6bb70903224d44c5d869e" exitCode=0 Sep 10 12:45:15.658434 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:15.657990 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" event={"ID":"d1441adb-8fbd-4593-898c-12106c6a8e30","Type":"ContainerDied","Data":"3b16aad31d9ddf4f85088c39397d794831fae5fa3be6bb70903224d44c5d869e"} Sep 10 12:45:16.664497 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:16.664463 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" event={"ID":"d1441adb-8fbd-4593-898c-12106c6a8e30","Type":"ContainerStarted","Data":"4648e107b5a6c3a40af599d5d8d3031012ab57260430e37bb46b9da37b1a8f62"} Sep 10 12:45:36.749207 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:36.749172 2583 generic.go:356] "Generic (PLEG): container finished" podID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerID="4648e107b5a6c3a40af599d5d8d3031012ab57260430e37bb46b9da37b1a8f62" exitCode=0 Sep 10 12:45:36.749587 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:36.749243 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" event={"ID":"d1441adb-8fbd-4593-898c-12106c6a8e30","Type":"ContainerDied","Data":"4648e107b5a6c3a40af599d5d8d3031012ab57260430e37bb46b9da37b1a8f62"} Sep 10 12:45:37.755930 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:37.755893 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" event={"ID":"d1441adb-8fbd-4593-898c-12106c6a8e30","Type":"ContainerStarted","Data":"c23fe8ee0d61cdb1e0af6aa671ebd4908623ecc1775ba89e38dd7e5a161bbf18"} Sep 10 12:45:37.756418 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:37.755942 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" event={"ID":"d1441adb-8fbd-4593-898c-12106c6a8e30","Type":"ContainerStarted","Data":"3a87713ab62335be1ce2eab83c55f075bae2a5198529de4db449180801dfe6d0"} Sep 10 12:45:37.756418 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:37.755957 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" event={"ID":"d1441adb-8fbd-4593-898c-12106c6a8e30","Type":"ContainerStarted","Data":"416eb610d43a60a4e73358b530e484b78ce60e197e1078778ac8158c1ca1eccd"} Sep 10 12:45:37.779409 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:37.779357 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" podStartSLOduration=28.779339573 podStartE2EDuration="28.779339573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:45:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:45:37.777936472 +0000 UTC m=+1665.783155933" watchObservedRunningTime="2026-09-10 12:45:37.779339573 +0000 UTC m=+1665.784559033" Sep 10 12:45:39.882237 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:39.882190 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:39.882641 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:39.882264 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:39.882641 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:39.882281 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:39.882641 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:39.882299 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:39.882641 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:39.882315 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:39.882641 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:39.882330 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:39.884570 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:39.884531 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.32:8443: connect: connection refused" Sep 10 12:45:39.884570 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:39.884548 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.32:8816: connect: connection refused" Sep 10 12:45:39.884767 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:39.884575 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused" Sep 10 12:45:42.882052 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:42.881998 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.32:8816: connect: connection refused" Sep 10 12:45:42.882478 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:42.882015 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.32:8443: connect: connection refused" Sep 10 12:45:42.882478 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:42.882022 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused" Sep 10 12:45:45.882811 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:45.882777 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:45.883374 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:45.883036 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:45.883551 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:45.883534 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:46.799269 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:46.799240 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:46.886999 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:46.886956 2583 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 10 12:45:49.529621 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:49.529586 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:45:50.195111 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:45:50.195073 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:46:51.760207 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:46:51.760162 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l"] Sep 10 12:46:51.760779 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:46:51.760459 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="online" containerID="cri-o://416eb610d43a60a4e73358b530e484b78ce60e197e1078778ac8158c1ca1eccd" gracePeriod=30 Sep 10 12:46:51.760779 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:46:51.760651 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="offline" containerID="cri-o://3a87713ab62335be1ce2eab83c55f075bae2a5198529de4db449180801dfe6d0" gracePeriod=30 Sep 10 12:46:51.760779 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:46:51.760655 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="ui" containerID="cri-o://c23fe8ee0d61cdb1e0af6aa671ebd4908623ecc1775ba89e38dd7e5a161bbf18" gracePeriod=30 Sep 10 12:46:51.901725 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:46:51.901679 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6"] Sep 10 12:46:51.902164 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:46:51.901956 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="registry" containerID="cri-o://f9037a2743e2cc99bb9ee342834036b7edda2ad96323ff506c4c3a56039b03c2" gracePeriod=30 Sep 10 12:46:51.902164 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:46:51.902010 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="offline" containerID="cri-o://7aacdc257fe24e88a9e7bd16bdd4b9a3e53e95217afe0d65c613dc968831b489" gracePeriod=30 Sep 10 12:46:51.902164 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:46:51.902035 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="online" containerID="cri-o://40cf400d85b84b20e9c1268cf3fb48448e8b59e73c0766ed766bb60b6f4c90a5" gracePeriod=30 Sep 10 12:46:51.902164 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:46:51.902040 2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="ui" containerID="cri-o://1b6ea7559f005c4bc4fd0e97e20171a285a6edb9faeaab923447b2b5228a00d6" gracePeriod=30 Sep 10 12:46:53.053365 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:46:53.053331 2583 generic.go:356] "Generic (PLEG): container finished" podID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerID="c23fe8ee0d61cdb1e0af6aa671ebd4908623ecc1775ba89e38dd7e5a161bbf18" exitCode=0 Sep 10 12:46:53.053365 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:46:53.053358 2583 generic.go:356] "Generic (PLEG): container finished" podID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerID="416eb610d43a60a4e73358b530e484b78ce60e197e1078778ac8158c1ca1eccd" exitCode=0 Sep 10 12:46:53.053866 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:46:53.053403 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" event={"ID":"d1441adb-8fbd-4593-898c-12106c6a8e30","Type":"ContainerDied","Data":"c23fe8ee0d61cdb1e0af6aa671ebd4908623ecc1775ba89e38dd7e5a161bbf18"} Sep 10 12:46:53.053866 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:46:53.053444 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" event={"ID":"d1441adb-8fbd-4593-898c-12106c6a8e30","Type":"ContainerDied","Data":"416eb610d43a60a4e73358b530e484b78ce60e197e1078778ac8158c1ca1eccd"} Sep 10 12:46:53.056323 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:46:53.056300 2583 generic.go:356] "Generic (PLEG): container finished" podID="f618c8bf-9256-42fc-8937-49680946bcd5" containerID="1b6ea7559f005c4bc4fd0e97e20171a285a6edb9faeaab923447b2b5228a00d6" exitCode=0 Sep 10 12:46:53.056323 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:46:53.056320 2583 generic.go:356] "Generic (PLEG): container finished" podID="f618c8bf-9256-42fc-8937-49680946bcd5" containerID="40cf400d85b84b20e9c1268cf3fb48448e8b59e73c0766ed766bb60b6f4c90a5" exitCode=0 Sep 10 12:46:53.056471 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:46:53.056344 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" event={"ID":"f618c8bf-9256-42fc-8937-49680946bcd5","Type":"ContainerDied","Data":"1b6ea7559f005c4bc4fd0e97e20171a285a6edb9faeaab923447b2b5228a00d6"} Sep 10 12:46:53.056471 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:46:53.056378 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" event={"ID":"f618c8bf-9256-42fc-8937-49680946bcd5","Type":"ContainerDied","Data":"40cf400d85b84b20e9c1268cf3fb48448e8b59e73c0766ed766bb60b6f4c90a5"} Sep 10 12:46:55.883524 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:46:55.883487 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.32:8443: connect: connection refused" Sep 10 12:46:56.307505 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:46:56.307464 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.31:8443: connect: connection refused" Sep 10 12:46:56.397636 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:46:56.397603 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": dial tcp 10.133.0.31:6567: connect: connection refused" Sep 10 12:46:59.530350 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:46:59.530306 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused" Sep 10 12:47:05.884013 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:05.883973 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.32:8443: connect: connection refused" Sep 10 12:47:06.308439 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:06.308376 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.31:8443: connect: connection refused" Sep 10 12:47:06.397227 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:06.397196 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": dial tcp 10.133.0.31:6567: connect: connection refused" Sep 10 12:47:09.530469 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:09.530433 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused" Sep 10 12:47:15.883732 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:15.883690 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.32:8443: connect: connection refused" Sep 10 12:47:15.884114 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:15.883798 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:47:16.306917 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:16.306877 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.31:8443: connect: connection refused" Sep 10 12:47:16.307104 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:16.307010 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:47:16.397621 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:16.397584 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": dial tcp 10.133.0.31:6567: connect: connection refused" Sep 10 12:47:16.397756 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:16.397697 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:47:19.530377 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:19.530337 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused" Sep 10 12:47:19.530755 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:19.530436 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:47:22.105003 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.104977 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:47:22.112341 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.112316 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f618c8bf-9256-42fc-8937-49680946bcd5-offline-tls\") pod \"f618c8bf-9256-42fc-8937-49680946bcd5\" (UID: \"f618c8bf-9256-42fc-8937-49680946bcd5\") " Sep 10 12:47:22.112463 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.112417 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f618c8bf-9256-42fc-8937-49680946bcd5-ui-tls\") pod \"f618c8bf-9256-42fc-8937-49680946bcd5\" (UID: \"f618c8bf-9256-42fc-8937-49680946bcd5\") " Sep 10 12:47:22.112463 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.112445 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f618c8bf-9256-42fc-8937-49680946bcd5-registry-tls\") pod \"f618c8bf-9256-42fc-8937-49680946bcd5\" (UID: \"f618c8bf-9256-42fc-8937-49680946bcd5\") " Sep 10 12:47:22.112579 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.112477 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9dwm\" (UniqueName: \"kubernetes.io/projected/f618c8bf-9256-42fc-8937-49680946bcd5-kube-api-access-b9dwm\") pod \"f618c8bf-9256-42fc-8937-49680946bcd5\" (UID: \"f618c8bf-9256-42fc-8937-49680946bcd5\") " Sep 10 12:47:22.112579 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.112521 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f618c8bf-9256-42fc-8937-49680946bcd5-feast-data\") pod \"f618c8bf-9256-42fc-8937-49680946bcd5\" (UID: \"f618c8bf-9256-42fc-8937-49680946bcd5\") " Sep 10 12:47:22.112579 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.112569 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f618c8bf-9256-42fc-8937-49680946bcd5-online-tls\") pod \"f618c8bf-9256-42fc-8937-49680946bcd5\" (UID: \"f618c8bf-9256-42fc-8937-49680946bcd5\") " Sep 10 12:47:22.113108 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.113083 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f618c8bf-9256-42fc-8937-49680946bcd5-feast-data" (OuterVolumeSpecName: "feast-data") pod "f618c8bf-9256-42fc-8937-49680946bcd5" (UID: "f618c8bf-9256-42fc-8937-49680946bcd5"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:47:22.114592 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.114566 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f618c8bf-9256-42fc-8937-49680946bcd5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f618c8bf-9256-42fc-8937-49680946bcd5" (UID: "f618c8bf-9256-42fc-8937-49680946bcd5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:47:22.114891 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.114865 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f618c8bf-9256-42fc-8937-49680946bcd5-kube-api-access-b9dwm" (OuterVolumeSpecName: "kube-api-access-b9dwm") pod "f618c8bf-9256-42fc-8937-49680946bcd5" (UID: "f618c8bf-9256-42fc-8937-49680946bcd5"). InnerVolumeSpecName "kube-api-access-b9dwm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:47:22.114976 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.114944 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f618c8bf-9256-42fc-8937-49680946bcd5-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "f618c8bf-9256-42fc-8937-49680946bcd5" (UID: "f618c8bf-9256-42fc-8937-49680946bcd5"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:47:22.114976 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.114943 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f618c8bf-9256-42fc-8937-49680946bcd5-online-tls" (OuterVolumeSpecName: "online-tls") pod "f618c8bf-9256-42fc-8937-49680946bcd5" (UID: "f618c8bf-9256-42fc-8937-49680946bcd5"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:47:22.114976 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.114965 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f618c8bf-9256-42fc-8937-49680946bcd5-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "f618c8bf-9256-42fc-8937-49680946bcd5" (UID: "f618c8bf-9256-42fc-8937-49680946bcd5"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:47:22.169106 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.169080 2583 generic.go:356] "Generic (PLEG): container finished" podID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerID="3a87713ab62335be1ce2eab83c55f075bae2a5198529de4db449180801dfe6d0" exitCode=137 Sep 10 12:47:22.169234 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.169156 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" event={"ID":"d1441adb-8fbd-4593-898c-12106c6a8e30","Type":"ContainerDied","Data":"3a87713ab62335be1ce2eab83c55f075bae2a5198529de4db449180801dfe6d0"} Sep 10 12:47:22.171996 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.171964 2583 generic.go:356] "Generic (PLEG): container finished" podID="f618c8bf-9256-42fc-8937-49680946bcd5" containerID="7aacdc257fe24e88a9e7bd16bdd4b9a3e53e95217afe0d65c613dc968831b489" exitCode=137 Sep 10 12:47:22.171996 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.171983 2583 generic.go:356] "Generic (PLEG): container finished" podID="f618c8bf-9256-42fc-8937-49680946bcd5" containerID="f9037a2743e2cc99bb9ee342834036b7edda2ad96323ff506c4c3a56039b03c2" exitCode=137 Sep 10 12:47:22.172147 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.172012 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" event={"ID":"f618c8bf-9256-42fc-8937-49680946bcd5","Type":"ContainerDied","Data":"7aacdc257fe24e88a9e7bd16bdd4b9a3e53e95217afe0d65c613dc968831b489"} Sep 10 12:47:22.172147 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.172048 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" event={"ID":"f618c8bf-9256-42fc-8937-49680946bcd5","Type":"ContainerDied","Data":"f9037a2743e2cc99bb9ee342834036b7edda2ad96323ff506c4c3a56039b03c2"} Sep 10 12:47:22.172147 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.172054 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" Sep 10 12:47:22.172147 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.172063 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6" event={"ID":"f618c8bf-9256-42fc-8937-49680946bcd5","Type":"ContainerDied","Data":"1e184b3de9fa28978af5ec3bbebaeba36d8bd349d2deba868a10399605f20073"} Sep 10 12:47:22.172147 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.172089 2583 scope.go:117] "RemoveContainer" containerID="1b6ea7559f005c4bc4fd0e97e20171a285a6edb9faeaab923447b2b5228a00d6" Sep 10 12:47:22.180008 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.179991 2583 scope.go:117] "RemoveContainer" containerID="7aacdc257fe24e88a9e7bd16bdd4b9a3e53e95217afe0d65c613dc968831b489" Sep 10 12:47:22.187522 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.187502 2583 scope.go:117] "RemoveContainer" containerID="40cf400d85b84b20e9c1268cf3fb48448e8b59e73c0766ed766bb60b6f4c90a5" Sep 10 12:47:22.191660 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.191639 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6"] Sep 10 12:47:22.194331 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.194311 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-8d6r6"] Sep 10 12:47:22.195185 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.195161 2583 scope.go:117] "RemoveContainer" containerID="f9037a2743e2cc99bb9ee342834036b7edda2ad96323ff506c4c3a56039b03c2" Sep 10 12:47:22.202121 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.202098 2583 scope.go:117] "RemoveContainer" containerID="4228ca52ccff209cbf6744901677de6020712d4608157617425cc90e60ad5fc2" Sep 10 12:47:22.208750 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.208733 2583 scope.go:117] "RemoveContainer" containerID="56145c218b2a28b86c74911881fd5121a9ea236ded0bdea8a27a9601d0bce9e0" Sep 10 12:47:22.213888 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.213868 2583 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f618c8bf-9256-42fc-8937-49680946bcd5-offline-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:47:22.213985 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.213890 2583 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f618c8bf-9256-42fc-8937-49680946bcd5-ui-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:47:22.213985 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.213903 2583 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f618c8bf-9256-42fc-8937-49680946bcd5-registry-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:47:22.213985 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.213915 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9dwm\" (UniqueName: \"kubernetes.io/projected/f618c8bf-9256-42fc-8937-49680946bcd5-kube-api-access-b9dwm\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:47:22.213985 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.213927 2583 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f618c8bf-9256-42fc-8937-49680946bcd5-feast-data\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:47:22.213985 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.213940 2583 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f618c8bf-9256-42fc-8937-49680946bcd5-online-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:47:22.215215 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.215197 2583 scope.go:117] "RemoveContainer" containerID="1b6ea7559f005c4bc4fd0e97e20171a285a6edb9faeaab923447b2b5228a00d6" Sep 10 12:47:22.215467 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:47:22.215450 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b6ea7559f005c4bc4fd0e97e20171a285a6edb9faeaab923447b2b5228a00d6\": container with ID starting with 1b6ea7559f005c4bc4fd0e97e20171a285a6edb9faeaab923447b2b5228a00d6 not found: ID does not exist" containerID="1b6ea7559f005c4bc4fd0e97e20171a285a6edb9faeaab923447b2b5228a00d6" Sep 10 12:47:22.215515 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.215477 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b6ea7559f005c4bc4fd0e97e20171a285a6edb9faeaab923447b2b5228a00d6"} err="failed to get container status \"1b6ea7559f005c4bc4fd0e97e20171a285a6edb9faeaab923447b2b5228a00d6\": rpc error: code = NotFound desc = could not find container \"1b6ea7559f005c4bc4fd0e97e20171a285a6edb9faeaab923447b2b5228a00d6\": container with ID starting with 1b6ea7559f005c4bc4fd0e97e20171a285a6edb9faeaab923447b2b5228a00d6 not found: ID does not exist" Sep 10 12:47:22.215515 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.215495 2583 scope.go:117] "RemoveContainer" containerID="7aacdc257fe24e88a9e7bd16bdd4b9a3e53e95217afe0d65c613dc968831b489" Sep 10 12:47:22.215766 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:47:22.215741 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7aacdc257fe24e88a9e7bd16bdd4b9a3e53e95217afe0d65c613dc968831b489\": container with ID starting with 7aacdc257fe24e88a9e7bd16bdd4b9a3e53e95217afe0d65c613dc968831b489 not found: ID does not exist" containerID="7aacdc257fe24e88a9e7bd16bdd4b9a3e53e95217afe0d65c613dc968831b489" Sep 10 12:47:22.215823 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.215774 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7aacdc257fe24e88a9e7bd16bdd4b9a3e53e95217afe0d65c613dc968831b489"} err="failed to get container status \"7aacdc257fe24e88a9e7bd16bdd4b9a3e53e95217afe0d65c613dc968831b489\": rpc error: code = NotFound desc = could not find container \"7aacdc257fe24e88a9e7bd16bdd4b9a3e53e95217afe0d65c613dc968831b489\": container with ID starting with 7aacdc257fe24e88a9e7bd16bdd4b9a3e53e95217afe0d65c613dc968831b489 not found: ID does not exist" Sep 10 12:47:22.215823 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.215791 2583 scope.go:117] "RemoveContainer" containerID="40cf400d85b84b20e9c1268cf3fb48448e8b59e73c0766ed766bb60b6f4c90a5" Sep 10 12:47:22.216043 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:47:22.216026 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40cf400d85b84b20e9c1268cf3fb48448e8b59e73c0766ed766bb60b6f4c90a5\": container with ID starting with 40cf400d85b84b20e9c1268cf3fb48448e8b59e73c0766ed766bb60b6f4c90a5 not found: ID does not exist" containerID="40cf400d85b84b20e9c1268cf3fb48448e8b59e73c0766ed766bb60b6f4c90a5" Sep 10 12:47:22.216082 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.216048 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40cf400d85b84b20e9c1268cf3fb48448e8b59e73c0766ed766bb60b6f4c90a5"} err="failed to get container status \"40cf400d85b84b20e9c1268cf3fb48448e8b59e73c0766ed766bb60b6f4c90a5\": rpc error: code = NotFound desc = could not find container \"40cf400d85b84b20e9c1268cf3fb48448e8b59e73c0766ed766bb60b6f4c90a5\": container with ID starting with 40cf400d85b84b20e9c1268cf3fb48448e8b59e73c0766ed766bb60b6f4c90a5 not found: ID does not exist" Sep 10 12:47:22.216082 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.216063 2583 scope.go:117] "RemoveContainer" containerID="f9037a2743e2cc99bb9ee342834036b7edda2ad96323ff506c4c3a56039b03c2" Sep 10 12:47:22.216305 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:47:22.216289 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9037a2743e2cc99bb9ee342834036b7edda2ad96323ff506c4c3a56039b03c2\": container with ID starting with f9037a2743e2cc99bb9ee342834036b7edda2ad96323ff506c4c3a56039b03c2 not found: ID does not exist" containerID="f9037a2743e2cc99bb9ee342834036b7edda2ad96323ff506c4c3a56039b03c2" Sep 10 12:47:22.216365 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.216307 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9037a2743e2cc99bb9ee342834036b7edda2ad96323ff506c4c3a56039b03c2"} err="failed to get container status \"f9037a2743e2cc99bb9ee342834036b7edda2ad96323ff506c4c3a56039b03c2\": rpc error: code = NotFound desc = could not find container \"f9037a2743e2cc99bb9ee342834036b7edda2ad96323ff506c4c3a56039b03c2\": container with ID starting with f9037a2743e2cc99bb9ee342834036b7edda2ad96323ff506c4c3a56039b03c2 not found: ID does not exist" Sep 10 12:47:22.216365 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.216323 2583 scope.go:117] "RemoveContainer" containerID="4228ca52ccff209cbf6744901677de6020712d4608157617425cc90e60ad5fc2" Sep 10 12:47:22.216572 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:47:22.216556 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4228ca52ccff209cbf6744901677de6020712d4608157617425cc90e60ad5fc2\": container with ID starting with 4228ca52ccff209cbf6744901677de6020712d4608157617425cc90e60ad5fc2 not found: ID does not exist" containerID="4228ca52ccff209cbf6744901677de6020712d4608157617425cc90e60ad5fc2" Sep 10 12:47:22.216613 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.216577 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4228ca52ccff209cbf6744901677de6020712d4608157617425cc90e60ad5fc2"} err="failed to get container status \"4228ca52ccff209cbf6744901677de6020712d4608157617425cc90e60ad5fc2\": rpc error: code = NotFound desc = could not find container \"4228ca52ccff209cbf6744901677de6020712d4608157617425cc90e60ad5fc2\": container with ID starting with 4228ca52ccff209cbf6744901677de6020712d4608157617425cc90e60ad5fc2 not found: ID does not exist" Sep 10 12:47:22.216613 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.216590 2583 scope.go:117] "RemoveContainer" containerID="56145c218b2a28b86c74911881fd5121a9ea236ded0bdea8a27a9601d0bce9e0" Sep 10 12:47:22.216832 ip-10-0-143-110 kubenswrapper[2583]: E0910 12:47:22.216815 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56145c218b2a28b86c74911881fd5121a9ea236ded0bdea8a27a9601d0bce9e0\": container with ID starting with 56145c218b2a28b86c74911881fd5121a9ea236ded0bdea8a27a9601d0bce9e0 not found: ID does not exist" containerID="56145c218b2a28b86c74911881fd5121a9ea236ded0bdea8a27a9601d0bce9e0" Sep 10 12:47:22.216972 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.216856 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56145c218b2a28b86c74911881fd5121a9ea236ded0bdea8a27a9601d0bce9e0"} err="failed to get container status \"56145c218b2a28b86c74911881fd5121a9ea236ded0bdea8a27a9601d0bce9e0\": rpc error: code = NotFound desc = could not find container \"56145c218b2a28b86c74911881fd5121a9ea236ded0bdea8a27a9601d0bce9e0\": container with ID starting with 56145c218b2a28b86c74911881fd5121a9ea236ded0bdea8a27a9601d0bce9e0 not found: ID does not exist" Sep 10 12:47:22.216972 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.216871 2583 scope.go:117] "RemoveContainer" containerID="1b6ea7559f005c4bc4fd0e97e20171a285a6edb9faeaab923447b2b5228a00d6" Sep 10 12:47:22.217148 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.217130 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b6ea7559f005c4bc4fd0e97e20171a285a6edb9faeaab923447b2b5228a00d6"} err="failed to get container status \"1b6ea7559f005c4bc4fd0e97e20171a285a6edb9faeaab923447b2b5228a00d6\": rpc error: code = NotFound desc = could not find container \"1b6ea7559f005c4bc4fd0e97e20171a285a6edb9faeaab923447b2b5228a00d6\": container with ID starting with 1b6ea7559f005c4bc4fd0e97e20171a285a6edb9faeaab923447b2b5228a00d6 not found: ID does not exist" Sep 10 12:47:22.217191 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.217150 2583 scope.go:117] "RemoveContainer" containerID="7aacdc257fe24e88a9e7bd16bdd4b9a3e53e95217afe0d65c613dc968831b489" Sep 10 12:47:22.217382 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.217362 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7aacdc257fe24e88a9e7bd16bdd4b9a3e53e95217afe0d65c613dc968831b489"} err="failed to get container status \"7aacdc257fe24e88a9e7bd16bdd4b9a3e53e95217afe0d65c613dc968831b489\": rpc error: code = NotFound desc = could not find container \"7aacdc257fe24e88a9e7bd16bdd4b9a3e53e95217afe0d65c613dc968831b489\": container with ID starting with 7aacdc257fe24e88a9e7bd16bdd4b9a3e53e95217afe0d65c613dc968831b489 not found: ID does not exist" Sep 10 12:47:22.217426 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.217383 2583 scope.go:117] "RemoveContainer" containerID="40cf400d85b84b20e9c1268cf3fb48448e8b59e73c0766ed766bb60b6f4c90a5" Sep 10 12:47:22.217586 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.217570 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40cf400d85b84b20e9c1268cf3fb48448e8b59e73c0766ed766bb60b6f4c90a5"} err="failed to get container status \"40cf400d85b84b20e9c1268cf3fb48448e8b59e73c0766ed766bb60b6f4c90a5\": rpc error: code = NotFound desc = could not find container \"40cf400d85b84b20e9c1268cf3fb48448e8b59e73c0766ed766bb60b6f4c90a5\": container with ID starting with 40cf400d85b84b20e9c1268cf3fb48448e8b59e73c0766ed766bb60b6f4c90a5 not found: ID does not exist" Sep 10 12:47:22.217634 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.217586 2583 scope.go:117] "RemoveContainer" containerID="f9037a2743e2cc99bb9ee342834036b7edda2ad96323ff506c4c3a56039b03c2" Sep 10 12:47:22.217813 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.217796 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9037a2743e2cc99bb9ee342834036b7edda2ad96323ff506c4c3a56039b03c2"} err="failed to get container status \"f9037a2743e2cc99bb9ee342834036b7edda2ad96323ff506c4c3a56039b03c2\": rpc error: code = NotFound desc = could not find container \"f9037a2743e2cc99bb9ee342834036b7edda2ad96323ff506c4c3a56039b03c2\": container with ID starting with f9037a2743e2cc99bb9ee342834036b7edda2ad96323ff506c4c3a56039b03c2 not found: ID does not exist" Sep 10 12:47:22.217813 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.217812 2583 scope.go:117] "RemoveContainer" containerID="4228ca52ccff209cbf6744901677de6020712d4608157617425cc90e60ad5fc2" Sep 10 12:47:22.218054 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.218032 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4228ca52ccff209cbf6744901677de6020712d4608157617425cc90e60ad5fc2"} err="failed to get container status \"4228ca52ccff209cbf6744901677de6020712d4608157617425cc90e60ad5fc2\": rpc error: code = NotFound desc = could not find container \"4228ca52ccff209cbf6744901677de6020712d4608157617425cc90e60ad5fc2\": container with ID starting with 4228ca52ccff209cbf6744901677de6020712d4608157617425cc90e60ad5fc2 not found: ID does not exist" Sep 10 12:47:22.218098 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.218058 2583 scope.go:117] "RemoveContainer" containerID="56145c218b2a28b86c74911881fd5121a9ea236ded0bdea8a27a9601d0bce9e0" Sep 10 12:47:22.218325 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.218307 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56145c218b2a28b86c74911881fd5121a9ea236ded0bdea8a27a9601d0bce9e0"} err="failed to get container status \"56145c218b2a28b86c74911881fd5121a9ea236ded0bdea8a27a9601d0bce9e0\": rpc error: code = NotFound desc = could not find container \"56145c218b2a28b86c74911881fd5121a9ea236ded0bdea8a27a9601d0bce9e0\": container with ID starting with 56145c218b2a28b86c74911881fd5121a9ea236ded0bdea8a27a9601d0bce9e0 not found: ID does not exist" Sep 10 12:47:22.389487 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.389466 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:47:22.415748 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.415719 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d1441adb-8fbd-4593-898c-12106c6a8e30-ui-tls\") pod \"d1441adb-8fbd-4593-898c-12106c6a8e30\" (UID: \"d1441adb-8fbd-4593-898c-12106c6a8e30\") " Sep 10 12:47:22.415941 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.415770 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d1441adb-8fbd-4593-898c-12106c6a8e30-registry-tls\") pod \"d1441adb-8fbd-4593-898c-12106c6a8e30\" (UID: \"d1441adb-8fbd-4593-898c-12106c6a8e30\") " Sep 10 12:47:22.416109 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.415832 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d1441adb-8fbd-4593-898c-12106c6a8e30-online-tls\") pod \"d1441adb-8fbd-4593-898c-12106c6a8e30\" (UID: \"d1441adb-8fbd-4593-898c-12106c6a8e30\") " Sep 10 12:47:22.416182 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.416158 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4hfz\" (UniqueName: \"kubernetes.io/projected/d1441adb-8fbd-4593-898c-12106c6a8e30-kube-api-access-c4hfz\") pod \"d1441adb-8fbd-4593-898c-12106c6a8e30\" (UID: \"d1441adb-8fbd-4593-898c-12106c6a8e30\") " Sep 10 12:47:22.416314 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.416207 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d1441adb-8fbd-4593-898c-12106c6a8e30-offline-tls\") pod \"d1441adb-8fbd-4593-898c-12106c6a8e30\" (UID: \"d1441adb-8fbd-4593-898c-12106c6a8e30\") " Sep 10 12:47:22.416314 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.416157 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1441adb-8fbd-4593-898c-12106c6a8e30-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d1441adb-8fbd-4593-898c-12106c6a8e30" (UID: "d1441adb-8fbd-4593-898c-12106c6a8e30"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:47:22.416314 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.416244 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d1441adb-8fbd-4593-898c-12106c6a8e30-feast-data\") pod \"d1441adb-8fbd-4593-898c-12106c6a8e30\" (UID: \"d1441adb-8fbd-4593-898c-12106c6a8e30\") " Sep 10 12:47:22.416485 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.416458 2583 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d1441adb-8fbd-4593-898c-12106c6a8e30-registry-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:47:22.417032 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.417000 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1441adb-8fbd-4593-898c-12106c6a8e30-feast-data" (OuterVolumeSpecName: "feast-data") pod "d1441adb-8fbd-4593-898c-12106c6a8e30" (UID: "d1441adb-8fbd-4593-898c-12106c6a8e30"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:47:22.418332 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.418306 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1441adb-8fbd-4593-898c-12106c6a8e30-kube-api-access-c4hfz" (OuterVolumeSpecName: "kube-api-access-c4hfz") pod "d1441adb-8fbd-4593-898c-12106c6a8e30" (UID: "d1441adb-8fbd-4593-898c-12106c6a8e30"). InnerVolumeSpecName "kube-api-access-c4hfz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:47:22.418455 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.418384 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1441adb-8fbd-4593-898c-12106c6a8e30-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "d1441adb-8fbd-4593-898c-12106c6a8e30" (UID: "d1441adb-8fbd-4593-898c-12106c6a8e30"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:47:22.418455 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.418427 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1441adb-8fbd-4593-898c-12106c6a8e30-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "d1441adb-8fbd-4593-898c-12106c6a8e30" (UID: "d1441adb-8fbd-4593-898c-12106c6a8e30"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:47:22.418455 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.418445 2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1441adb-8fbd-4593-898c-12106c6a8e30-online-tls" (OuterVolumeSpecName: "online-tls") pod "d1441adb-8fbd-4593-898c-12106c6a8e30" (UID: "d1441adb-8fbd-4593-898c-12106c6a8e30"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:47:22.469305 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.469278 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" path="/var/lib/kubelet/pods/f618c8bf-9256-42fc-8937-49680946bcd5/volumes" Sep 10 12:47:22.517713 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.517687 2583 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d1441adb-8fbd-4593-898c-12106c6a8e30-ui-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:47:22.517713 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.517709 2583 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d1441adb-8fbd-4593-898c-12106c6a8e30-online-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:47:22.517875 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.517719 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4hfz\" (UniqueName: \"kubernetes.io/projected/d1441adb-8fbd-4593-898c-12106c6a8e30-kube-api-access-c4hfz\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:47:22.517875 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.517729 2583 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d1441adb-8fbd-4593-898c-12106c6a8e30-offline-tls\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:47:22.517875 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:22.517739 2583 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d1441adb-8fbd-4593-898c-12106c6a8e30-feast-data\") on node \"ip-10-0-143-110.ec2.internal\" DevicePath \"\"" Sep 10 12:47:23.179208 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:23.179173 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" event={"ID":"d1441adb-8fbd-4593-898c-12106c6a8e30","Type":"ContainerDied","Data":"c045cfdd2bdb96e705df6f2e00004bdffde8f1048f43e9c5d3bb98945a7712fb"} Sep 10 12:47:23.179628 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:23.179233 2583 scope.go:117] "RemoveContainer" containerID="c23fe8ee0d61cdb1e0af6aa671ebd4908623ecc1775ba89e38dd7e5a161bbf18" Sep 10 12:47:23.179628 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:23.179184 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l" Sep 10 12:47:23.188831 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:23.188810 2583 scope.go:117] "RemoveContainer" containerID="3a87713ab62335be1ce2eab83c55f075bae2a5198529de4db449180801dfe6d0" Sep 10 12:47:23.196280 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:23.196261 2583 scope.go:117] "RemoveContainer" containerID="416eb610d43a60a4e73358b530e484b78ce60e197e1078778ac8158c1ca1eccd" Sep 10 12:47:23.196453 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:23.196431 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l"] Sep 10 12:47:23.200834 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:23.200806 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-cqx9l"] Sep 10 12:47:23.203831 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:23.203810 2583 scope.go:117] "RemoveContainer" containerID="4648e107b5a6c3a40af599d5d8d3031012ab57260430e37bb46b9da37b1a8f62" Sep 10 12:47:23.210627 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:23.210613 2583 scope.go:117] "RemoveContainer" containerID="3b16aad31d9ddf4f85088c39397d794831fae5fa3be6bb70903224d44c5d869e" Sep 10 12:47:24.469474 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:24.469439 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" path="/var/lib/kubelet/pods/d1441adb-8fbd-4593-898c-12106c6a8e30/volumes" Sep 10 12:47:52.465364 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:52.465330 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/ovn-acl-logging/0.log" Sep 10 12:47:52.469947 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:52.469926 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/ovn-acl-logging/0.log" Sep 10 12:47:58.223352 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223312 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dxwh5/must-gather-jlfgq"] Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223570 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="feast-apply" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223582 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="feast-apply" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223593 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="offline" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223598 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="offline" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223606 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="feast-apply" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223612 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="feast-apply" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223620 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="ui" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223625 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="ui" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223633 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="feast-init" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223638 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="feast-init" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223644 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="offline" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223649 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="offline" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223654 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="feast-init" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223659 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="feast-init" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223668 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="ui" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223672 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="ui" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223678 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="registry" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223683 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="registry" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223688 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="online" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223693 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="online" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223698 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="online" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223703 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="online" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223742 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="offline" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223750 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="ui" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223757 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="registry" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223763 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="online" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223769 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="offline" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223774 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1441adb-8fbd-4593-898c-12106c6a8e30" containerName="ui" Sep 10 12:47:58.223874 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.223779 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f618c8bf-9256-42fc-8937-49680946bcd5" containerName="online" Sep 10 12:47:58.226516 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.226500 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dxwh5/must-gather-jlfgq" Sep 10 12:47:58.229084 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.229062 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dxwh5\"/\"openshift-service-ca.crt\"" Sep 10 12:47:58.229210 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.229064 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dxwh5\"/\"kube-root-ca.crt\"" Sep 10 12:47:58.229737 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.229713 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dxwh5\"/\"default-dockercfg-dhffc\"" Sep 10 12:47:58.234831 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.234810 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dxwh5/must-gather-jlfgq"] Sep 10 12:47:58.378650 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.378603 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww72w\" (UniqueName: \"kubernetes.io/projected/16787e54-3e79-43d8-bc4c-a73c8ed47248-kube-api-access-ww72w\") pod \"must-gather-jlfgq\" (UID: \"16787e54-3e79-43d8-bc4c-a73c8ed47248\") " pod="openshift-must-gather-dxwh5/must-gather-jlfgq" Sep 10 12:47:58.378815 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.378658 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/16787e54-3e79-43d8-bc4c-a73c8ed47248-must-gather-output\") pod \"must-gather-jlfgq\" (UID: \"16787e54-3e79-43d8-bc4c-a73c8ed47248\") " pod="openshift-must-gather-dxwh5/must-gather-jlfgq" Sep 10 12:47:58.479772 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.479702 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww72w\" (UniqueName: \"kubernetes.io/projected/16787e54-3e79-43d8-bc4c-a73c8ed47248-kube-api-access-ww72w\") pod \"must-gather-jlfgq\" (UID: \"16787e54-3e79-43d8-bc4c-a73c8ed47248\") " pod="openshift-must-gather-dxwh5/must-gather-jlfgq" Sep 10 12:47:58.479772 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.479736 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/16787e54-3e79-43d8-bc4c-a73c8ed47248-must-gather-output\") pod \"must-gather-jlfgq\" (UID: \"16787e54-3e79-43d8-bc4c-a73c8ed47248\") " pod="openshift-must-gather-dxwh5/must-gather-jlfgq" Sep 10 12:47:58.480106 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.480087 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/16787e54-3e79-43d8-bc4c-a73c8ed47248-must-gather-output\") pod \"must-gather-jlfgq\" (UID: \"16787e54-3e79-43d8-bc4c-a73c8ed47248\") " pod="openshift-must-gather-dxwh5/must-gather-jlfgq" Sep 10 12:47:58.488342 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.488318 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww72w\" (UniqueName: \"kubernetes.io/projected/16787e54-3e79-43d8-bc4c-a73c8ed47248-kube-api-access-ww72w\") pod \"must-gather-jlfgq\" (UID: \"16787e54-3e79-43d8-bc4c-a73c8ed47248\") " pod="openshift-must-gather-dxwh5/must-gather-jlfgq" Sep 10 12:47:58.545713 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.545687 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dxwh5/must-gather-jlfgq" Sep 10 12:47:58.811960 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.811930 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dxwh5/must-gather-jlfgq"] Sep 10 12:47:58.815234 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:47:58.815203 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16787e54_3e79_43d8_bc4c_a73c8ed47248.slice/crio-5e514e81869fc9e296391ec444fdaca33f89e727174da46a692115efeea18296 WatchSource:0}: Error finding container 5e514e81869fc9e296391ec444fdaca33f89e727174da46a692115efeea18296: Status 404 returned error can't find the container with id 5e514e81869fc9e296391ec444fdaca33f89e727174da46a692115efeea18296 Sep 10 12:47:58.817338 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:58.817316 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 12:47:59.310028 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:47:59.309995 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dxwh5/must-gather-jlfgq" event={"ID":"16787e54-3e79-43d8-bc4c-a73c8ed47248","Type":"ContainerStarted","Data":"5e514e81869fc9e296391ec444fdaca33f89e727174da46a692115efeea18296"} Sep 10 12:48:00.317097 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:00.317053 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dxwh5/must-gather-jlfgq" event={"ID":"16787e54-3e79-43d8-bc4c-a73c8ed47248","Type":"ContainerStarted","Data":"320c79b28bf53224b4cff52030f6644371891900743ad1f77f64920f6ada554d"} Sep 10 12:48:00.317578 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:00.317114 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dxwh5/must-gather-jlfgq" event={"ID":"16787e54-3e79-43d8-bc4c-a73c8ed47248","Type":"ContainerStarted","Data":"0cc1b509281766c891a14111a6fb71e7f9ee57d4e9e08f4a924f1819d2caf8af"} Sep 10 12:48:00.335051 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:00.334998 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dxwh5/must-gather-jlfgq" podStartSLOduration=1.543924642 podStartE2EDuration="2.334981795s" totalImagesPullingTime="791.057153ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:47:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:48:00.333916662 +0000 UTC m=+1808.339136121" watchObservedRunningTime="2026-09-10 12:48:00.334981795 +0000 UTC m=+1808.340201264" Sep 10 12:48:00.942699 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:00.942666 2583 ???:1] "http2: server: error reading preface from client 10.0.143.110:57108: read tcp 10.0.143.110:10250->10.0.143.110:57108: read: connection reset by peer" Sep 10 12:48:00.947218 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:00.947190 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dgddq_6960f97e-f623-4368-b27a-41d81ac1be1f/global-pull-secret-syncer/0.log" Sep 10 12:48:01.056724 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:01.056693 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qkhpj_57d3f064-15f5-4771-b729-cdae4f1ae208/konnectivity-agent/0.log" Sep 10 12:48:01.149851 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:01.149767 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-110.ec2.internal_abdf679f5f7e1ecb5f64c1178ad16227/haproxy/0.log" Sep 10 12:48:05.185510 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:05.185429 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-cq4k5_a6364599-ee05-41d5-a75c-c9fae216b75d/cluster-monitoring-operator/0.log" Sep 10 12:48:05.430881 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:05.430834 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c7rdt_8c893810-45a6-4fe5-a004-32668d671629/node-exporter/0.log" Sep 10 12:48:05.454261 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:05.454189 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c7rdt_8c893810-45a6-4fe5-a004-32668d671629/kube-rbac-proxy/0.log" Sep 10 12:48:05.475118 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:05.475076 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c7rdt_8c893810-45a6-4fe5-a004-32668d671629/init-textfile/0.log" Sep 10 12:48:08.034572 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:08.034545 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-86d4f87cf6-5gs99_c82f3625-b939-4036-9c71-5bc21b83f685/console/0.log" Sep 10 12:48:08.066698 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:08.066645 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-zscmp_bfa6a4f9-3186-4067-961d-4b9130110942/download-server/0.log" Sep 10 12:48:08.499465 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:08.499424 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96"] Sep 10 12:48:08.505513 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:08.505483 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96" Sep 10 12:48:08.512785 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:08.512753 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96"] Sep 10 12:48:08.676993 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:08.676961 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/90301cc9-5e21-4a5c-aad7-76cd4de8f784-podres\") pod \"perf-node-gather-daemonset-59f96\" (UID: \"90301cc9-5e21-4a5c-aad7-76cd4de8f784\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96" Sep 10 12:48:08.677173 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:08.676998 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/90301cc9-5e21-4a5c-aad7-76cd4de8f784-proc\") pod \"perf-node-gather-daemonset-59f96\" (UID: \"90301cc9-5e21-4a5c-aad7-76cd4de8f784\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96" Sep 10 12:48:08.677173 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:08.677097 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90301cc9-5e21-4a5c-aad7-76cd4de8f784-lib-modules\") pod \"perf-node-gather-daemonset-59f96\" (UID: \"90301cc9-5e21-4a5c-aad7-76cd4de8f784\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96" Sep 10 12:48:08.677173 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:08.677124 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90301cc9-5e21-4a5c-aad7-76cd4de8f784-sys\") pod \"perf-node-gather-daemonset-59f96\" (UID: \"90301cc9-5e21-4a5c-aad7-76cd4de8f784\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96" Sep 10 12:48:08.677173 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:08.677142 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q772\" (UniqueName: \"kubernetes.io/projected/90301cc9-5e21-4a5c-aad7-76cd4de8f784-kube-api-access-6q772\") pod \"perf-node-gather-daemonset-59f96\" (UID: \"90301cc9-5e21-4a5c-aad7-76cd4de8f784\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96" Sep 10 12:48:08.777947 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:08.777862 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90301cc9-5e21-4a5c-aad7-76cd4de8f784-lib-modules\") pod \"perf-node-gather-daemonset-59f96\" (UID: \"90301cc9-5e21-4a5c-aad7-76cd4de8f784\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96" Sep 10 12:48:08.777947 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:08.777900 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90301cc9-5e21-4a5c-aad7-76cd4de8f784-sys\") pod \"perf-node-gather-daemonset-59f96\" (UID: \"90301cc9-5e21-4a5c-aad7-76cd4de8f784\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96" Sep 10 12:48:08.777947 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:08.777926 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q772\" (UniqueName: \"kubernetes.io/projected/90301cc9-5e21-4a5c-aad7-76cd4de8f784-kube-api-access-6q772\") pod \"perf-node-gather-daemonset-59f96\" (UID: \"90301cc9-5e21-4a5c-aad7-76cd4de8f784\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96" Sep 10 12:48:08.778227 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:08.777968 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/90301cc9-5e21-4a5c-aad7-76cd4de8f784-podres\") pod \"perf-node-gather-daemonset-59f96\" (UID: \"90301cc9-5e21-4a5c-aad7-76cd4de8f784\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96" Sep 10 12:48:08.778227 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:08.777992 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/90301cc9-5e21-4a5c-aad7-76cd4de8f784-proc\") pod \"perf-node-gather-daemonset-59f96\" (UID: \"90301cc9-5e21-4a5c-aad7-76cd4de8f784\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96" Sep 10 12:48:08.778227 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:08.777997 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90301cc9-5e21-4a5c-aad7-76cd4de8f784-sys\") pod \"perf-node-gather-daemonset-59f96\" (UID: \"90301cc9-5e21-4a5c-aad7-76cd4de8f784\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96" Sep 10 12:48:08.778227 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:08.778042 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90301cc9-5e21-4a5c-aad7-76cd4de8f784-lib-modules\") pod \"perf-node-gather-daemonset-59f96\" (UID: \"90301cc9-5e21-4a5c-aad7-76cd4de8f784\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96" Sep 10 12:48:08.778227 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:08.778070 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/90301cc9-5e21-4a5c-aad7-76cd4de8f784-proc\") pod \"perf-node-gather-daemonset-59f96\" (UID: \"90301cc9-5e21-4a5c-aad7-76cd4de8f784\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96" Sep 10 12:48:08.778227 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:08.778102 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/90301cc9-5e21-4a5c-aad7-76cd4de8f784-podres\") pod \"perf-node-gather-daemonset-59f96\" (UID: \"90301cc9-5e21-4a5c-aad7-76cd4de8f784\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96" Sep 10 12:48:08.787664 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:08.787639 2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q772\" (UniqueName: \"kubernetes.io/projected/90301cc9-5e21-4a5c-aad7-76cd4de8f784-kube-api-access-6q772\") pod \"perf-node-gather-daemonset-59f96\" (UID: \"90301cc9-5e21-4a5c-aad7-76cd4de8f784\") " pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96" Sep 10 12:48:08.820075 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:08.819849 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96" Sep 10 12:48:09.097582 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:09.097555 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96"] Sep 10 12:48:09.099939 ip-10-0-143-110 kubenswrapper[2583]: W0910 12:48:09.099910 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod90301cc9_5e21_4a5c_aad7_76cd4de8f784.slice/crio-47b374040532b62304606e444686257c4d126a303c279ef7045a6983d56ba3ae WatchSource:0}: Error finding container 47b374040532b62304606e444686257c4d126a303c279ef7045a6983d56ba3ae: Status 404 returned error can't find the container with id 47b374040532b62304606e444686257c4d126a303c279ef7045a6983d56ba3ae Sep 10 12:48:09.369405 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:09.369325 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96" event={"ID":"90301cc9-5e21-4a5c-aad7-76cd4de8f784","Type":"ContainerStarted","Data":"1513f00ee2d9ee0504d07672221b61f8144ecc96e76969d1bd819d14ba0de625"} Sep 10 12:48:09.369405 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:09.369375 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96" event={"ID":"90301cc9-5e21-4a5c-aad7-76cd4de8f784","Type":"ContainerStarted","Data":"47b374040532b62304606e444686257c4d126a303c279ef7045a6983d56ba3ae"} Sep 10 12:48:09.369574 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:09.369407 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96" Sep 10 12:48:09.386888 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:09.386824 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96" podStartSLOduration=1.3868095550000001 podStartE2EDuration="1.386809555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:48:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:48:09.385822755 +0000 UTC m=+1817.391042215" watchObservedRunningTime="2026-09-10 12:48:09.386809555 +0000 UTC m=+1817.392029016" Sep 10 12:48:09.409621 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:09.409603 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dssrq_3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2/dns/0.log" Sep 10 12:48:09.430323 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:09.430306 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dssrq_3c3ec4cf-9df3-42a2-829b-27b75b5e0ab2/kube-rbac-proxy/0.log" Sep 10 12:48:09.492968 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:09.492940 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lm7qv_f7244cbf-0a28-477e-9012-5dbe58beaa77/dns-node-resolver/0.log" Sep 10 12:48:09.912081 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:09.912046 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-cbd876cd-gdc7j_c0707deb-a9da-4cb3-b67e-38ffbee97f42/registry/0.log" Sep 10 12:48:09.977059 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:09.977027 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vqcn2_a1521572-7823-459a-a435-0248c1ae1b12/node-ca/0.log" Sep 10 12:48:10.615346 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:10.615318 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-85f68cb7f6-gsvtn_d174be6e-11f3-4679-a7d9-13ad404888f2/router/0.log" Sep 10 12:48:10.935912 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:10.935830 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kgd4g_d295dd14-e004-4178-9b90-086fe26e6778/serve-healthcheck-canary/0.log" Sep 10 12:48:11.358747 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:11.358716 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pqcdq_abfa64ea-18f1-4a06-9292-15a385850835/kube-rbac-proxy/0.log" Sep 10 12:48:11.377578 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:11.377547 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pqcdq_abfa64ea-18f1-4a06-9292-15a385850835/exporter/0.log" Sep 10 12:48:11.397251 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:11.397230 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pqcdq_abfa64ea-18f1-4a06-9292-15a385850835/extractor/0.log" Sep 10 12:48:15.385642 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:15.385604 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dxwh5/perf-node-gather-daemonset-59f96" Sep 10 12:48:16.327636 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:16.327602 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5fct5_d34fb4ae-61a3-4533-a553-9dc4e2f6daaf/kube-multus-additional-cni-plugins/0.log" Sep 10 12:48:16.349761 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:16.349735 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5fct5_d34fb4ae-61a3-4533-a553-9dc4e2f6daaf/egress-router-binary-copy/0.log" Sep 10 12:48:16.369256 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:16.369229 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5fct5_d34fb4ae-61a3-4533-a553-9dc4e2f6daaf/cni-plugins/0.log" Sep 10 12:48:16.388083 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:16.388055 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5fct5_d34fb4ae-61a3-4533-a553-9dc4e2f6daaf/bond-cni-plugin/0.log" Sep 10 12:48:16.408294 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:16.408270 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5fct5_d34fb4ae-61a3-4533-a553-9dc4e2f6daaf/routeoverride-cni/0.log" Sep 10 12:48:16.427535 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:16.427512 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5fct5_d34fb4ae-61a3-4533-a553-9dc4e2f6daaf/whereabouts-cni-bincopy/0.log" Sep 10 12:48:16.446948 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:16.446927 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5fct5_d34fb4ae-61a3-4533-a553-9dc4e2f6daaf/whereabouts-cni/0.log" Sep 10 12:48:16.819627 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:16.819601 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-smplh_3ab78f8a-127b-4d1a-9d6f-0b9e06251841/kube-multus/0.log" Sep 10 12:48:16.838214 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:16.838184 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6j4mf_c6ad9571-3788-4c07-8778-c647b1a2cdce/network-metrics-daemon/0.log" Sep 10 12:48:16.856305 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:16.856213 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6j4mf_c6ad9571-3788-4c07-8778-c647b1a2cdce/kube-rbac-proxy/0.log" Sep 10 12:48:17.771491 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:17.771466 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/ovn-controller/0.log" Sep 10 12:48:17.789410 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:17.789362 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/ovn-acl-logging/1.log" Sep 10 12:48:17.790326 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:17.790291 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/ovn-acl-logging/0.log" Sep 10 12:48:17.807807 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:17.807777 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/kube-rbac-proxy-node/0.log" Sep 10 12:48:17.830583 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:17.830486 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/kube-rbac-proxy-ovn-metrics/0.log" Sep 10 12:48:17.850588 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:17.850546 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/northd/0.log" Sep 10 12:48:17.869743 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:17.869635 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/nbdb/0.log" Sep 10 12:48:17.890074 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:17.890048 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/sbdb/0.log" Sep 10 12:48:17.939205 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:17.939169 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4qqqh_7bc3c1d2-e9d8-4fd4-bd94-878d61758b97/ovnkube-controller/0.log" Sep 10 12:48:19.321613 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:19.321576 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-k9bgv_1aa57da6-29e7-4701-8b1b-be81cf4ebed6/network-check-target-container/0.log" Sep 10 12:48:20.163176 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:20.163146 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-wqv99_adee8f79-7570-4cb5-be43-089a5a464107/iptables-alerter/0.log" Sep 10 12:48:20.783305 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:20.783279 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-vpqd2_be18f526-aa95-4956-9d00-8e31388c5c26/tuned/0.log" Sep 10 12:48:22.331319 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:22.331284 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-7cdcfcf76c-wftws_1727f13e-a084-4cc0-a35c-d7fc811a4c40/cluster-samples-operator/0.log" Sep 10 12:48:22.345653 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:22.345626 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-7cdcfcf76c-wftws_1727f13e-a084-4cc0-a35c-d7fc811a4c40/cluster-samples-operator-watch/0.log" Sep 10 12:48:23.182332 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:23.182301 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca-operator_service-ca-operator-859f94c4b-kmrbv_5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2/service-ca-operator/1.log" Sep 10 12:48:23.183518 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:23.183498 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca-operator_service-ca-operator-859f94c4b-kmrbv_5bf4bb00-f3f9-4b88-8f1d-d08391b9bde2/service-ca-operator/0.log" Sep 10 12:48:23.466948 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:23.466857 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca_service-ca-75d99c484-2vg96_ed074ff5-21b2-4590-9b9f-55cd870f7664/service-ca-controller/0.log" Sep 10 12:48:23.931345 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:23.931315 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-vwvsl_3e79b263-6658-4ca2-8d1f-df5cfc8c24da/csi-driver/0.log" Sep 10 12:48:23.952575 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:23.952548 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-vwvsl_3e79b263-6658-4ca2-8d1f-df5cfc8c24da/csi-node-driver-registrar/0.log" Sep 10 12:48:23.974454 ip-10-0-143-110 kubenswrapper[2583]: I0910 12:48:23.974431 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-vwvsl_3e79b263-6658-4ca2-8d1f-df5cfc8c24da/csi-liveness-probe/0.log"