Sep 10 12:28:12.801250 ip-10-0-135-250 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 10 12:28:12.801261 ip-10-0-135-250 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 10 12:28:12.801268 ip-10-0-135-250 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 10 12:28:12.801489 ip-10-0-135-250 systemd[1]: Failed to start Kubernetes Kubelet. Sep 10 12:28:22.108353 ip-10-0-135-250 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Sep 10 12:28:22.108367 ip-10-0-135-250 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot a690bd5309444f2a8249df14f31b778b -- Sep 10 12:29:52.787479 ip-10-0-135-250 systemd[1]: Starting Kubernetes Kubelet... Sep 10 12:29:53.239917 ip-10-0-135-250 kubenswrapper[2588]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 12:29:53.239917 ip-10-0-135-250 kubenswrapper[2588]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 10 12:29:53.239917 ip-10-0-135-250 kubenswrapper[2588]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 12:29:53.239917 ip-10-0-135-250 kubenswrapper[2588]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 10 12:29:53.239917 ip-10-0-135-250 kubenswrapper[2588]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 12:29:53.241938 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.241814 2588 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 10 12:29:53.245237 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245217 2588 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 12:29:53.245237 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245236 2588 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 12:29:53.245311 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245240 2588 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 12:29:53.245311 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245243 2588 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 12:29:53.245311 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245247 2588 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 12:29:53.245311 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245249 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 12:29:53.245311 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245259 2588 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 12:29:53.245311 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245263 2588 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 12:29:53.245311 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245265 2588 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 12:29:53.245311 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245270 2588 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 12:29:53.245311 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245275 2588 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 12:29:53.245311 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245278 2588 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 12:29:53.245311 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245281 2588 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 12:29:53.245311 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245285 2588 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 12:29:53.245311 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245288 2588 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 12:29:53.245311 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245290 2588 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 12:29:53.245311 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245293 2588 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 12:29:53.245311 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245296 2588 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 12:29:53.245311 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245298 2588 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 12:29:53.245311 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245301 2588 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 12:29:53.245311 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245304 2588 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 12:29:53.245776 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245306 2588 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 12:29:53.245776 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245309 2588 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 12:29:53.245776 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245312 2588 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 12:29:53.245776 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245314 2588 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 12:29:53.245776 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245317 2588 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 12:29:53.245776 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245320 2588 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 12:29:53.245776 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245324 2588 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 12:29:53.245776 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245327 2588 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 12:29:53.245776 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245329 2588 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 12:29:53.245776 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245332 2588 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 12:29:53.245776 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245334 2588 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 12:29:53.245776 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245337 2588 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 12:29:53.245776 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245339 2588 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 12:29:53.245776 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245342 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 12:29:53.245776 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245345 2588 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 12:29:53.245776 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245348 2588 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 12:29:53.245776 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245350 2588 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 12:29:53.245776 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245353 2588 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 12:29:53.245776 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245356 2588 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 12:29:53.245776 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245358 2588 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 12:29:53.246310 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245360 2588 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 12:29:53.246310 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245363 2588 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 12:29:53.246310 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245366 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 12:29:53.246310 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245368 2588 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 12:29:53.246310 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245370 2588 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 12:29:53.246310 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245373 2588 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 12:29:53.246310 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245375 2588 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 12:29:53.246310 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245378 2588 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 12:29:53.246310 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245380 2588 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 12:29:53.246310 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245383 2588 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 12:29:53.246310 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245385 2588 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 12:29:53.246310 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245388 2588 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 12:29:53.246310 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245390 2588 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 12:29:53.246310 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245394 2588 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 12:29:53.246310 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245397 2588 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 12:29:53.246310 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245399 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 12:29:53.246310 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245402 2588 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 12:29:53.246310 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245405 2588 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 12:29:53.246310 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245407 2588 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 12:29:53.246310 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245410 2588 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 12:29:53.246824 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245413 2588 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 12:29:53.246824 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245418 2588 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 12:29:53.246824 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245421 2588 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 12:29:53.246824 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245424 2588 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 12:29:53.246824 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245427 2588 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 12:29:53.246824 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245429 2588 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 12:29:53.246824 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245432 2588 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 12:29:53.246824 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245436 2588 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 12:29:53.246824 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245439 2588 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 12:29:53.246824 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245442 2588 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 12:29:53.246824 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245445 2588 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 12:29:53.246824 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245448 2588 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 12:29:53.246824 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245451 2588 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 12:29:53.246824 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245454 2588 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 12:29:53.246824 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245456 2588 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 12:29:53.246824 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245459 2588 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 12:29:53.246824 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245462 2588 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 12:29:53.246824 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245465 2588 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 12:29:53.246824 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245468 2588 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 12:29:53.247316 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245471 2588 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 12:29:53.247316 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245475 2588 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 12:29:53.247316 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245478 2588 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 12:29:53.247316 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245480 2588 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 12:29:53.247316 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245483 2588 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 12:29:53.247316 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245486 2588 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 12:29:53.247316 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245490 2588 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 12:29:53.247316 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245493 2588 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 12:29:53.247316 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245496 2588 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 12:29:53.247316 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245499 2588 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 12:29:53.247316 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245501 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 12:29:53.247316 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245504 2588 feature_gate.go:328] unrecognized feature gate: Example Sep 10 12:29:53.247316 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245507 2588 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 12:29:53.247316 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245509 2588 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 12:29:53.247316 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245512 2588 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 12:29:53.247316 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245515 2588 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 12:29:53.247316 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245517 2588 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 12:29:53.247316 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245520 2588 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 12:29:53.247316 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.245523 2588 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 12:29:53.247316 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246295 2588 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 12:29:53.247316 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246303 2588 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 12:29:53.247870 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246306 2588 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 12:29:53.247870 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246309 2588 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 12:29:53.247870 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246312 2588 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 12:29:53.247870 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246315 2588 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 12:29:53.247870 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246318 2588 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 12:29:53.247870 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246321 2588 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 12:29:53.247870 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246324 2588 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 12:29:53.247870 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246326 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 12:29:53.247870 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246329 2588 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 12:29:53.247870 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246332 2588 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 12:29:53.247870 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246334 2588 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 12:29:53.247870 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246337 2588 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 12:29:53.247870 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246340 2588 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 12:29:53.247870 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246343 2588 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 12:29:53.247870 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246345 2588 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 12:29:53.247870 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246348 2588 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 12:29:53.247870 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246350 2588 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 12:29:53.247870 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246355 2588 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 12:29:53.247870 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246359 2588 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 12:29:53.247870 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246363 2588 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 12:29:53.248388 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246366 2588 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 12:29:53.248388 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246369 2588 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 12:29:53.248388 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246372 2588 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 12:29:53.248388 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246374 2588 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 12:29:53.248388 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246377 2588 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 12:29:53.248388 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246379 2588 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 12:29:53.248388 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246382 2588 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 12:29:53.248388 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246385 2588 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 12:29:53.248388 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246387 2588 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 12:29:53.248388 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246390 2588 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 12:29:53.248388 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246392 2588 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 12:29:53.248388 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246395 2588 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 12:29:53.248388 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246398 2588 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 12:29:53.248388 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246401 2588 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 12:29:53.248388 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246404 2588 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 12:29:53.248388 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246407 2588 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 12:29:53.248388 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246410 2588 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 12:29:53.248388 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246413 2588 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 12:29:53.248388 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246416 2588 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 12:29:53.248388 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246419 2588 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 12:29:53.248894 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246421 2588 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 12:29:53.248894 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246424 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 12:29:53.248894 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246426 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 12:29:53.248894 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246429 2588 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 12:29:53.248894 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246432 2588 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 12:29:53.248894 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246434 2588 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 12:29:53.248894 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246437 2588 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 12:29:53.248894 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246439 2588 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 12:29:53.248894 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246442 2588 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 12:29:53.248894 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246447 2588 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 12:29:53.248894 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246451 2588 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 12:29:53.248894 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246454 2588 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 12:29:53.248894 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246457 2588 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 12:29:53.248894 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246461 2588 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 12:29:53.248894 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246464 2588 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 12:29:53.248894 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246467 2588 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 12:29:53.248894 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246469 2588 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 12:29:53.248894 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246472 2588 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 12:29:53.248894 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246475 2588 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 12:29:53.249392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246477 2588 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 12:29:53.249392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246480 2588 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 12:29:53.249392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246482 2588 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 12:29:53.249392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246485 2588 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 12:29:53.249392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246488 2588 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 12:29:53.249392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246490 2588 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 12:29:53.249392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246493 2588 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 12:29:53.249392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246495 2588 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 12:29:53.249392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246498 2588 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 12:29:53.249392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246500 2588 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 12:29:53.249392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246503 2588 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 12:29:53.249392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246506 2588 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 12:29:53.249392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246508 2588 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 12:29:53.249392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246511 2588 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 12:29:53.249392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246513 2588 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 12:29:53.249392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246516 2588 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 12:29:53.249392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246518 2588 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 12:29:53.249392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246521 2588 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 12:29:53.249392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246524 2588 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 12:29:53.249877 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246528 2588 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 12:29:53.249877 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246531 2588 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 12:29:53.249877 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246534 2588 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 12:29:53.249877 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246537 2588 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 12:29:53.249877 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246539 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 12:29:53.249877 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246542 2588 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 12:29:53.249877 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246544 2588 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 12:29:53.249877 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246546 2588 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 12:29:53.249877 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246550 2588 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 12:29:53.249877 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246552 2588 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 12:29:53.249877 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246555 2588 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 12:29:53.249877 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246557 2588 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 12:29:53.249877 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246560 2588 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 12:29:53.249877 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246564 2588 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 12:29:53.249877 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246567 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 12:29:53.249877 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246570 2588 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 12:29:53.249877 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246573 2588 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 12:29:53.249877 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246575 2588 feature_gate.go:328] unrecognized feature gate: Example Sep 10 12:29:53.249877 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246578 2588 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 12:29:53.249877 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246979 2588 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 12:29:53.250403 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246985 2588 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 12:29:53.250403 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246988 2588 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 12:29:53.250403 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246991 2588 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 12:29:53.250403 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246994 2588 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 12:29:53.250403 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246997 2588 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 12:29:53.250403 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.246999 2588 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 12:29:53.250403 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247002 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 12:29:53.250403 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247004 2588 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 12:29:53.250403 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247007 2588 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 12:29:53.250403 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247009 2588 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 12:29:53.250403 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247012 2588 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 12:29:53.250403 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247015 2588 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 12:29:53.250403 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247018 2588 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 12:29:53.250403 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247020 2588 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 12:29:53.250403 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247023 2588 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 12:29:53.250403 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247025 2588 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 12:29:53.250403 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247029 2588 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 12:29:53.250403 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247031 2588 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 12:29:53.250403 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247034 2588 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 12:29:53.250403 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247036 2588 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 12:29:53.250908 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247039 2588 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 12:29:53.250908 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247042 2588 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 12:29:53.250908 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247044 2588 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 12:29:53.250908 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247046 2588 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 12:29:53.250908 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247050 2588 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 12:29:53.250908 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247054 2588 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 12:29:53.250908 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247057 2588 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 12:29:53.250908 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247060 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 12:29:53.250908 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247063 2588 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 12:29:53.250908 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247066 2588 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 12:29:53.250908 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247069 2588 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 12:29:53.250908 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247071 2588 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 12:29:53.250908 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247074 2588 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 12:29:53.250908 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247077 2588 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 12:29:53.250908 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247080 2588 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 12:29:53.250908 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247082 2588 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 12:29:53.250908 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247086 2588 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 12:29:53.250908 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247089 2588 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 12:29:53.250908 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247091 2588 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 12:29:53.250908 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247094 2588 feature_gate.go:328] unrecognized feature gate: Example Sep 10 12:29:53.251416 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247096 2588 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 12:29:53.251416 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247099 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 12:29:53.251416 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247102 2588 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 12:29:53.251416 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247104 2588 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 12:29:53.251416 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247106 2588 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 12:29:53.251416 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247109 2588 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 12:29:53.251416 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247111 2588 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 12:29:53.251416 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247114 2588 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 12:29:53.251416 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247117 2588 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 12:29:53.251416 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247119 2588 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 12:29:53.251416 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247122 2588 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 12:29:53.251416 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247124 2588 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 12:29:53.251416 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247127 2588 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 12:29:53.251416 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247130 2588 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 12:29:53.251416 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247132 2588 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 12:29:53.251416 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247135 2588 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 12:29:53.251416 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247137 2588 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 12:29:53.251416 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247140 2588 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 12:29:53.251416 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247142 2588 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 12:29:53.251416 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247145 2588 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 12:29:53.251920 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247147 2588 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 12:29:53.251920 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247152 2588 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 12:29:53.251920 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247155 2588 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 12:29:53.251920 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247158 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 12:29:53.251920 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247174 2588 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 12:29:53.251920 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247177 2588 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 12:29:53.251920 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247180 2588 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 12:29:53.251920 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247182 2588 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 12:29:53.251920 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247185 2588 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 12:29:53.251920 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247189 2588 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 12:29:53.251920 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247191 2588 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 12:29:53.251920 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247194 2588 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 12:29:53.251920 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247196 2588 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 12:29:53.251920 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247199 2588 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 12:29:53.251920 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247204 2588 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 12:29:53.251920 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247206 2588 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 12:29:53.251920 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247209 2588 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 12:29:53.251920 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247211 2588 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 12:29:53.251920 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247214 2588 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 12:29:53.252392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247218 2588 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 12:29:53.252392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247221 2588 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 12:29:53.252392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247224 2588 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 12:29:53.252392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247227 2588 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 12:29:53.252392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247229 2588 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 12:29:53.252392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247232 2588 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 12:29:53.252392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247235 2588 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 12:29:53.252392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247238 2588 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 12:29:53.252392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247241 2588 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 12:29:53.252392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247244 2588 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 12:29:53.252392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247247 2588 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 12:29:53.252392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247250 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 12:29:53.252392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247252 2588 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 12:29:53.252392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247255 2588 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 12:29:53.252392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247258 2588 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 12:29:53.252392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247260 2588 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 12:29:53.252392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247263 2588 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 12:29:53.252392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247266 2588 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 12:29:53.252392 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.247268 2588 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 12:29:53.252392 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247348 2588 flags.go:64] FLAG: --address="0.0.0.0" Sep 10 12:29:53.252931 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247383 2588 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 10 12:29:53.252931 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247390 2588 flags.go:64] FLAG: --anonymous-auth="true" Sep 10 12:29:53.252931 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247395 2588 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 10 12:29:53.252931 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247403 2588 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 10 12:29:53.252931 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247406 2588 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 10 12:29:53.252931 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247411 2588 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 10 12:29:53.252931 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247415 2588 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 10 12:29:53.252931 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247420 2588 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 10 12:29:53.252931 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247423 2588 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 10 12:29:53.252931 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247427 2588 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 10 12:29:53.252931 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247430 2588 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 10 12:29:53.252931 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247433 2588 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 10 12:29:53.252931 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247438 2588 flags.go:64] FLAG: --cgroup-root="" Sep 10 12:29:53.252931 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247441 2588 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 10 12:29:53.252931 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247444 2588 flags.go:64] FLAG: --client-ca-file="" Sep 10 12:29:53.252931 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247447 2588 flags.go:64] FLAG: --cloud-provider="external" Sep 10 12:29:53.252931 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247450 2588 flags.go:64] FLAG: --cluster-dns="[]" Sep 10 12:29:53.252931 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247455 2588 flags.go:64] FLAG: --cluster-domain="" Sep 10 12:29:53.252931 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247458 2588 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 10 12:29:53.252931 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247461 2588 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 10 12:29:53.252931 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247464 2588 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 10 12:29:53.252931 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247467 2588 flags.go:64] FLAG: --container-log-max-files="5" Sep 10 12:29:53.252931 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247471 2588 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 10 12:29:53.252931 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247474 2588 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247477 2588 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247480 2588 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247483 2588 flags.go:64] FLAG: --contention-profiling="false" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247486 2588 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247489 2588 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247493 2588 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247496 2588 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247500 2588 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247503 2588 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247506 2588 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247508 2588 flags.go:64] FLAG: --enable-load-reader="false" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247511 2588 flags.go:64] FLAG: --enable-server="true" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247517 2588 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247521 2588 flags.go:64] FLAG: --event-burst="100" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247525 2588 flags.go:64] FLAG: --event-qps="50" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247529 2588 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247532 2588 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247535 2588 flags.go:64] FLAG: --eviction-hard="" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247539 2588 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247542 2588 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247546 2588 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247550 2588 flags.go:64] FLAG: --eviction-soft="" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247553 2588 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247555 2588 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 10 12:29:53.253529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247558 2588 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 10 12:29:53.254144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247561 2588 flags.go:64] FLAG: --experimental-mounter-path="" Sep 10 12:29:53.254144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247564 2588 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 10 12:29:53.254144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247567 2588 flags.go:64] FLAG: --fail-swap-on="true" Sep 10 12:29:53.254144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247570 2588 flags.go:64] FLAG: --feature-gates="" Sep 10 12:29:53.254144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247574 2588 flags.go:64] FLAG: --file-check-frequency="20s" Sep 10 12:29:53.254144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247577 2588 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 10 12:29:53.254144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247580 2588 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 10 12:29:53.254144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247583 2588 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 10 12:29:53.254144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247586 2588 flags.go:64] FLAG: --healthz-port="10248" Sep 10 12:29:53.254144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247589 2588 flags.go:64] FLAG: --help="false" Sep 10 12:29:53.254144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247592 2588 flags.go:64] FLAG: --hostname-override="ip-10-0-135-250.ec2.internal" Sep 10 12:29:53.254144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247595 2588 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 10 12:29:53.254144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247598 2588 flags.go:64] FLAG: --http-check-frequency="20s" Sep 10 12:29:53.254144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247601 2588 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 10 12:29:53.254144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247604 2588 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 10 12:29:53.254144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247608 2588 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 10 12:29:53.254144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247611 2588 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 10 12:29:53.254144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247613 2588 flags.go:64] FLAG: --image-service-endpoint="" Sep 10 12:29:53.254144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247616 2588 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 10 12:29:53.254144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247619 2588 flags.go:64] FLAG: --kube-api-burst="100" Sep 10 12:29:53.254144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247622 2588 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 10 12:29:53.254144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247625 2588 flags.go:64] FLAG: --kube-api-qps="50" Sep 10 12:29:53.254144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247629 2588 flags.go:64] FLAG: --kube-reserved="" Sep 10 12:29:53.254144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247632 2588 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247635 2588 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247638 2588 flags.go:64] FLAG: --kubelet-cgroups="" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247641 2588 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247645 2588 flags.go:64] FLAG: --lock-file="" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247648 2588 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247651 2588 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247654 2588 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247660 2588 flags.go:64] FLAG: --log-json-split-stream="false" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247662 2588 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247665 2588 flags.go:64] FLAG: --log-text-split-stream="false" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247668 2588 flags.go:64] FLAG: --logging-format="text" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247671 2588 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247674 2588 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247677 2588 flags.go:64] FLAG: --manifest-url="" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247680 2588 flags.go:64] FLAG: --manifest-url-header="" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247684 2588 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247687 2588 flags.go:64] FLAG: --max-open-files="1000000" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247691 2588 flags.go:64] FLAG: --max-pods="110" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247694 2588 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247697 2588 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247700 2588 flags.go:64] FLAG: --memory-manager-policy="None" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247703 2588 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247706 2588 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247709 2588 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 10 12:29:53.254762 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247712 2588 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247719 2588 flags.go:64] FLAG: --node-status-max-images="50" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247722 2588 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247726 2588 flags.go:64] FLAG: --oom-score-adj="-999" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247729 2588 flags.go:64] FLAG: --pod-cidr="" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247732 2588 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247738 2588 flags.go:64] FLAG: --pod-manifest-path="" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247742 2588 flags.go:64] FLAG: --pod-max-pids="-1" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247746 2588 flags.go:64] FLAG: --pods-per-core="0" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247748 2588 flags.go:64] FLAG: --port="10250" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247752 2588 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247756 2588 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-037a7c2c9bde614e9" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247759 2588 flags.go:64] FLAG: --qos-reserved="" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247762 2588 flags.go:64] FLAG: --read-only-port="10255" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247765 2588 flags.go:64] FLAG: --register-node="true" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247768 2588 flags.go:64] FLAG: --register-with-taints="" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247771 2588 flags.go:64] FLAG: --registry-burst="10" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247774 2588 flags.go:64] FLAG: --registry-qps="5" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247777 2588 flags.go:64] FLAG: --reserved-cpus="" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247780 2588 flags.go:64] FLAG: --reserved-memory="" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247784 2588 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247787 2588 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247790 2588 flags.go:64] FLAG: --rotate-certificates="false" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247793 2588 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247796 2588 flags.go:64] FLAG: --runonce="false" Sep 10 12:29:53.255405 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247798 2588 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247801 2588 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247804 2588 flags.go:64] FLAG: --seccomp-default="false" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247807 2588 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247810 2588 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247814 2588 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247816 2588 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247820 2588 flags.go:64] FLAG: --storage-driver-password="root" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247822 2588 flags.go:64] FLAG: --storage-driver-secure="false" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247825 2588 flags.go:64] FLAG: --storage-driver-table="stats" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247828 2588 flags.go:64] FLAG: --storage-driver-user="root" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247831 2588 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247834 2588 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247837 2588 flags.go:64] FLAG: --system-cgroups="" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247842 2588 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247847 2588 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247850 2588 flags.go:64] FLAG: --tls-cert-file="" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247853 2588 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247858 2588 flags.go:64] FLAG: --tls-min-version="" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247861 2588 flags.go:64] FLAG: --tls-private-key-file="" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247864 2588 flags.go:64] FLAG: --topology-manager-policy="none" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247867 2588 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247870 2588 flags.go:64] FLAG: --topology-manager-scope="container" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247873 2588 flags.go:64] FLAG: --v="2" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247877 2588 flags.go:64] FLAG: --version="false" Sep 10 12:29:53.256062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247882 2588 flags.go:64] FLAG: --vmodule="" Sep 10 12:29:53.257088 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247885 2588 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 10 12:29:53.257088 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.247889 2588 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 10 12:29:53.257088 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248001 2588 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 12:29:53.257088 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248004 2588 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 12:29:53.257088 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248008 2588 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 12:29:53.257088 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248011 2588 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 12:29:53.257088 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248014 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 12:29:53.257088 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248017 2588 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 12:29:53.257088 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248019 2588 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 12:29:53.257088 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248022 2588 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 12:29:53.257088 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248024 2588 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 12:29:53.257088 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248027 2588 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 12:29:53.257088 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248030 2588 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 12:29:53.257088 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248033 2588 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 12:29:53.257088 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248036 2588 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 12:29:53.257088 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248040 2588 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 12:29:53.257088 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248043 2588 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 12:29:53.257088 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248046 2588 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 12:29:53.257088 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248049 2588 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 12:29:53.257633 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248052 2588 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 12:29:53.257633 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248058 2588 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 12:29:53.257633 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248061 2588 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 12:29:53.257633 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248064 2588 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 12:29:53.257633 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248066 2588 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 12:29:53.257633 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248069 2588 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 12:29:53.257633 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248072 2588 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 12:29:53.257633 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248075 2588 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 12:29:53.257633 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248078 2588 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 12:29:53.257633 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248081 2588 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 12:29:53.257633 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248083 2588 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 12:29:53.257633 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248086 2588 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 12:29:53.257633 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248089 2588 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 12:29:53.257633 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248091 2588 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 12:29:53.257633 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248094 2588 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 12:29:53.257633 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248096 2588 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 12:29:53.257633 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248099 2588 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 12:29:53.257633 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248102 2588 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 12:29:53.257633 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248104 2588 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 12:29:53.258112 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248107 2588 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 12:29:53.258112 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248109 2588 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 12:29:53.258112 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248113 2588 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 12:29:53.258112 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248115 2588 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 12:29:53.258112 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248118 2588 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 12:29:53.258112 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248120 2588 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 12:29:53.258112 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248123 2588 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 12:29:53.258112 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248126 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 12:29:53.258112 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248129 2588 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 12:29:53.258112 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248133 2588 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 12:29:53.258112 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248136 2588 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 12:29:53.258112 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248138 2588 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 12:29:53.258112 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248141 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 12:29:53.258112 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248143 2588 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 12:29:53.258112 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248147 2588 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 12:29:53.258112 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248150 2588 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 12:29:53.258112 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248153 2588 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 12:29:53.258112 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248156 2588 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 12:29:53.258112 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248172 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 12:29:53.258112 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248175 2588 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 12:29:53.258635 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248178 2588 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 12:29:53.258635 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248181 2588 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 12:29:53.258635 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248184 2588 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 12:29:53.258635 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248186 2588 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 12:29:53.258635 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248189 2588 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 12:29:53.258635 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248192 2588 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 12:29:53.258635 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248194 2588 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 12:29:53.258635 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248197 2588 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 12:29:53.258635 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248200 2588 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 12:29:53.258635 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248202 2588 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 12:29:53.258635 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248205 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 12:29:53.258635 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248208 2588 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 12:29:53.258635 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248210 2588 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 12:29:53.258635 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248213 2588 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 12:29:53.258635 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248216 2588 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 12:29:53.258635 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248218 2588 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 12:29:53.258635 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248221 2588 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 12:29:53.258635 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248224 2588 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 12:29:53.258635 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248226 2588 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 12:29:53.258635 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248229 2588 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 12:29:53.259220 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248231 2588 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 12:29:53.259220 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248234 2588 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 12:29:53.259220 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248236 2588 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 12:29:53.259220 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248239 2588 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 12:29:53.259220 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248242 2588 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 12:29:53.259220 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248245 2588 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 12:29:53.259220 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248249 2588 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 12:29:53.259220 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248251 2588 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 12:29:53.259220 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248255 2588 feature_gate.go:328] unrecognized feature gate: Example Sep 10 12:29:53.259220 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248258 2588 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 12:29:53.259220 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248261 2588 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 12:29:53.259220 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248263 2588 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 12:29:53.259220 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248266 2588 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 12:29:53.259220 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248270 2588 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 12:29:53.259220 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248274 2588 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 12:29:53.259220 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248277 2588 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 12:29:53.259220 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248280 2588 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 12:29:53.259220 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248282 2588 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 12:29:53.259220 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248285 2588 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 12:29:53.259220 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248287 2588 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 12:29:53.259715 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248290 2588 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 12:29:53.259715 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248293 2588 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 12:29:53.259715 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.248296 2588 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 12:29:53.259715 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.249102 2588 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 12:29:53.259715 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.259644 2588 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 10 12:29:53.259715 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.259668 2588 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 10 12:29:53.259715 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.259694 2588 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 10 12:29:53.259715 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.259704 2588 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 10 12:29:53.259931 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259816 2588 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 12:29:53.259931 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259821 2588 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 12:29:53.259931 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259825 2588 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 12:29:53.259931 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259829 2588 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 12:29:53.259931 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259832 2588 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 12:29:53.259931 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259837 2588 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 12:29:53.259931 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259840 2588 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 12:29:53.259931 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259843 2588 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 12:29:53.259931 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259846 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 12:29:53.259931 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259849 2588 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 12:29:53.259931 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259852 2588 feature_gate.go:328] unrecognized feature gate: Example Sep 10 12:29:53.259931 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259855 2588 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 12:29:53.259931 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259858 2588 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 12:29:53.259931 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259861 2588 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 12:29:53.259931 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259863 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 12:29:53.259931 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259869 2588 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 12:29:53.259931 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259872 2588 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 12:29:53.259931 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259879 2588 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 12:29:53.259931 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259884 2588 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 12:29:53.259931 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259887 2588 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 12:29:53.260471 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259895 2588 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 12:29:53.260471 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259898 2588 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 12:29:53.260471 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259901 2588 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 12:29:53.260471 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259904 2588 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 12:29:53.260471 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259906 2588 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 12:29:53.260471 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259909 2588 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 12:29:53.260471 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259912 2588 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 12:29:53.260471 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259914 2588 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 12:29:53.260471 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259917 2588 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 12:29:53.260471 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259919 2588 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 12:29:53.260471 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259925 2588 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 12:29:53.260471 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259927 2588 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 12:29:53.260471 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259930 2588 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 12:29:53.260471 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259934 2588 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 12:29:53.260471 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259938 2588 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 12:29:53.260471 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259941 2588 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 12:29:53.260471 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259943 2588 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 12:29:53.260471 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259946 2588 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 12:29:53.260471 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259949 2588 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 12:29:53.260471 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259952 2588 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 12:29:53.260960 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259955 2588 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 12:29:53.260960 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259958 2588 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 12:29:53.260960 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259963 2588 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 12:29:53.260960 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259966 2588 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 12:29:53.260960 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259969 2588 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 12:29:53.260960 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259972 2588 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 12:29:53.260960 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259974 2588 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 12:29:53.260960 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259977 2588 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 12:29:53.260960 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259980 2588 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 12:29:53.260960 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259982 2588 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 12:29:53.260960 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259985 2588 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 12:29:53.260960 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259988 2588 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 12:29:53.260960 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259990 2588 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 12:29:53.260960 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.259999 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 12:29:53.260960 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260001 2588 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 12:29:53.260960 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260006 2588 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 12:29:53.260960 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260010 2588 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 12:29:53.260960 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260015 2588 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 12:29:53.260960 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260018 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 12:29:53.261443 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260021 2588 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 12:29:53.261443 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260024 2588 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 12:29:53.261443 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260027 2588 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 12:29:53.261443 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260030 2588 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 12:29:53.261443 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260033 2588 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 12:29:53.261443 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260036 2588 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 12:29:53.261443 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260040 2588 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 12:29:53.261443 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260042 2588 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 12:29:53.261443 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260048 2588 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 12:29:53.261443 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260052 2588 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 12:29:53.261443 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260056 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 12:29:53.261443 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260059 2588 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 12:29:53.261443 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260061 2588 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 12:29:53.261443 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260064 2588 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 12:29:53.261443 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260067 2588 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 12:29:53.261443 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260069 2588 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 12:29:53.261443 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260072 2588 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 12:29:53.261443 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260075 2588 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 12:29:53.261443 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260078 2588 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 12:29:53.261917 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260081 2588 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 12:29:53.261917 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260086 2588 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 12:29:53.261917 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260089 2588 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 12:29:53.261917 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260092 2588 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 12:29:53.261917 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260094 2588 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 12:29:53.261917 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260097 2588 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 12:29:53.261917 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260100 2588 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 12:29:53.261917 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260103 2588 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 12:29:53.261917 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260113 2588 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 12:29:53.261917 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260116 2588 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 12:29:53.261917 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260118 2588 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 12:29:53.261917 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260121 2588 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 12:29:53.261917 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260124 2588 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 12:29:53.261917 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260127 2588 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 12:29:53.261917 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260132 2588 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 12:29:53.261917 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260135 2588 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 12:29:53.261917 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260138 2588 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 12:29:53.261917 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260141 2588 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 12:29:53.261917 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260144 2588 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 12:29:53.262409 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260147 2588 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 12:29:53.262409 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260150 2588 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 12:29:53.262409 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.260155 2588 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 12:29:53.262409 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260422 2588 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 12:29:53.262409 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260430 2588 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 12:29:53.262409 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260434 2588 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 12:29:53.262409 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260437 2588 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 12:29:53.262409 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260440 2588 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 12:29:53.262409 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260443 2588 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 12:29:53.262409 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260446 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 12:29:53.262409 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260450 2588 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 12:29:53.262409 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260454 2588 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 12:29:53.262409 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260457 2588 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 12:29:53.262409 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260460 2588 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 12:29:53.262409 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260464 2588 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 12:29:53.262792 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260466 2588 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 12:29:53.262792 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260472 2588 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 12:29:53.262792 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260475 2588 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 12:29:53.262792 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260478 2588 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 12:29:53.262792 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260481 2588 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 12:29:53.262792 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260484 2588 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 12:29:53.262792 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260487 2588 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 12:29:53.262792 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260500 2588 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 12:29:53.262792 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260503 2588 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 12:29:53.262792 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260506 2588 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 12:29:53.262792 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260509 2588 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 12:29:53.262792 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260512 2588 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 12:29:53.262792 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260514 2588 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 12:29:53.262792 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260517 2588 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 12:29:53.262792 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260523 2588 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 12:29:53.262792 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260526 2588 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 12:29:53.262792 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260529 2588 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 12:29:53.262792 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260532 2588 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 12:29:53.262792 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260535 2588 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 12:29:53.262792 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260537 2588 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 12:29:53.263299 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260540 2588 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 12:29:53.263299 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260543 2588 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 12:29:53.263299 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260546 2588 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 12:29:53.263299 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260549 2588 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 12:29:53.263299 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260561 2588 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 12:29:53.263299 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260564 2588 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 12:29:53.263299 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260570 2588 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 12:29:53.263299 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260573 2588 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 12:29:53.263299 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260576 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 12:29:53.263299 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260579 2588 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 12:29:53.263299 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260581 2588 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 12:29:53.263299 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260584 2588 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 12:29:53.263299 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260587 2588 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 12:29:53.263299 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260590 2588 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 12:29:53.263299 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260592 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 12:29:53.263299 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260595 2588 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 12:29:53.263299 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260598 2588 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 12:29:53.263299 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260600 2588 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 12:29:53.263299 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260603 2588 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 12:29:53.263299 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260608 2588 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 12:29:53.263794 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260617 2588 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 12:29:53.263794 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260620 2588 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 12:29:53.263794 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260623 2588 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 12:29:53.263794 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260625 2588 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 12:29:53.263794 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260628 2588 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 12:29:53.263794 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260632 2588 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 12:29:53.263794 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260636 2588 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 12:29:53.263794 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260640 2588 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 12:29:53.263794 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260644 2588 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 12:29:53.263794 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260649 2588 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 12:29:53.263794 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260652 2588 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 12:29:53.263794 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260658 2588 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 12:29:53.263794 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260661 2588 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 12:29:53.263794 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260663 2588 feature_gate.go:328] unrecognized feature gate: Example Sep 10 12:29:53.263794 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260666 2588 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 12:29:53.263794 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260669 2588 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 12:29:53.263794 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260672 2588 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 12:29:53.263794 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260675 2588 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 12:29:53.263794 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260678 2588 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 12:29:53.264289 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260680 2588 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 12:29:53.264289 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260683 2588 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 12:29:53.264289 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260686 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 12:29:53.264289 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260689 2588 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 12:29:53.264289 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260694 2588 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 12:29:53.264289 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260696 2588 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 12:29:53.264289 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260699 2588 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 12:29:53.264289 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260702 2588 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 12:29:53.264289 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260705 2588 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 12:29:53.264289 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260707 2588 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 12:29:53.264289 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260710 2588 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 12:29:53.264289 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260712 2588 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 12:29:53.264289 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260715 2588 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 12:29:53.264289 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260718 2588 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 12:29:53.264289 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260727 2588 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 12:29:53.264289 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260730 2588 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 12:29:53.264289 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260732 2588 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 12:29:53.264289 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260738 2588 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 12:29:53.264289 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260741 2588 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 12:29:53.264289 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260743 2588 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 12:29:53.264780 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260746 2588 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 12:29:53.264780 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260749 2588 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 12:29:53.264780 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260752 2588 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 12:29:53.264780 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260755 2588 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 12:29:53.264780 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260758 2588 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 12:29:53.264780 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260760 2588 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 12:29:53.264780 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260763 2588 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 12:29:53.264780 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:53.260766 2588 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 12:29:53.264780 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.260771 2588 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 12:29:53.264780 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.260951 2588 server.go:962] "Client rotation is on, will bootstrap in background" Sep 10 12:29:53.265082 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.264886 2588 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 10 12:29:53.266392 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.266374 2588 server.go:1019] "Starting client certificate rotation" Sep 10 12:29:53.266501 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.266481 2588 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 12:29:53.266551 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.266522 2588 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 12:29:53.269413 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.269396 2588 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 12:29:53.271478 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.271459 2588 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 12:29:53.290750 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.290720 2588 log.go:25] "Validated CRI v1 runtime API" Sep 10 12:29:53.294404 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.294385 2588 log.go:25] "Validated CRI v1 image API" Sep 10 12:29:53.295282 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.295262 2588 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 10 12:29:53.298705 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.298667 2588 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 aa4eb012-8a0f-4820-a63a-48c2660cd03c:/dev/nvme0n1p3 d172351c-1e31-4203-b657-62b5a4a91378:/dev/nvme0n1p4] Sep 10 12:29:53.298705 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.298701 2588 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 10 12:29:53.301508 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.301322 2588 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 12:29:53.304981 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.304865 2588 manager.go:216] Machine: {Timestamp:2026-09-10 12:29:53.30437024 +0000 UTC m=+0.386538734 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101073 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2aaf76fc55e30b3af38493866ad308 SystemUUID:ec2aaf76-fc55-e30b-3af3-8493866ad308 BootID:a690bd53-0944-4f2a-8249-df14f31b778b Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1d:dd:61:66:cd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1d:dd:61:66:cd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e2:e0:9d:64:ec:50 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 10 12:29:53.304981 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.304972 2588 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 10 12:29:53.305096 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.305068 2588 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 10 12:29:53.309053 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.309017 2588 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 10 12:29:53.309228 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.309063 2588 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-250.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 10 12:29:53.310619 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.310607 2588 topology_manager.go:138] "Creating topology manager with none policy" Sep 10 12:29:53.310658 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.310622 2588 container_manager_linux.go:309] "Creating device plugin manager" Sep 10 12:29:53.310658 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.310631 2588 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 12:29:53.311622 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.311610 2588 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 12:29:53.313528 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.313515 2588 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 10 12:29:53.318911 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.318894 2588 state_mem.go:36] "Initialized new in-memory state store" Sep 10 12:29:53.319054 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.319044 2588 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 10 12:29:53.319093 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.319085 2588 kubelet.go:486] "Attempting to sync node with API server" Sep 10 12:29:53.319124 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.319098 2588 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 10 12:29:53.319124 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.319109 2588 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 10 12:29:53.319124 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.319120 2588 kubelet.go:398] "Adding apiserver pod source" Sep 10 12:29:53.319247 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.319129 2588 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 10 12:29:53.320086 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.320072 2588 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 12:29:53.320181 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.320092 2588 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 12:29:53.323435 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.323402 2588 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 10 12:29:53.326549 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.326527 2588 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 10 12:29:53.326662 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.326567 2588 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 10 12:29:53.326805 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.326790 2588 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 10 12:29:53.326869 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.326811 2588 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 10 12:29:53.326869 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.326821 2588 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 10 12:29:53.326869 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.326830 2588 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 10 12:29:53.326869 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.326842 2588 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 10 12:29:53.326869 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.326851 2588 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 10 12:29:53.327386 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.327375 2588 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 10 12:29:53.327447 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.327388 2588 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 10 12:29:53.327447 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.327400 2588 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 10 12:29:53.327447 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.327408 2588 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 10 12:29:53.327447 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.327422 2588 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 10 12:29:53.327447 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.327435 2588 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 10 12:29:53.328278 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.328265 2588 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 10 12:29:53.328278 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.328279 2588 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 10 12:29:53.330748 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:53.330716 2588 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 12:29:53.330748 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:53.330717 2588 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-250.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 12:29:53.332507 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.332489 2588 server.go:1268] "Started kubelet" Sep 10 12:29:53.332626 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.332558 2588 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 10 12:29:53.332697 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.332584 2588 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 10 12:29:53.332737 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.332726 2588 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 10 12:29:53.333501 ip-10-0-135-250 systemd[1]: Started Kubernetes Kubelet. Sep 10 12:29:53.334284 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.334264 2588 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 10 12:29:53.334549 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.334413 2588 server.go:310] "Adding debug handlers to kubelet server" Sep 10 12:29:53.338901 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:53.338871 2588 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 10 12:29:53.340546 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.340515 2588 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 10 12:29:53.341225 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.341206 2588 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-250.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 12:29:53.341302 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.341273 2588 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 10 12:29:53.341446 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:53.341378 2588 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-250.ec2.internal\" not found" Sep 10 12:29:53.342069 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.342050 2588 factory.go:55] Registering systemd factory Sep 10 12:29:53.342069 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.342072 2588 factory.go:223] Registration of the systemd container factory successfully Sep 10 12:29:53.342223 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.342073 2588 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 10 12:29:53.342223 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.342075 2588 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 10 12:29:53.342223 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.342085 2588 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 10 12:29:53.342223 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.342219 2588 reconstruct.go:97] "Volume reconstruction finished" Sep 10 12:29:53.342223 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.342228 2588 reconciler.go:29] "Reconciler: start to sync state" Sep 10 12:29:53.342380 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.342302 2588 factory.go:153] Registering CRI-O factory Sep 10 12:29:53.342380 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.342315 2588 factory.go:223] Registration of the crio container factory successfully Sep 10 12:29:53.342380 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.342361 2588 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 10 12:29:53.342380 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.342379 2588 factory.go:103] Registering Raw factory Sep 10 12:29:53.342501 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.342391 2588 manager.go:1196] Started watching for new ooms in manager Sep 10 12:29:53.342501 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:53.341182 2588 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-250.ec2.internal.18d3f57ccb572f8c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-250.ec2.internal,UID:ip-10-0-135-250.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-250.ec2.internal,},FirstTimestamp:2026-09-10 12:29:53.332457356 +0000 UTC m=+0.414625850,LastTimestamp:2026-09-10 12:29:53.332457356 +0000 UTC m=+0.414625850,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-250.ec2.internal,}" Sep 10 12:29:53.342857 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.342844 2588 manager.go:349] Starting recovery of all containers Sep 10 12:29:53.346556 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:53.346371 2588 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-250.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 10 12:29:53.346556 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:53.346425 2588 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 10 12:29:53.354096 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.354051 2588 manager.go:354] Recovery completed Sep 10 12:29:53.359118 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.359087 2588 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4b6tm" Sep 10 12:29:53.359815 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.359802 2588 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:29:53.361676 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.361653 2588 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-250.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:29:53.361779 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.361684 2588 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-250.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:29:53.361779 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.361695 2588 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-250.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:29:53.362292 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.362279 2588 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 10 12:29:53.362349 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.362292 2588 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 10 12:29:53.362349 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.362312 2588 state_mem.go:36] "Initialized new in-memory state store" Sep 10 12:29:53.362349 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.362325 2588 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:29:53.363009 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.362994 2588 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-250.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:29:53.363067 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.363024 2588 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-250.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:29:53.363067 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.363039 2588 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-250.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:29:53.364431 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.364418 2588 policy_none.go:49] "None policy: Start" Sep 10 12:29:53.364495 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.364434 2588 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 10 12:29:53.364495 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.364444 2588 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 10 12:29:53.365540 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:53.365472 2588 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-250.ec2.internal.18d3f57ccd14fd7b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-250.ec2.internal,UID:ip-10-0-135-250.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-250.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-250.ec2.internal,},FirstTimestamp:2026-09-10 12:29:53.361673595 +0000 UTC m=+0.443842089,LastTimestamp:2026-09-10 12:29:53.361673595 +0000 UTC m=+0.443842089,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-250.ec2.internal,}" Sep 10 12:29:53.366365 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.366352 2588 policy_none.go:47] "Start" Sep 10 12:29:53.366768 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.366753 2588 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4b6tm" Sep 10 12:29:53.402659 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.402644 2588 manager.go:337] "Starting Device Plugin manager" Sep 10 12:29:53.402743 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:53.402683 2588 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 10 12:29:53.402743 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.402693 2588 server.go:87] "Starting device plugin registration server" Sep 10 12:29:53.403064 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.403053 2588 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 10 12:29:53.403111 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.403066 2588 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 10 12:29:53.403217 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.403197 2588 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 10 12:29:53.403285 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.403272 2588 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 10 12:29:53.403336 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.403286 2588 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 10 12:29:53.403896 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:53.403872 2588 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 10 12:29:53.403996 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:53.403930 2588 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-250.ec2.internal\" not found" Sep 10 12:29:53.420221 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.420182 2588 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 10 12:29:53.431910 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.421440 2588 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 10 12:29:53.431910 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.421483 2588 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 10 12:29:53.431910 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.421513 2588 kubelet.go:2448] "Starting kubelet main sync loop" Sep 10 12:29:53.431910 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:53.421559 2588 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 10 12:29:53.431910 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.423581 2588 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 12:29:53.504105 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.503995 2588 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:29:53.505064 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.505044 2588 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-250.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:29:53.505199 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.505075 2588 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-250.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:29:53.505199 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.505088 2588 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-250.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:29:53.505199 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.505112 2588 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-250.ec2.internal" Sep 10 12:29:53.518973 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.518945 2588 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-250.ec2.internal" Sep 10 12:29:53.518973 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:53.518977 2588 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-250.ec2.internal\": node \"ip-10-0-135-250.ec2.internal\" not found" Sep 10 12:29:53.522264 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.522241 2588 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-250.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-250.ec2.internal"] Sep 10 12:29:53.522329 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.522315 2588 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:29:53.523340 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.523323 2588 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-250.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:29:53.523415 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.523352 2588 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-250.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:29:53.523415 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.523362 2588 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-250.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:29:53.525639 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.525621 2588 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:29:53.525755 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.525740 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-250.ec2.internal" Sep 10 12:29:53.525807 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.525771 2588 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:29:53.526358 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.526342 2588 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-250.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:29:53.526438 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.526362 2588 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-250.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:29:53.526438 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.526373 2588 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-250.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:29:53.526438 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.526384 2588 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-250.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:29:53.526438 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.526395 2588 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-250.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:29:53.526438 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.526385 2588 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-250.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:29:53.528656 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.528639 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-250.ec2.internal" Sep 10 12:29:53.528725 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.528673 2588 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 12:29:53.529390 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.529375 2588 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-250.ec2.internal" event="NodeHasSufficientMemory" Sep 10 12:29:53.529456 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.529401 2588 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-250.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 12:29:53.529456 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.529416 2588 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-250.ec2.internal" event="NodeHasSufficientPID" Sep 10 12:29:53.546869 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:53.546843 2588 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-250.ec2.internal\" not found" node="ip-10-0-135-250.ec2.internal" Sep 10 12:29:53.550652 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:53.550631 2588 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-250.ec2.internal\" not found" node="ip-10-0-135-250.ec2.internal" Sep 10 12:29:53.569723 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:53.569689 2588 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-250.ec2.internal\" not found" Sep 10 12:29:53.643179 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.643139 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b683db139541b51f843e32d0a18fc2d6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-250.ec2.internal\" (UID: \"b683db139541b51f843e32d0a18fc2d6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-250.ec2.internal" Sep 10 12:29:53.643351 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.643202 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b683db139541b51f843e32d0a18fc2d6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-250.ec2.internal\" (UID: \"b683db139541b51f843e32d0a18fc2d6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-250.ec2.internal" Sep 10 12:29:53.643351 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.643254 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/928ccfc5fa7f66e685a010324e61a3c7-config\") pod \"kube-apiserver-proxy-ip-10-0-135-250.ec2.internal\" (UID: \"928ccfc5fa7f66e685a010324e61a3c7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-250.ec2.internal" Sep 10 12:29:53.669846 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:53.669809 2588 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-250.ec2.internal\" not found" Sep 10 12:29:53.743707 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.743669 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b683db139541b51f843e32d0a18fc2d6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-250.ec2.internal\" (UID: \"b683db139541b51f843e32d0a18fc2d6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-250.ec2.internal" Sep 10 12:29:53.743707 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.743706 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b683db139541b51f843e32d0a18fc2d6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-250.ec2.internal\" (UID: \"b683db139541b51f843e32d0a18fc2d6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-250.ec2.internal" Sep 10 12:29:53.743892 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.743725 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/928ccfc5fa7f66e685a010324e61a3c7-config\") pod \"kube-apiserver-proxy-ip-10-0-135-250.ec2.internal\" (UID: \"928ccfc5fa7f66e685a010324e61a3c7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-250.ec2.internal" Sep 10 12:29:53.743892 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.743765 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b683db139541b51f843e32d0a18fc2d6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-250.ec2.internal\" (UID: \"b683db139541b51f843e32d0a18fc2d6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-250.ec2.internal" Sep 10 12:29:53.743892 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.743781 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b683db139541b51f843e32d0a18fc2d6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-250.ec2.internal\" (UID: \"b683db139541b51f843e32d0a18fc2d6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-250.ec2.internal" Sep 10 12:29:53.743892 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:53.743807 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/928ccfc5fa7f66e685a010324e61a3c7-config\") pod \"kube-apiserver-proxy-ip-10-0-135-250.ec2.internal\" (UID: \"928ccfc5fa7f66e685a010324e61a3c7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-250.ec2.internal" Sep 10 12:29:55.119567 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:55.119495 2588 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-250.ec2.internal\" not found" Sep 10 12:29:55.199133 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:55.199103 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-250.ec2.internal" Sep 10 12:29:55.202939 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:55.202904 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-250.ec2.internal" Sep 10 12:29:55.220284 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:55.220253 2588 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-250.ec2.internal\" not found" Sep 10 12:29:55.320867 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:55.320812 2588 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-250.ec2.internal\" not found" Sep 10 12:29:55.421409 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:55.421327 2588 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-250.ec2.internal\" not found" Sep 10 12:29:55.521951 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:55.521915 2588 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-250.ec2.internal\" not found" Sep 10 12:29:55.614477 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:55.614438 2588 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 10 12:29:55.615014 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:55.614636 2588 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 12:29:55.622782 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:55.622754 2588 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-250.ec2.internal\" not found" Sep 10 12:29:55.639895 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:55.639863 2588 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 12:29:55.682471 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:55.682401 2588 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 12:29:55.689362 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:55.689340 2588 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 10 12:29:55.690442 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:55.690424 2588 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-250.ec2.internal" Sep 10 12:29:55.701031 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:55.701005 2588 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 12:29:55.708976 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:55.708956 2588 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 12:29:55.710995 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:55.710977 2588 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-250.ec2.internal" Sep 10 12:29:55.718561 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:55.718524 2588 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 12:24:54 +0000 UTC" deadline="2028-06-07 05:05:22.605229554 +0000 UTC" Sep 10 12:29:55.718561 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:55.718556 2588 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15256h35m26.886677651s" Sep 10 12:29:55.725325 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:55.725306 2588 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 12:29:55.728817 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:55.728793 2588 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-htmzr" Sep 10 12:29:55.739333 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:55.739302 2588 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-htmzr" Sep 10 12:29:55.838831 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:55.838637 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod928ccfc5fa7f66e685a010324e61a3c7.slice/crio-e62edc79f1fb58871ef19ab8c8d1a8d31d8ddf26db0b41b77cd90e10c9f0f5e6 WatchSource:0}: Error finding container e62edc79f1fb58871ef19ab8c8d1a8d31d8ddf26db0b41b77cd90e10c9f0f5e6: Status 404 returned error can't find the container with id e62edc79f1fb58871ef19ab8c8d1a8d31d8ddf26db0b41b77cd90e10c9f0f5e6 Sep 10 12:29:55.839050 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:55.839030 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb683db139541b51f843e32d0a18fc2d6.slice/crio-727eaaa5fe2370bfb1869e173a819e00faa885330705187522b4fb2246d5e68b WatchSource:0}: Error finding container 727eaaa5fe2370bfb1869e173a819e00faa885330705187522b4fb2246d5e68b: Status 404 returned error can't find the container with id 727eaaa5fe2370bfb1869e173a819e00faa885330705187522b4fb2246d5e68b Sep 10 12:29:55.843485 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:55.843463 2588 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 12:29:55.912943 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:55.912902 2588 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 12:29:56.424883 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.424851 2588 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 12:29:56.668644 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.668581 2588 apiserver.go:52] "Watching apiserver" Sep 10 12:29:56.676333 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.676058 2588 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 10 12:29:56.676505 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.676480 2588 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-km6jk","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp","openshift-cluster-node-tuning-operator/tuned-xhmjp","openshift-dns/node-resolver-rwdsg","openshift-multus/multus-tc789","openshift-network-diagnostics/network-check-target-p5wcp","openshift-network-operator/iptables-alerter-bjnv5","openshift-ovn-kubernetes/ovnkube-node-b47ts","kube-system/global-pull-secret-syncer-wclkd","kube-system/kube-apiserver-proxy-ip-10-0-135-250.ec2.internal","openshift-image-registry/node-ca-kmxmc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-250.ec2.internal","openshift-multus/multus-additional-cni-plugins-478c6","openshift-multus/network-metrics-daemon-cmbbq"] Sep 10 12:29:56.679599 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.679137 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bjnv5" Sep 10 12:29:56.681521 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.681493 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 10 12:29:56.681521 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.681518 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fhkcw\"" Sep 10 12:29:56.681712 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.681497 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 10 12:29:56.681712 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.681495 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 10 12:29:56.681712 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.681670 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-478c6" Sep 10 12:29:56.684565 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.683977 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 10 12:29:56.684565 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.684214 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.684565 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.684225 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-n2s8w\"" Sep 10 12:29:56.684565 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.684258 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 10 12:29:56.684565 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.684317 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 10 12:29:56.684565 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.684453 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 10 12:29:56.684893 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.684588 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 10 12:29:56.686307 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.686281 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kmxmc" Sep 10 12:29:56.687269 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.686282 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 10 12:29:56.687269 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.686883 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pg7jb\"" Sep 10 12:29:56.687269 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.687246 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 10 12:29:56.689171 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.689152 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rwdsg" Sep 10 12:29:56.689583 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.689370 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 10 12:29:56.689583 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.689506 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-f2rxn\"" Sep 10 12:29:56.689791 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.689776 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 10 12:29:56.690047 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.689871 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 10 12:29:56.691080 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.691061 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:29:56.691203 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:56.691167 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5wcp" podUID="bdfce9e5-4daa-4b23-8f66-62549603ef7f" Sep 10 12:29:56.691486 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.691467 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 10 12:29:56.691570 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.691511 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lgtl6\"" Sep 10 12:29:56.692048 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.692029 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 10 12:29:56.693564 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.693544 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tc789" Sep 10 12:29:56.695616 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.695593 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 10 12:29:56.695875 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.695857 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-hgct5\"" Sep 10 12:29:56.696206 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.696189 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.699597 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.699111 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:29:56.699597 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:56.699219 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wclkd" podUID="cb45e2aa-d5a7-47e9-828e-186e39ffd6b8" Sep 10 12:29:56.699597 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.699328 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 10 12:29:56.699795 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.699621 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 10 12:29:56.699795 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.699633 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 10 12:29:56.701064 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.700406 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ztcjf\"" Sep 10 12:29:56.701064 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.700490 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 10 12:29:56.701064 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.700515 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 10 12:29:56.701064 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.700646 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 10 12:29:56.701759 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.701655 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1019e2de-09cf-419b-a4ce-7e09921a4e04-host\") pod \"node-ca-kmxmc\" (UID: \"1019e2de-09cf-419b-a4ce-7e09921a4e04\") " pod="openshift-image-registry/node-ca-kmxmc" Sep 10 12:29:56.701759 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.701706 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d9a758b-9a83-405b-9b01-7143273959ff-tmp-dir\") pod \"node-resolver-rwdsg\" (UID: \"2d9a758b-9a83-405b-9b01-7143273959ff\") " pod="openshift-dns/node-resolver-rwdsg" Sep 10 12:29:56.701904 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.701794 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-km6jk" Sep 10 12:29:56.702125 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.701979 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g9ft\" (UniqueName: \"kubernetes.io/projected/2d9a758b-9a83-405b-9b01-7143273959ff-kube-api-access-8g9ft\") pod \"node-resolver-rwdsg\" (UID: \"2d9a758b-9a83-405b-9b01-7143273959ff\") " pod="openshift-dns/node-resolver-rwdsg" Sep 10 12:29:56.702125 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.702028 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/334e2bed-0266-4d0f-964f-96dcb9843a25-system-cni-dir\") pod \"multus-additional-cni-plugins-478c6\" (UID: \"334e2bed-0266-4d0f-964f-96dcb9843a25\") " pod="openshift-multus/multus-additional-cni-plugins-478c6" Sep 10 12:29:56.702125 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.702060 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/334e2bed-0266-4d0f-964f-96dcb9843a25-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-478c6\" (UID: \"334e2bed-0266-4d0f-964f-96dcb9843a25\") " pod="openshift-multus/multus-additional-cni-plugins-478c6" Sep 10 12:29:56.702125 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.702084 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-tmp\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.702378 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.702131 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-host-run-netns\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.702378 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.702199 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/334e2bed-0266-4d0f-964f-96dcb9843a25-tuning-conf-dir\") pod \"multus-additional-cni-plugins-478c6\" (UID: \"334e2bed-0266-4d0f-964f-96dcb9843a25\") " pod="openshift-multus/multus-additional-cni-plugins-478c6" Sep 10 12:29:56.702378 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.702245 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-etc-kubernetes\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.702378 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.702296 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-multus-cni-dir\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.702378 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.702328 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-host-var-lib-cni-bin\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.702610 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.702421 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1019e2de-09cf-419b-a4ce-7e09921a4e04-serviceca\") pod \"node-ca-kmxmc\" (UID: \"1019e2de-09cf-419b-a4ce-7e09921a4e04\") " pod="openshift-image-registry/node-ca-kmxmc" Sep 10 12:29:56.702610 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.702472 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqgph\" (UniqueName: \"kubernetes.io/projected/bdfce9e5-4daa-4b23-8f66-62549603ef7f-kube-api-access-jqgph\") pod \"network-check-target-p5wcp\" (UID: \"bdfce9e5-4daa-4b23-8f66-62549603ef7f\") " pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:29:56.702610 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.702506 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/334e2bed-0266-4d0f-964f-96dcb9843a25-cnibin\") pod \"multus-additional-cni-plugins-478c6\" (UID: \"334e2bed-0266-4d0f-964f-96dcb9843a25\") " pod="openshift-multus/multus-additional-cni-plugins-478c6" Sep 10 12:29:56.702610 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.702540 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/334e2bed-0266-4d0f-964f-96dcb9843a25-cni-binary-copy\") pod \"multus-additional-cni-plugins-478c6\" (UID: \"334e2bed-0266-4d0f-964f-96dcb9843a25\") " pod="openshift-multus/multus-additional-cni-plugins-478c6" Sep 10 12:29:56.702610 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.702573 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jccj\" (UniqueName: \"kubernetes.io/projected/334e2bed-0266-4d0f-964f-96dcb9843a25-kube-api-access-9jccj\") pod \"multus-additional-cni-plugins-478c6\" (UID: \"334e2bed-0266-4d0f-964f-96dcb9843a25\") " pod="openshift-multus/multus-additional-cni-plugins-478c6" Sep 10 12:29:56.702610 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.702604 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-etc-sysctl-conf\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.702903 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.702645 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-lib-modules\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.702903 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.702677 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64dj2\" (UniqueName: \"kubernetes.io/projected/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-kube-api-access-64dj2\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.702903 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.702705 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-host-var-lib-kubelet\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.702903 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.702751 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-hostroot\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.702903 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.702793 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-etc-sysconfig\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.702903 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.702846 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-etc-sysctl-d\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.703179 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.702916 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/df440146-c599-4722-b96d-986b68d93e40-iptables-alerter-script\") pod \"iptables-alerter-bjnv5\" (UID: \"df440146-c599-4722-b96d-986b68d93e40\") " pod="openshift-network-operator/iptables-alerter-bjnv5" Sep 10 12:29:56.703179 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.702964 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b66t\" (UniqueName: \"kubernetes.io/projected/df440146-c599-4722-b96d-986b68d93e40-kube-api-access-4b66t\") pod \"iptables-alerter-bjnv5\" (UID: \"df440146-c599-4722-b96d-986b68d93e40\") " pod="openshift-network-operator/iptables-alerter-bjnv5" Sep 10 12:29:56.703179 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.703035 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2d9a758b-9a83-405b-9b01-7143273959ff-hosts-file\") pod \"node-resolver-rwdsg\" (UID: \"2d9a758b-9a83-405b-9b01-7143273959ff\") " pod="openshift-dns/node-resolver-rwdsg" Sep 10 12:29:56.703179 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.703071 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76bf8bce-b7c3-4617-87e4-0be0765a14d0-cni-binary-copy\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.703179 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.703096 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-host-var-lib-cni-multus\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.703179 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.703134 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-multus-conf-dir\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.703466 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.703179 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-host-run-multus-certs\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.703466 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.703239 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-etc-tuned\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.703466 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.703271 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-multus-socket-dir-parent\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.703466 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.703298 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6t28\" (UniqueName: \"kubernetes.io/projected/76bf8bce-b7c3-4617-87e4-0be0765a14d0-kube-api-access-r6t28\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.703466 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.703354 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkcv6\" (UniqueName: \"kubernetes.io/projected/1019e2de-09cf-419b-a4ce-7e09921a4e04-kube-api-access-gkcv6\") pod \"node-ca-kmxmc\" (UID: \"1019e2de-09cf-419b-a4ce-7e09921a4e04\") " pod="openshift-image-registry/node-ca-kmxmc" Sep 10 12:29:56.703466 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.703379 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/334e2bed-0266-4d0f-964f-96dcb9843a25-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-478c6\" (UID: \"334e2bed-0266-4d0f-964f-96dcb9843a25\") " pod="openshift-multus/multus-additional-cni-plugins-478c6" Sep 10 12:29:56.703466 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.703405 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-etc-modprobe-d\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.703466 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.703432 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-etc-systemd\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.703466 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.703449 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-host\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.704115 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.703538 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-cnibin\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.704115 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.703567 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-host-run-k8s-cni-cncf-io\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.704115 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.703584 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/76bf8bce-b7c3-4617-87e4-0be0765a14d0-multus-daemon-config\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.704115 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.703624 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-etc-kubernetes\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.704115 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.703640 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-run\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.704115 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.703665 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-sys\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.704115 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.703693 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-var-lib-kubelet\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.704115 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.703749 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-system-cni-dir\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.704115 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.703866 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/df440146-c599-4722-b96d-986b68d93e40-host-slash\") pod \"iptables-alerter-bjnv5\" (UID: \"df440146-c599-4722-b96d-986b68d93e40\") " pod="openshift-network-operator/iptables-alerter-bjnv5" Sep 10 12:29:56.704793 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.704461 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 10 12:29:56.704793 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.704501 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-khq45\"" Sep 10 12:29:56.704793 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.704558 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 10 12:29:56.705571 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.705551 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" Sep 10 12:29:56.707934 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.707759 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 10 12:29:56.707934 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.707882 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 10 12:29:56.708180 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.708163 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-j7vwc\"" Sep 10 12:29:56.708365 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.708326 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:29:56.708454 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.708379 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 10 12:29:56.708454 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:56.708397 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmbbq" podUID="90ce254f-e418-4e45-ab41-427cd3a1626e" Sep 10 12:29:56.742420 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.742350 2588 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 12:24:55 +0000 UTC" deadline="2028-02-07 12:05:48.548968529 +0000 UTC" Sep 10 12:29:56.742420 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.742414 2588 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12359h35m51.806558096s" Sep 10 12:29:56.775258 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.775199 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-250.ec2.internal" event={"ID":"928ccfc5fa7f66e685a010324e61a3c7","Type":"ContainerStarted","Data":"e62edc79f1fb58871ef19ab8c8d1a8d31d8ddf26db0b41b77cd90e10c9f0f5e6"} Sep 10 12:29:56.776521 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.776486 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-250.ec2.internal" event={"ID":"b683db139541b51f843e32d0a18fc2d6","Type":"ContainerStarted","Data":"727eaaa5fe2370bfb1869e173a819e00faa885330705187522b4fb2246d5e68b"} Sep 10 12:29:56.791634 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.791598 2588 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 10 12:29:56.805014 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.804979 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/334e2bed-0266-4d0f-964f-96dcb9843a25-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-478c6\" (UID: \"334e2bed-0266-4d0f-964f-96dcb9843a25\") " pod="openshift-multus/multus-additional-cni-plugins-478c6" Sep 10 12:29:56.805014 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805016 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-etc-systemd\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.805268 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805031 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-host\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.805268 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805049 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-etc-kubernetes\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.805268 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805064 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g9ft\" (UniqueName: \"kubernetes.io/projected/2d9a758b-9a83-405b-9b01-7143273959ff-kube-api-access-8g9ft\") pod \"node-resolver-rwdsg\" (UID: \"2d9a758b-9a83-405b-9b01-7143273959ff\") " pod="openshift-dns/node-resolver-rwdsg" Sep 10 12:29:56.805268 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805082 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-host-slash\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.805268 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805099 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-host-cni-bin\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.805268 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805123 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-host-cni-netd\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.805268 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805137 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-etc-systemd\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.805268 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805138 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-host\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.805268 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805138 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-etc-kubernetes\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.805268 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805146 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-run\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.805712 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805290 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-run\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.805712 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805339 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-sys\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.805712 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805378 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6b43b97f-6877-402b-87be-390b1a42c4f8-konnectivity-ca\") pod \"konnectivity-agent-km6jk\" (UID: \"6b43b97f-6877-402b-87be-390b1a42c4f8\") " pod="kube-system/konnectivity-agent-km6jk" Sep 10 12:29:56.805712 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805400 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/25672a2d-d6e3-47aa-880d-0751856a6b47-env-overrides\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.805712 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805415 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-sys\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.805712 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805436 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/25672a2d-d6e3-47aa-880d-0751856a6b47-ovn-node-metrics-cert\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.805712 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805501 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km28k\" (UniqueName: \"kubernetes.io/projected/90ce254f-e418-4e45-ab41-427cd3a1626e-kube-api-access-km28k\") pod \"network-metrics-daemon-cmbbq\" (UID: \"90ce254f-e418-4e45-ab41-427cd3a1626e\") " pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:29:56.805712 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805567 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/334e2bed-0266-4d0f-964f-96dcb9843a25-system-cni-dir\") pod \"multus-additional-cni-plugins-478c6\" (UID: \"334e2bed-0266-4d0f-964f-96dcb9843a25\") " pod="openshift-multus/multus-additional-cni-plugins-478c6" Sep 10 12:29:56.805712 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805628 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/334e2bed-0266-4d0f-964f-96dcb9843a25-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-478c6\" (UID: \"334e2bed-0266-4d0f-964f-96dcb9843a25\") " pod="openshift-multus/multus-additional-cni-plugins-478c6" Sep 10 12:29:56.805712 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805618 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/334e2bed-0266-4d0f-964f-96dcb9843a25-system-cni-dir\") pod \"multus-additional-cni-plugins-478c6\" (UID: \"334e2bed-0266-4d0f-964f-96dcb9843a25\") " pod="openshift-multus/multus-additional-cni-plugins-478c6" Sep 10 12:29:56.805712 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805672 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-tmp\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.805712 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805718 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-host-run-netns\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.806277 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805756 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/334e2bed-0266-4d0f-964f-96dcb9843a25-tuning-conf-dir\") pod \"multus-additional-cni-plugins-478c6\" (UID: \"334e2bed-0266-4d0f-964f-96dcb9843a25\") " pod="openshift-multus/multus-additional-cni-plugins-478c6" Sep 10 12:29:56.806277 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805782 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-multus-cni-dir\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.806277 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805812 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-host-run-netns\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.806277 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805812 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqgph\" (UniqueName: \"kubernetes.io/projected/bdfce9e5-4daa-4b23-8f66-62549603ef7f-kube-api-access-jqgph\") pod \"network-check-target-p5wcp\" (UID: \"bdfce9e5-4daa-4b23-8f66-62549603ef7f\") " pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:29:56.806277 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805881 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-etc-openvswitch\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.806277 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805918 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-original-pull-secret\") pod \"global-pull-secret-syncer-wclkd\" (UID: \"cb45e2aa-d5a7-47e9-828e-186e39ffd6b8\") " pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:29:56.806277 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805943 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/334e2bed-0266-4d0f-964f-96dcb9843a25-cnibin\") pod \"multus-additional-cni-plugins-478c6\" (UID: \"334e2bed-0266-4d0f-964f-96dcb9843a25\") " pod="openshift-multus/multus-additional-cni-plugins-478c6" Sep 10 12:29:56.806277 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805985 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/334e2bed-0266-4d0f-964f-96dcb9843a25-tuning-conf-dir\") pod \"multus-additional-cni-plugins-478c6\" (UID: \"334e2bed-0266-4d0f-964f-96dcb9843a25\") " pod="openshift-multus/multus-additional-cni-plugins-478c6" Sep 10 12:29:56.806277 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.805992 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/334e2bed-0266-4d0f-964f-96dcb9843a25-cni-binary-copy\") pod \"multus-additional-cni-plugins-478c6\" (UID: \"334e2bed-0266-4d0f-964f-96dcb9843a25\") " pod="openshift-multus/multus-additional-cni-plugins-478c6" Sep 10 12:29:56.806277 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806040 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jccj\" (UniqueName: \"kubernetes.io/projected/334e2bed-0266-4d0f-964f-96dcb9843a25-kube-api-access-9jccj\") pod \"multus-additional-cni-plugins-478c6\" (UID: \"334e2bed-0266-4d0f-964f-96dcb9843a25\") " pod="openshift-multus/multus-additional-cni-plugins-478c6" Sep 10 12:29:56.806277 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806067 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-etc-sysctl-conf\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.806277 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806095 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64dj2\" (UniqueName: \"kubernetes.io/projected/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-kube-api-access-64dj2\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.806277 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806120 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-host-var-lib-kubelet\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.806277 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806152 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-hostroot\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.806277 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806181 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b66t\" (UniqueName: \"kubernetes.io/projected/df440146-c599-4722-b96d-986b68d93e40-kube-api-access-4b66t\") pod \"iptables-alerter-bjnv5\" (UID: \"df440146-c599-4722-b96d-986b68d93e40\") " pod="openshift-network-operator/iptables-alerter-bjnv5" Sep 10 12:29:56.806277 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806204 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-etc-sysconfig\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.806277 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806231 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-etc-sysctl-d\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.807057 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806255 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-host-run-netns\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.807057 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806289 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/25672a2d-d6e3-47aa-880d-0751856a6b47-ovnkube-script-lib\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.807057 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806339 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-kubelet-config\") pod \"global-pull-secret-syncer-wclkd\" (UID: \"cb45e2aa-d5a7-47e9-828e-186e39ffd6b8\") " pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:29:56.807057 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806385 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-dbus\") pod \"global-pull-secret-syncer-wclkd\" (UID: \"cb45e2aa-d5a7-47e9-828e-186e39ffd6b8\") " pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:29:56.807057 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806414 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76bf8bce-b7c3-4617-87e4-0be0765a14d0-cni-binary-copy\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.807057 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806418 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-multus-cni-dir\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.807057 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806467 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-host-var-lib-cni-multus\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.807057 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806495 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-multus-conf-dir\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.807057 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806515 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/334e2bed-0266-4d0f-964f-96dcb9843a25-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-478c6\" (UID: \"334e2bed-0266-4d0f-964f-96dcb9843a25\") " pod="openshift-multus/multus-additional-cni-plugins-478c6" Sep 10 12:29:56.807057 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806527 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/334e2bed-0266-4d0f-964f-96dcb9843a25-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-478c6\" (UID: \"334e2bed-0266-4d0f-964f-96dcb9843a25\") " pod="openshift-multus/multus-additional-cni-plugins-478c6" Sep 10 12:29:56.807057 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806535 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fcfb96de-97a4-4abc-a215-32d210dda11d-registration-dir\") pod \"aws-ebs-csi-driver-node-5dwzp\" (UID: \"fcfb96de-97a4-4abc-a215-32d210dda11d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" Sep 10 12:29:56.807057 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806558 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-run-ovn\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.807057 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806584 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-etc-tuned\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.807057 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806572 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/334e2bed-0266-4d0f-964f-96dcb9843a25-cni-binary-copy\") pod \"multus-additional-cni-plugins-478c6\" (UID: \"334e2bed-0266-4d0f-964f-96dcb9843a25\") " pod="openshift-multus/multus-additional-cni-plugins-478c6" Sep 10 12:29:56.807057 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806318 2588 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 10 12:29:56.807057 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806683 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkcv6\" (UniqueName: \"kubernetes.io/projected/1019e2de-09cf-419b-a4ce-7e09921a4e04-kube-api-access-gkcv6\") pod \"node-ca-kmxmc\" (UID: \"1019e2de-09cf-419b-a4ce-7e09921a4e04\") " pod="openshift-image-registry/node-ca-kmxmc" Sep 10 12:29:56.807057 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806715 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-etc-modprobe-d\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.807858 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806764 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-cnibin\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.807858 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806794 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-host-run-k8s-cni-cncf-io\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.807858 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806849 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/76bf8bce-b7c3-4617-87e4-0be0765a14d0-multus-daemon-config\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.807858 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806870 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-etc-sysconfig\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.807858 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806884 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d9a758b-9a83-405b-9b01-7143273959ff-tmp-dir\") pod \"node-resolver-rwdsg\" (UID: \"2d9a758b-9a83-405b-9b01-7143273959ff\") " pod="openshift-dns/node-resolver-rwdsg" Sep 10 12:29:56.807858 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806889 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-host-var-lib-kubelet\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.807858 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806915 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-node-log\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.807858 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806969 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-etc-sysctl-d\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.807858 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.806853 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-etc-sysctl-conf\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.807858 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807047 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-var-lib-kubelet\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.807858 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807075 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-system-cni-dir\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.807858 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807103 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/df440146-c599-4722-b96d-986b68d93e40-host-slash\") pod \"iptables-alerter-bjnv5\" (UID: \"df440146-c599-4722-b96d-986b68d93e40\") " pod="openshift-network-operator/iptables-alerter-bjnv5" Sep 10 12:29:56.807858 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807113 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-var-lib-kubelet\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.807858 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807128 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1019e2de-09cf-419b-a4ce-7e09921a4e04-host\") pod \"node-ca-kmxmc\" (UID: \"1019e2de-09cf-419b-a4ce-7e09921a4e04\") " pod="openshift-image-registry/node-ca-kmxmc" Sep 10 12:29:56.807858 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807156 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dc5dm\" (UniqueName: \"kubernetes.io/projected/fcfb96de-97a4-4abc-a215-32d210dda11d-kube-api-access-dc5dm\") pod \"aws-ebs-csi-driver-node-5dwzp\" (UID: \"fcfb96de-97a4-4abc-a215-32d210dda11d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" Sep 10 12:29:56.807858 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807186 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-systemd-units\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.807858 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807223 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-run-openvswitch\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.807858 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807174 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-multus-conf-dir\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.808604 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807296 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-hostroot\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.808604 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807317 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1019e2de-09cf-419b-a4ce-7e09921a4e04-host\") pod \"node-ca-kmxmc\" (UID: \"1019e2de-09cf-419b-a4ce-7e09921a4e04\") " pod="openshift-image-registry/node-ca-kmxmc" Sep 10 12:29:56.808604 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807326 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-etc-kubernetes\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.808604 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807351 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-etc-modprobe-d\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.808604 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807355 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-host-var-lib-cni-multus\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.808604 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807368 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-host-var-lib-cni-bin\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.808604 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807392 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/df440146-c599-4722-b96d-986b68d93e40-host-slash\") pod \"iptables-alerter-bjnv5\" (UID: \"df440146-c599-4722-b96d-986b68d93e40\") " pod="openshift-network-operator/iptables-alerter-bjnv5" Sep 10 12:29:56.808604 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807359 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-system-cni-dir\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.808604 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807396 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1019e2de-09cf-419b-a4ce-7e09921a4e04-serviceca\") pod \"node-ca-kmxmc\" (UID: \"1019e2de-09cf-419b-a4ce-7e09921a4e04\") " pod="openshift-image-registry/node-ca-kmxmc" Sep 10 12:29:56.808604 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807425 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76bf8bce-b7c3-4617-87e4-0be0765a14d0-cni-binary-copy\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.808604 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807438 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/334e2bed-0266-4d0f-964f-96dcb9843a25-cnibin\") pod \"multus-additional-cni-plugins-478c6\" (UID: \"334e2bed-0266-4d0f-964f-96dcb9843a25\") " pod="openshift-multus/multus-additional-cni-plugins-478c6" Sep 10 12:29:56.808604 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807446 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-etc-kubernetes\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.808604 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807404 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-cnibin\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.808604 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807462 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-host-var-lib-cni-bin\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.808604 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807563 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-host-run-k8s-cni-cncf-io\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.808604 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807608 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fcfb96de-97a4-4abc-a215-32d210dda11d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5dwzp\" (UID: \"fcfb96de-97a4-4abc-a215-32d210dda11d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" Sep 10 12:29:56.808604 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807701 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fcfb96de-97a4-4abc-a215-32d210dda11d-socket-dir\") pod \"aws-ebs-csi-driver-node-5dwzp\" (UID: \"fcfb96de-97a4-4abc-a215-32d210dda11d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" Sep 10 12:29:56.808604 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807770 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fcfb96de-97a4-4abc-a215-32d210dda11d-etc-selinux\") pod \"aws-ebs-csi-driver-node-5dwzp\" (UID: \"fcfb96de-97a4-4abc-a215-32d210dda11d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" Sep 10 12:29:56.809414 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807819 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/25672a2d-d6e3-47aa-880d-0751856a6b47-ovnkube-config\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.809414 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807853 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1019e2de-09cf-419b-a4ce-7e09921a4e04-serviceca\") pod \"node-ca-kmxmc\" (UID: \"1019e2de-09cf-419b-a4ce-7e09921a4e04\") " pod="openshift-image-registry/node-ca-kmxmc" Sep 10 12:29:56.809414 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807846 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-lib-modules\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.809414 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807900 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2d9a758b-9a83-405b-9b01-7143273959ff-hosts-file\") pod \"node-resolver-rwdsg\" (UID: \"2d9a758b-9a83-405b-9b01-7143273959ff\") " pod="openshift-dns/node-resolver-rwdsg" Sep 10 12:29:56.809414 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807916 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d9a758b-9a83-405b-9b01-7143273959ff-tmp-dir\") pod \"node-resolver-rwdsg\" (UID: \"2d9a758b-9a83-405b-9b01-7143273959ff\") " pod="openshift-dns/node-resolver-rwdsg" Sep 10 12:29:56.809414 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807947 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-lib-modules\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.809414 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807964 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fcfb96de-97a4-4abc-a215-32d210dda11d-device-dir\") pod \"aws-ebs-csi-driver-node-5dwzp\" (UID: \"fcfb96de-97a4-4abc-a215-32d210dda11d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" Sep 10 12:29:56.809414 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807984 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2d9a758b-9a83-405b-9b01-7143273959ff-hosts-file\") pod \"node-resolver-rwdsg\" (UID: \"2d9a758b-9a83-405b-9b01-7143273959ff\") " pod="openshift-dns/node-resolver-rwdsg" Sep 10 12:29:56.809414 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.807993 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs\") pod \"network-metrics-daemon-cmbbq\" (UID: \"90ce254f-e418-4e45-ab41-427cd3a1626e\") " pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:29:56.809414 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.808027 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/df440146-c599-4722-b96d-986b68d93e40-iptables-alerter-script\") pod \"iptables-alerter-bjnv5\" (UID: \"df440146-c599-4722-b96d-986b68d93e40\") " pod="openshift-network-operator/iptables-alerter-bjnv5" Sep 10 12:29:56.809414 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.808031 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/76bf8bce-b7c3-4617-87e4-0be0765a14d0-multus-daemon-config\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.809414 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.808054 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-host-kubelet\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.809414 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.808092 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-host-run-multus-certs\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.809414 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.808118 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fcfb96de-97a4-4abc-a215-32d210dda11d-sys-fs\") pod \"aws-ebs-csi-driver-node-5dwzp\" (UID: \"fcfb96de-97a4-4abc-a215-32d210dda11d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" Sep 10 12:29:56.809414 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.808144 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-run-systemd\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.809414 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.808148 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-host-run-multus-certs\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.809414 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.808166 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-var-lib-openvswitch\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.810203 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.808190 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-log-socket\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.810203 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.808211 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-host-run-ovn-kubernetes\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.810203 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.808231 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.810203 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.808260 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-multus-socket-dir-parent\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.810203 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.808283 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6t28\" (UniqueName: \"kubernetes.io/projected/76bf8bce-b7c3-4617-87e4-0be0765a14d0-kube-api-access-r6t28\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.810203 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.808300 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6b43b97f-6877-402b-87be-390b1a42c4f8-agent-certs\") pod \"konnectivity-agent-km6jk\" (UID: \"6b43b97f-6877-402b-87be-390b1a42c4f8\") " pod="kube-system/konnectivity-agent-km6jk" Sep 10 12:29:56.810203 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.808323 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5nfx\" (UniqueName: \"kubernetes.io/projected/25672a2d-d6e3-47aa-880d-0751856a6b47-kube-api-access-j5nfx\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.810203 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.808327 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/76bf8bce-b7c3-4617-87e4-0be0765a14d0-multus-socket-dir-parent\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.810203 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.808490 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/df440146-c599-4722-b96d-986b68d93e40-iptables-alerter-script\") pod \"iptables-alerter-bjnv5\" (UID: \"df440146-c599-4722-b96d-986b68d93e40\") " pod="openshift-network-operator/iptables-alerter-bjnv5" Sep 10 12:29:56.811181 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.811134 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-etc-tuned\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.811181 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.811171 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-tmp\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.817970 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:56.817935 2588 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:29:56.817970 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:56.817973 2588 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:29:56.818183 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:56.817988 2588 projected.go:196] Error preparing data for projected volume kube-api-access-jqgph for pod openshift-network-diagnostics/network-check-target-p5wcp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:29:56.818244 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:56.818092 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdfce9e5-4daa-4b23-8f66-62549603ef7f-kube-api-access-jqgph podName:bdfce9e5-4daa-4b23-8f66-62549603ef7f nodeName:}" failed. No retries permitted until 2026-09-10 12:29:57.31805363 +0000 UTC m=+3.051648345 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jqgph" (UniqueName: "kubernetes.io/projected/bdfce9e5-4daa-4b23-8f66-62549603ef7f-kube-api-access-jqgph") pod "network-check-target-p5wcp" (UID: "bdfce9e5-4daa-4b23-8f66-62549603ef7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:29:56.819557 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.819053 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g9ft\" (UniqueName: \"kubernetes.io/projected/2d9a758b-9a83-405b-9b01-7143273959ff-kube-api-access-8g9ft\") pod \"node-resolver-rwdsg\" (UID: \"2d9a758b-9a83-405b-9b01-7143273959ff\") " pod="openshift-dns/node-resolver-rwdsg" Sep 10 12:29:56.823758 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.823682 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkcv6\" (UniqueName: \"kubernetes.io/projected/1019e2de-09cf-419b-a4ce-7e09921a4e04-kube-api-access-gkcv6\") pod \"node-ca-kmxmc\" (UID: \"1019e2de-09cf-419b-a4ce-7e09921a4e04\") " pod="openshift-image-registry/node-ca-kmxmc" Sep 10 12:29:56.824417 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.824389 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jccj\" (UniqueName: \"kubernetes.io/projected/334e2bed-0266-4d0f-964f-96dcb9843a25-kube-api-access-9jccj\") pod \"multus-additional-cni-plugins-478c6\" (UID: \"334e2bed-0266-4d0f-964f-96dcb9843a25\") " pod="openshift-multus/multus-additional-cni-plugins-478c6" Sep 10 12:29:56.825779 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.825749 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64dj2\" (UniqueName: \"kubernetes.io/projected/2ed00d5c-9e03-40b5-b261-deeb8aaa46d1-kube-api-access-64dj2\") pod \"tuned-xhmjp\" (UID: \"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1\") " pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:56.825982 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.825955 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6t28\" (UniqueName: \"kubernetes.io/projected/76bf8bce-b7c3-4617-87e4-0be0765a14d0-kube-api-access-r6t28\") pod \"multus-tc789\" (UID: \"76bf8bce-b7c3-4617-87e4-0be0765a14d0\") " pod="openshift-multus/multus-tc789" Sep 10 12:29:56.826891 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.826869 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b66t\" (UniqueName: \"kubernetes.io/projected/df440146-c599-4722-b96d-986b68d93e40-kube-api-access-4b66t\") pod \"iptables-alerter-bjnv5\" (UID: \"df440146-c599-4722-b96d-986b68d93e40\") " pod="openshift-network-operator/iptables-alerter-bjnv5" Sep 10 12:29:56.909192 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.908970 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-host-run-ovn-kubernetes\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.909192 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909025 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.909192 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909056 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6b43b97f-6877-402b-87be-390b1a42c4f8-agent-certs\") pod \"konnectivity-agent-km6jk\" (UID: \"6b43b97f-6877-402b-87be-390b1a42c4f8\") " pod="kube-system/konnectivity-agent-km6jk" Sep 10 12:29:56.909192 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909081 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5nfx\" (UniqueName: \"kubernetes.io/projected/25672a2d-d6e3-47aa-880d-0751856a6b47-kube-api-access-j5nfx\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.909192 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909090 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-host-run-ovn-kubernetes\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.909192 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909109 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-host-slash\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.909192 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909133 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-host-cni-bin\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.909192 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909157 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-host-cni-netd\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.909192 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909159 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.909192 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909181 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6b43b97f-6877-402b-87be-390b1a42c4f8-konnectivity-ca\") pod \"konnectivity-agent-km6jk\" (UID: \"6b43b97f-6877-402b-87be-390b1a42c4f8\") " pod="kube-system/konnectivity-agent-km6jk" Sep 10 12:29:56.909192 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909202 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/25672a2d-d6e3-47aa-880d-0751856a6b47-env-overrides\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.909955 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909223 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/25672a2d-d6e3-47aa-880d-0751856a6b47-ovn-node-metrics-cert\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.909955 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909227 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-host-cni-bin\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.909955 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909245 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km28k\" (UniqueName: \"kubernetes.io/projected/90ce254f-e418-4e45-ab41-427cd3a1626e-kube-api-access-km28k\") pod \"network-metrics-daemon-cmbbq\" (UID: \"90ce254f-e418-4e45-ab41-427cd3a1626e\") " pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:29:56.909955 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909307 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-etc-openvswitch\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.909955 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909335 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-original-pull-secret\") pod \"global-pull-secret-syncer-wclkd\" (UID: \"cb45e2aa-d5a7-47e9-828e-186e39ffd6b8\") " pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:29:56.909955 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909369 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-host-run-netns\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.909955 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909393 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/25672a2d-d6e3-47aa-880d-0751856a6b47-ovnkube-script-lib\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.909955 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909417 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-kubelet-config\") pod \"global-pull-secret-syncer-wclkd\" (UID: \"cb45e2aa-d5a7-47e9-828e-186e39ffd6b8\") " pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:29:56.909955 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909439 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-dbus\") pod \"global-pull-secret-syncer-wclkd\" (UID: \"cb45e2aa-d5a7-47e9-828e-186e39ffd6b8\") " pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:29:56.909955 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909467 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fcfb96de-97a4-4abc-a215-32d210dda11d-registration-dir\") pod \"aws-ebs-csi-driver-node-5dwzp\" (UID: \"fcfb96de-97a4-4abc-a215-32d210dda11d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" Sep 10 12:29:56.909955 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909492 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-run-ovn\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.909955 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909201 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-host-slash\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.909955 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909574 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-node-log\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.909955 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:56.909616 2588 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:29:56.909955 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909633 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-etc-openvswitch\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.909955 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909696 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-kubelet-config\") pod \"global-pull-secret-syncer-wclkd\" (UID: \"cb45e2aa-d5a7-47e9-828e-186e39ffd6b8\") " pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:29:56.909955 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909754 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-host-run-netns\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.910673 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:56.909780 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-original-pull-secret podName:cb45e2aa-d5a7-47e9-828e-186e39ffd6b8 nodeName:}" failed. No retries permitted until 2026-09-10 12:29:57.409761416 +0000 UTC m=+3.143356128 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-original-pull-secret") pod "global-pull-secret-syncer-wclkd" (UID: "cb45e2aa-d5a7-47e9-828e-186e39ffd6b8") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:29:56.910673 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909829 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fcfb96de-97a4-4abc-a215-32d210dda11d-registration-dir\") pod \"aws-ebs-csi-driver-node-5dwzp\" (UID: \"fcfb96de-97a4-4abc-a215-32d210dda11d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" Sep 10 12:29:56.910673 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909864 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/25672a2d-d6e3-47aa-880d-0751856a6b47-env-overrides\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.910673 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909866 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-run-ovn\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.910673 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909910 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-host-cni-netd\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.910673 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.909522 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-node-log\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.910673 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910077 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dc5dm\" (UniqueName: \"kubernetes.io/projected/fcfb96de-97a4-4abc-a215-32d210dda11d-kube-api-access-dc5dm\") pod \"aws-ebs-csi-driver-node-5dwzp\" (UID: \"fcfb96de-97a4-4abc-a215-32d210dda11d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" Sep 10 12:29:56.910673 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910104 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-systemd-units\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.910673 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910127 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-run-openvswitch\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.910673 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910153 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fcfb96de-97a4-4abc-a215-32d210dda11d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5dwzp\" (UID: \"fcfb96de-97a4-4abc-a215-32d210dda11d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" Sep 10 12:29:56.910673 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910177 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fcfb96de-97a4-4abc-a215-32d210dda11d-socket-dir\") pod \"aws-ebs-csi-driver-node-5dwzp\" (UID: \"fcfb96de-97a4-4abc-a215-32d210dda11d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" Sep 10 12:29:56.910673 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910201 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fcfb96de-97a4-4abc-a215-32d210dda11d-etc-selinux\") pod \"aws-ebs-csi-driver-node-5dwzp\" (UID: \"fcfb96de-97a4-4abc-a215-32d210dda11d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" Sep 10 12:29:56.910673 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910231 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/25672a2d-d6e3-47aa-880d-0751856a6b47-ovnkube-config\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.910673 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910269 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/25672a2d-d6e3-47aa-880d-0751856a6b47-ovnkube-script-lib\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.910673 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910288 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fcfb96de-97a4-4abc-a215-32d210dda11d-device-dir\") pod \"aws-ebs-csi-driver-node-5dwzp\" (UID: \"fcfb96de-97a4-4abc-a215-32d210dda11d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" Sep 10 12:29:56.910673 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910312 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs\") pod \"network-metrics-daemon-cmbbq\" (UID: \"90ce254f-e418-4e45-ab41-427cd3a1626e\") " pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:29:56.911433 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910334 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-host-kubelet\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.911433 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910356 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fcfb96de-97a4-4abc-a215-32d210dda11d-sys-fs\") pod \"aws-ebs-csi-driver-node-5dwzp\" (UID: \"fcfb96de-97a4-4abc-a215-32d210dda11d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" Sep 10 12:29:56.911433 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910359 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6b43b97f-6877-402b-87be-390b1a42c4f8-konnectivity-ca\") pod \"konnectivity-agent-km6jk\" (UID: \"6b43b97f-6877-402b-87be-390b1a42c4f8\") " pod="kube-system/konnectivity-agent-km6jk" Sep 10 12:29:56.911433 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910387 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-run-systemd\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.911433 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910416 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-dbus\") pod \"global-pull-secret-syncer-wclkd\" (UID: \"cb45e2aa-d5a7-47e9-828e-186e39ffd6b8\") " pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:29:56.911433 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910443 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-var-lib-openvswitch\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.911433 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910459 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-run-openvswitch\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.911433 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910470 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-log-socket\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.911433 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910498 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-systemd-units\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.911433 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910586 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fcfb96de-97a4-4abc-a215-32d210dda11d-sys-fs\") pod \"aws-ebs-csi-driver-node-5dwzp\" (UID: \"fcfb96de-97a4-4abc-a215-32d210dda11d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" Sep 10 12:29:56.911433 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:56.910592 2588 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:29:56.911433 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910616 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fcfb96de-97a4-4abc-a215-32d210dda11d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5dwzp\" (UID: \"fcfb96de-97a4-4abc-a215-32d210dda11d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" Sep 10 12:29:56.911433 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910625 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-run-systemd\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.911433 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:56.910647 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs podName:90ce254f-e418-4e45-ab41-427cd3a1626e nodeName:}" failed. No retries permitted until 2026-09-10 12:29:57.410632093 +0000 UTC m=+3.144226797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs") pod "network-metrics-daemon-cmbbq" (UID: "90ce254f-e418-4e45-ab41-427cd3a1626e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:29:56.911433 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910657 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fcfb96de-97a4-4abc-a215-32d210dda11d-device-dir\") pod \"aws-ebs-csi-driver-node-5dwzp\" (UID: \"fcfb96de-97a4-4abc-a215-32d210dda11d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" Sep 10 12:29:56.911433 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910678 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-host-kubelet\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.911433 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910718 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-var-lib-openvswitch\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.912238 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910720 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fcfb96de-97a4-4abc-a215-32d210dda11d-etc-selinux\") pod \"aws-ebs-csi-driver-node-5dwzp\" (UID: \"fcfb96de-97a4-4abc-a215-32d210dda11d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" Sep 10 12:29:56.912238 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910778 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/25672a2d-d6e3-47aa-880d-0751856a6b47-log-socket\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.912238 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.910874 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fcfb96de-97a4-4abc-a215-32d210dda11d-socket-dir\") pod \"aws-ebs-csi-driver-node-5dwzp\" (UID: \"fcfb96de-97a4-4abc-a215-32d210dda11d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" Sep 10 12:29:56.912238 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.911068 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/25672a2d-d6e3-47aa-880d-0751856a6b47-ovnkube-config\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.913054 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.913022 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/25672a2d-d6e3-47aa-880d-0751856a6b47-ovn-node-metrics-cert\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.913192 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.913122 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6b43b97f-6877-402b-87be-390b1a42c4f8-agent-certs\") pod \"konnectivity-agent-km6jk\" (UID: \"6b43b97f-6877-402b-87be-390b1a42c4f8\") " pod="kube-system/konnectivity-agent-km6jk" Sep 10 12:29:56.921857 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.921825 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-km28k\" (UniqueName: \"kubernetes.io/projected/90ce254f-e418-4e45-ab41-427cd3a1626e-kube-api-access-km28k\") pod \"network-metrics-daemon-cmbbq\" (UID: \"90ce254f-e418-4e45-ab41-427cd3a1626e\") " pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:29:56.922945 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.922918 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5nfx\" (UniqueName: \"kubernetes.io/projected/25672a2d-d6e3-47aa-880d-0751856a6b47-kube-api-access-j5nfx\") pod \"ovnkube-node-b47ts\" (UID: \"25672a2d-d6e3-47aa-880d-0751856a6b47\") " pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:56.924019 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.923997 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dc5dm\" (UniqueName: \"kubernetes.io/projected/fcfb96de-97a4-4abc-a215-32d210dda11d-kube-api-access-dc5dm\") pod \"aws-ebs-csi-driver-node-5dwzp\" (UID: \"fcfb96de-97a4-4abc-a215-32d210dda11d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" Sep 10 12:29:56.993559 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:56.993462 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bjnv5" Sep 10 12:29:57.007716 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:57.007679 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-478c6" Sep 10 12:29:57.016802 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:57.016772 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" Sep 10 12:29:57.022767 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:57.022720 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kmxmc" Sep 10 12:29:57.030529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:57.030494 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rwdsg" Sep 10 12:29:57.041678 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:57.041641 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tc789" Sep 10 12:29:57.048960 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:57.048851 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:29:57.056026 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:57.055852 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-km6jk" Sep 10 12:29:57.063883 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:57.063805 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" Sep 10 12:29:57.414893 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:57.414858 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqgph\" (UniqueName: \"kubernetes.io/projected/bdfce9e5-4daa-4b23-8f66-62549603ef7f-kube-api-access-jqgph\") pod \"network-check-target-p5wcp\" (UID: \"bdfce9e5-4daa-4b23-8f66-62549603ef7f\") " pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:29:57.414893 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:57.414897 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-original-pull-secret\") pod \"global-pull-secret-syncer-wclkd\" (UID: \"cb45e2aa-d5a7-47e9-828e-186e39ffd6b8\") " pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:29:57.415091 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:57.414934 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs\") pod \"network-metrics-daemon-cmbbq\" (UID: \"90ce254f-e418-4e45-ab41-427cd3a1626e\") " pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:29:57.415091 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:57.415032 2588 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:29:57.415091 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:57.415091 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-original-pull-secret podName:cb45e2aa-d5a7-47e9-828e-186e39ffd6b8 nodeName:}" failed. No retries permitted until 2026-09-10 12:29:58.415078234 +0000 UTC m=+4.148672933 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-original-pull-secret") pod "global-pull-secret-syncer-wclkd" (UID: "cb45e2aa-d5a7-47e9-828e-186e39ffd6b8") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:29:57.415192 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:57.415032 2588 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:29:57.415192 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:57.415170 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs podName:90ce254f-e418-4e45-ab41-427cd3a1626e nodeName:}" failed. No retries permitted until 2026-09-10 12:29:58.415157844 +0000 UTC m=+4.148752551 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs") pod "network-metrics-daemon-cmbbq" (UID: "90ce254f-e418-4e45-ab41-427cd3a1626e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:29:57.415192 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:57.415034 2588 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:29:57.415282 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:57.415195 2588 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:29:57.415282 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:57.415205 2588 projected.go:196] Error preparing data for projected volume kube-api-access-jqgph for pod openshift-network-diagnostics/network-check-target-p5wcp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:29:57.415282 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:57.415235 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdfce9e5-4daa-4b23-8f66-62549603ef7f-kube-api-access-jqgph podName:bdfce9e5-4daa-4b23-8f66-62549603ef7f nodeName:}" failed. No retries permitted until 2026-09-10 12:29:58.415227772 +0000 UTC m=+4.148822467 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jqgph" (UniqueName: "kubernetes.io/projected/bdfce9e5-4daa-4b23-8f66-62549603ef7f-kube-api-access-jqgph") pod "network-check-target-p5wcp" (UID: "bdfce9e5-4daa-4b23-8f66-62549603ef7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:29:57.743342 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:57.743218 2588 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 12:24:55 +0000 UTC" deadline="2028-02-21 17:46:01.635471127 +0000 UTC" Sep 10 12:29:57.743342 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:57.743281 2588 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12701h16m3.892194505s" Sep 10 12:29:58.033955 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:58.033923 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25672a2d_d6e3_47aa_880d_0751856a6b47.slice/crio-85785cc5062ff080be1d70bfd445f2b153117ef3851f54e98f6ca34941689eb6 WatchSource:0}: Error finding container 85785cc5062ff080be1d70bfd445f2b153117ef3851f54e98f6ca34941689eb6: Status 404 returned error can't find the container with id 85785cc5062ff080be1d70bfd445f2b153117ef3851f54e98f6ca34941689eb6 Sep 10 12:29:58.043028 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:58.043000 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76bf8bce_b7c3_4617_87e4_0be0765a14d0.slice/crio-7e15d17df0f77aeb830a193d416c68d789894106838a374e1d6968d6ec1e1483 WatchSource:0}: Error finding container 7e15d17df0f77aeb830a193d416c68d789894106838a374e1d6968d6ec1e1483: Status 404 returned error can't find the container with id 7e15d17df0f77aeb830a193d416c68d789894106838a374e1d6968d6ec1e1483 Sep 10 12:29:58.044697 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:58.044669 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf440146_c599_4722_b96d_986b68d93e40.slice/crio-3b2bfbfb2bd7d90359099ac59a013a584d2e5931376f7b351fbc553f7a76e312 WatchSource:0}: Error finding container 3b2bfbfb2bd7d90359099ac59a013a584d2e5931376f7b351fbc553f7a76e312: Status 404 returned error can't find the container with id 3b2bfbfb2bd7d90359099ac59a013a584d2e5931376f7b351fbc553f7a76e312 Sep 10 12:29:58.045486 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:58.045450 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod334e2bed_0266_4d0f_964f_96dcb9843a25.slice/crio-dbc3b742b5acb2c5ff3447c23cdc211b0ea3675014a0bc31f5609feed49dd549 WatchSource:0}: Error finding container dbc3b742b5acb2c5ff3447c23cdc211b0ea3675014a0bc31f5609feed49dd549: Status 404 returned error can't find the container with id dbc3b742b5acb2c5ff3447c23cdc211b0ea3675014a0bc31f5609feed49dd549 Sep 10 12:29:58.046217 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:58.046189 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d9a758b_9a83_405b_9b01_7143273959ff.slice/crio-a42bd6a88a60a33cb5c0578b32fec5d1f03c55cad64ff5a307bdb870b53e2da8 WatchSource:0}: Error finding container a42bd6a88a60a33cb5c0578b32fec5d1f03c55cad64ff5a307bdb870b53e2da8: Status 404 returned error can't find the container with id a42bd6a88a60a33cb5c0578b32fec5d1f03c55cad64ff5a307bdb870b53e2da8 Sep 10 12:29:58.047322 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:58.047239 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcfb96de_97a4_4abc_a215_32d210dda11d.slice/crio-8f3e620f7bc39714f64fb6d953384df64698a5a88ee83f774e013aeff14df5a3 WatchSource:0}: Error finding container 8f3e620f7bc39714f64fb6d953384df64698a5a88ee83f774e013aeff14df5a3: Status 404 returned error can't find the container with id 8f3e620f7bc39714f64fb6d953384df64698a5a88ee83f774e013aeff14df5a3 Sep 10 12:29:58.049203 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:29:58.048894 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b43b97f_6877_402b_87be_390b1a42c4f8.slice/crio-6622137d20933bdbb54ea2324c3769ab26f76d50796b06e9e79243a94a418983 WatchSource:0}: Error finding container 6622137d20933bdbb54ea2324c3769ab26f76d50796b06e9e79243a94a418983: Status 404 returned error can't find the container with id 6622137d20933bdbb54ea2324c3769ab26f76d50796b06e9e79243a94a418983 Sep 10 12:29:58.421665 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:58.421616 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs\") pod \"network-metrics-daemon-cmbbq\" (UID: \"90ce254f-e418-4e45-ab41-427cd3a1626e\") " pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:29:58.421893 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:58.421682 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqgph\" (UniqueName: \"kubernetes.io/projected/bdfce9e5-4daa-4b23-8f66-62549603ef7f-kube-api-access-jqgph\") pod \"network-check-target-p5wcp\" (UID: \"bdfce9e5-4daa-4b23-8f66-62549603ef7f\") " pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:29:58.421893 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:58.421711 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-original-pull-secret\") pod \"global-pull-secret-syncer-wclkd\" (UID: \"cb45e2aa-d5a7-47e9-828e-186e39ffd6b8\") " pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:29:58.421893 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:58.421808 2588 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:29:58.421893 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:58.421829 2588 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:29:58.421893 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:58.421891 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-original-pull-secret podName:cb45e2aa-d5a7-47e9-828e-186e39ffd6b8 nodeName:}" failed. No retries permitted until 2026-09-10 12:30:00.421874128 +0000 UTC m=+6.155468840 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-original-pull-secret") pod "global-pull-secret-syncer-wclkd" (UID: "cb45e2aa-d5a7-47e9-828e-186e39ffd6b8") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:29:58.422157 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:58.421829 2588 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:29:58.422157 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:58.421959 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs podName:90ce254f-e418-4e45-ab41-427cd3a1626e nodeName:}" failed. No retries permitted until 2026-09-10 12:30:00.42194651 +0000 UTC m=+6.155541205 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs") pod "network-metrics-daemon-cmbbq" (UID: "90ce254f-e418-4e45-ab41-427cd3a1626e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:29:58.422157 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:58.421831 2588 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:29:58.422157 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:58.421979 2588 projected.go:196] Error preparing data for projected volume kube-api-access-jqgph for pod openshift-network-diagnostics/network-check-target-p5wcp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:29:58.422157 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:58.422006 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdfce9e5-4daa-4b23-8f66-62549603ef7f-kube-api-access-jqgph podName:bdfce9e5-4daa-4b23-8f66-62549603ef7f nodeName:}" failed. No retries permitted until 2026-09-10 12:30:00.421999484 +0000 UTC m=+6.155594178 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jqgph" (UniqueName: "kubernetes.io/projected/bdfce9e5-4daa-4b23-8f66-62549603ef7f-kube-api-access-jqgph") pod "network-check-target-p5wcp" (UID: "bdfce9e5-4daa-4b23-8f66-62549603ef7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:29:58.773756 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:58.773703 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:29:58.774698 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:58.773868 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wclkd" podUID="cb45e2aa-d5a7-47e9-828e-186e39ffd6b8" Sep 10 12:29:58.774698 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:58.774047 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:29:58.774698 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:58.774139 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmbbq" podUID="90ce254f-e418-4e45-ab41-427cd3a1626e" Sep 10 12:29:58.774698 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:58.774200 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:29:58.774698 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:29:58.774262 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5wcp" podUID="bdfce9e5-4daa-4b23-8f66-62549603ef7f" Sep 10 12:29:58.787769 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:58.787702 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-250.ec2.internal" event={"ID":"928ccfc5fa7f66e685a010324e61a3c7","Type":"ContainerStarted","Data":"3b29c9b485411c84a52081410550f3ae0dc27c8c275e2e4455f4e93758ca8e83"} Sep 10 12:29:58.795883 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:58.795847 2588 generic.go:356] "Generic (PLEG): container finished" podID="b683db139541b51f843e32d0a18fc2d6" containerID="8951bd731dfaa7c11d26edaafa6e69e678305bce041780c502fd568ac1083584" exitCode=0 Sep 10 12:29:58.796169 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:58.796137 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-250.ec2.internal" event={"ID":"b683db139541b51f843e32d0a18fc2d6","Type":"ContainerDied","Data":"8951bd731dfaa7c11d26edaafa6e69e678305bce041780c502fd568ac1083584"} Sep 10 12:29:58.801435 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:58.801401 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-km6jk" event={"ID":"6b43b97f-6877-402b-87be-390b1a42c4f8","Type":"ContainerStarted","Data":"6622137d20933bdbb54ea2324c3769ab26f76d50796b06e9e79243a94a418983"} Sep 10 12:29:58.805149 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:58.805078 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" event={"ID":"fcfb96de-97a4-4abc-a215-32d210dda11d","Type":"ContainerStarted","Data":"8f3e620f7bc39714f64fb6d953384df64698a5a88ee83f774e013aeff14df5a3"} Sep 10 12:29:58.809958 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:58.809903 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rwdsg" event={"ID":"2d9a758b-9a83-405b-9b01-7143273959ff","Type":"ContainerStarted","Data":"a42bd6a88a60a33cb5c0578b32fec5d1f03c55cad64ff5a307bdb870b53e2da8"} Sep 10 12:29:58.816445 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:58.816351 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bjnv5" event={"ID":"df440146-c599-4722-b96d-986b68d93e40","Type":"ContainerStarted","Data":"3b2bfbfb2bd7d90359099ac59a013a584d2e5931376f7b351fbc553f7a76e312"} Sep 10 12:29:58.821161 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:58.821128 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tc789" event={"ID":"76bf8bce-b7c3-4617-87e4-0be0765a14d0","Type":"ContainerStarted","Data":"7e15d17df0f77aeb830a193d416c68d789894106838a374e1d6968d6ec1e1483"} Sep 10 12:29:58.829144 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:58.829102 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" event={"ID":"25672a2d-d6e3-47aa-880d-0751856a6b47","Type":"ContainerStarted","Data":"85785cc5062ff080be1d70bfd445f2b153117ef3851f54e98f6ca34941689eb6"} Sep 10 12:29:58.830767 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:58.830694 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-250.ec2.internal" podStartSLOduration=3.830672933 podStartE2EDuration="3.830672933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:29:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:29:58.810178501 +0000 UTC m=+4.543773219" watchObservedRunningTime="2026-09-10 12:29:58.830672933 +0000 UTC m=+4.564267651" Sep 10 12:29:58.835188 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:58.835155 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kmxmc" event={"ID":"1019e2de-09cf-419b-a4ce-7e09921a4e04","Type":"ContainerStarted","Data":"234d349f42ac92ce7960686fc4c3e4df82527613ea4f4d31e0327c525e340268"} Sep 10 12:29:58.839750 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:58.839702 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" event={"ID":"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1","Type":"ContainerStarted","Data":"470e1157370f40e0ffd140ea9bd53d70da11e26293e0c38cf23c370be339a20e"} Sep 10 12:29:58.841556 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:58.841525 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-478c6" event={"ID":"334e2bed-0266-4d0f-964f-96dcb9843a25","Type":"ContainerStarted","Data":"dbc3b742b5acb2c5ff3447c23cdc211b0ea3675014a0bc31f5609feed49dd549"} Sep 10 12:29:59.873081 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:59.872818 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-250.ec2.internal" event={"ID":"b683db139541b51f843e32d0a18fc2d6","Type":"ContainerStarted","Data":"904caf69528dc37d22d0712d8664f64b624f7e825d43b14f5f5f421635899be1"} Sep 10 12:29:59.892376 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:29:59.892314 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-250.ec2.internal" podStartSLOduration=4.8922909919999995 podStartE2EDuration="4.892290992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:29:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:29:59.892104708 +0000 UTC m=+5.625699426" watchObservedRunningTime="2026-09-10 12:29:59.892290992 +0000 UTC m=+5.625885710" Sep 10 12:30:00.439107 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:00.439064 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs\") pod \"network-metrics-daemon-cmbbq\" (UID: \"90ce254f-e418-4e45-ab41-427cd3a1626e\") " pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:30:00.439289 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:00.439142 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqgph\" (UniqueName: \"kubernetes.io/projected/bdfce9e5-4daa-4b23-8f66-62549603ef7f-kube-api-access-jqgph\") pod \"network-check-target-p5wcp\" (UID: \"bdfce9e5-4daa-4b23-8f66-62549603ef7f\") " pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:30:00.439289 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:00.439173 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-original-pull-secret\") pod \"global-pull-secret-syncer-wclkd\" (UID: \"cb45e2aa-d5a7-47e9-828e-186e39ffd6b8\") " pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:30:00.439442 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:00.439423 2588 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:30:00.439505 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:00.439498 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs podName:90ce254f-e418-4e45-ab41-427cd3a1626e nodeName:}" failed. No retries permitted until 2026-09-10 12:30:04.439478004 +0000 UTC m=+10.173072702 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs") pod "network-metrics-daemon-cmbbq" (UID: "90ce254f-e418-4e45-ab41-427cd3a1626e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:30:00.439759 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:00.439727 2588 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:30:00.439833 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:00.439792 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-original-pull-secret podName:cb45e2aa-d5a7-47e9-828e-186e39ffd6b8 nodeName:}" failed. No retries permitted until 2026-09-10 12:30:04.439778956 +0000 UTC m=+10.173373664 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-original-pull-secret") pod "global-pull-secret-syncer-wclkd" (UID: "cb45e2aa-d5a7-47e9-828e-186e39ffd6b8") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:30:00.439907 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:00.439881 2588 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:30:00.439907 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:00.439900 2588 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:30:00.440002 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:00.439914 2588 projected.go:196] Error preparing data for projected volume kube-api-access-jqgph for pod openshift-network-diagnostics/network-check-target-p5wcp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:30:00.440002 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:00.439957 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdfce9e5-4daa-4b23-8f66-62549603ef7f-kube-api-access-jqgph podName:bdfce9e5-4daa-4b23-8f66-62549603ef7f nodeName:}" failed. No retries permitted until 2026-09-10 12:30:04.439944561 +0000 UTC m=+10.173539271 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jqgph" (UniqueName: "kubernetes.io/projected/bdfce9e5-4daa-4b23-8f66-62549603ef7f-kube-api-access-jqgph") pod "network-check-target-p5wcp" (UID: "bdfce9e5-4daa-4b23-8f66-62549603ef7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:30:00.770900 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:00.770853 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:30:00.770900 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:00.770876 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:30:00.771218 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:00.771000 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5wcp" podUID="bdfce9e5-4daa-4b23-8f66-62549603ef7f" Sep 10 12:30:00.771218 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:00.771081 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:30:00.771218 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:00.771197 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmbbq" podUID="90ce254f-e418-4e45-ab41-427cd3a1626e" Sep 10 12:30:00.771364 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:00.771273 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wclkd" podUID="cb45e2aa-d5a7-47e9-828e-186e39ffd6b8" Sep 10 12:30:02.773686 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:02.772872 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:30:02.773686 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:02.773027 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wclkd" podUID="cb45e2aa-d5a7-47e9-828e-186e39ffd6b8" Sep 10 12:30:02.773686 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:02.773375 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:30:02.773686 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:02.773490 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmbbq" podUID="90ce254f-e418-4e45-ab41-427cd3a1626e" Sep 10 12:30:02.773686 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:02.773559 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:30:02.773686 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:02.773649 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5wcp" podUID="bdfce9e5-4daa-4b23-8f66-62549603ef7f" Sep 10 12:30:04.476029 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:04.475973 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs\") pod \"network-metrics-daemon-cmbbq\" (UID: \"90ce254f-e418-4e45-ab41-427cd3a1626e\") " pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:30:04.476500 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:04.476150 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqgph\" (UniqueName: \"kubernetes.io/projected/bdfce9e5-4daa-4b23-8f66-62549603ef7f-kube-api-access-jqgph\") pod \"network-check-target-p5wcp\" (UID: \"bdfce9e5-4daa-4b23-8f66-62549603ef7f\") " pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:30:04.476500 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:04.476172 2588 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:30:04.476500 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:04.476188 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-original-pull-secret\") pod \"global-pull-secret-syncer-wclkd\" (UID: \"cb45e2aa-d5a7-47e9-828e-186e39ffd6b8\") " pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:30:04.476500 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:04.476252 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs podName:90ce254f-e418-4e45-ab41-427cd3a1626e nodeName:}" failed. No retries permitted until 2026-09-10 12:30:12.476229988 +0000 UTC m=+18.209824704 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs") pod "network-metrics-daemon-cmbbq" (UID: "90ce254f-e418-4e45-ab41-427cd3a1626e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:30:04.476500 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:04.476319 2588 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:30:04.476500 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:04.476370 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-original-pull-secret podName:cb45e2aa-d5a7-47e9-828e-186e39ffd6b8 nodeName:}" failed. No retries permitted until 2026-09-10 12:30:12.476355582 +0000 UTC m=+18.209950280 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-original-pull-secret") pod "global-pull-secret-syncer-wclkd" (UID: "cb45e2aa-d5a7-47e9-828e-186e39ffd6b8") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:30:04.476500 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:04.476445 2588 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:30:04.476500 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:04.476463 2588 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:30:04.476500 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:04.476475 2588 projected.go:196] Error preparing data for projected volume kube-api-access-jqgph for pod openshift-network-diagnostics/network-check-target-p5wcp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:30:04.476500 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:04.476509 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdfce9e5-4daa-4b23-8f66-62549603ef7f-kube-api-access-jqgph podName:bdfce9e5-4daa-4b23-8f66-62549603ef7f nodeName:}" failed. No retries permitted until 2026-09-10 12:30:12.476498052 +0000 UTC m=+18.210092750 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jqgph" (UniqueName: "kubernetes.io/projected/bdfce9e5-4daa-4b23-8f66-62549603ef7f-kube-api-access-jqgph") pod "network-check-target-p5wcp" (UID: "bdfce9e5-4daa-4b23-8f66-62549603ef7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:30:04.771651 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:04.771617 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:30:04.771861 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:04.771751 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmbbq" podUID="90ce254f-e418-4e45-ab41-427cd3a1626e" Sep 10 12:30:04.771861 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:04.771805 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:30:04.771995 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:04.771912 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5wcp" podUID="bdfce9e5-4daa-4b23-8f66-62549603ef7f" Sep 10 12:30:04.771995 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:04.771972 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:30:04.772095 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:04.772032 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wclkd" podUID="cb45e2aa-d5a7-47e9-828e-186e39ffd6b8" Sep 10 12:30:06.770781 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:06.770729 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:30:06.771235 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:06.770729 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:30:06.771235 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:06.770899 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wclkd" podUID="cb45e2aa-d5a7-47e9-828e-186e39ffd6b8" Sep 10 12:30:06.771235 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:06.770916 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:30:06.771235 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:06.771033 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5wcp" podUID="bdfce9e5-4daa-4b23-8f66-62549603ef7f" Sep 10 12:30:06.771235 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:06.771123 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmbbq" podUID="90ce254f-e418-4e45-ab41-427cd3a1626e" Sep 10 12:30:08.770399 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:08.770361 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:30:08.770884 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:08.770361 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:30:08.770884 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:08.770508 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:30:08.770884 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:08.770499 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wclkd" podUID="cb45e2aa-d5a7-47e9-828e-186e39ffd6b8" Sep 10 12:30:08.770884 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:08.770635 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5wcp" podUID="bdfce9e5-4daa-4b23-8f66-62549603ef7f" Sep 10 12:30:08.770884 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:08.770751 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmbbq" podUID="90ce254f-e418-4e45-ab41-427cd3a1626e" Sep 10 12:30:10.771104 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:10.770727 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:30:10.771104 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:10.770766 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:30:10.771618 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:10.771120 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wclkd" podUID="cb45e2aa-d5a7-47e9-828e-186e39ffd6b8" Sep 10 12:30:10.771618 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:10.771563 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5wcp" podUID="bdfce9e5-4daa-4b23-8f66-62549603ef7f" Sep 10 12:30:10.772043 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:10.771797 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:30:10.772043 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:10.771958 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmbbq" podUID="90ce254f-e418-4e45-ab41-427cd3a1626e" Sep 10 12:30:12.540873 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:12.540838 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqgph\" (UniqueName: \"kubernetes.io/projected/bdfce9e5-4daa-4b23-8f66-62549603ef7f-kube-api-access-jqgph\") pod \"network-check-target-p5wcp\" (UID: \"bdfce9e5-4daa-4b23-8f66-62549603ef7f\") " pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:30:12.540873 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:12.540875 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-original-pull-secret\") pod \"global-pull-secret-syncer-wclkd\" (UID: \"cb45e2aa-d5a7-47e9-828e-186e39ffd6b8\") " pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:30:12.541402 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:12.540909 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs\") pod \"network-metrics-daemon-cmbbq\" (UID: \"90ce254f-e418-4e45-ab41-427cd3a1626e\") " pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:30:12.541402 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:12.541010 2588 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:30:12.541402 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:12.541015 2588 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:30:12.541402 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:12.541045 2588 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:30:12.541402 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:12.541057 2588 projected.go:196] Error preparing data for projected volume kube-api-access-jqgph for pod openshift-network-diagnostics/network-check-target-p5wcp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:30:12.541402 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:12.541071 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs podName:90ce254f-e418-4e45-ab41-427cd3a1626e nodeName:}" failed. No retries permitted until 2026-09-10 12:30:28.541052968 +0000 UTC m=+34.274647670 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs") pod "network-metrics-daemon-cmbbq" (UID: "90ce254f-e418-4e45-ab41-427cd3a1626e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:30:12.541402 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:12.541105 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdfce9e5-4daa-4b23-8f66-62549603ef7f-kube-api-access-jqgph podName:bdfce9e5-4daa-4b23-8f66-62549603ef7f nodeName:}" failed. No retries permitted until 2026-09-10 12:30:28.541093005 +0000 UTC m=+34.274687716 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jqgph" (UniqueName: "kubernetes.io/projected/bdfce9e5-4daa-4b23-8f66-62549603ef7f-kube-api-access-jqgph") pod "network-check-target-p5wcp" (UID: "bdfce9e5-4daa-4b23-8f66-62549603ef7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:30:12.541402 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:12.541013 2588 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:30:12.541402 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:12.541174 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-original-pull-secret podName:cb45e2aa-d5a7-47e9-828e-186e39ffd6b8 nodeName:}" failed. No retries permitted until 2026-09-10 12:30:28.541156147 +0000 UTC m=+34.274750860 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-original-pull-secret") pod "global-pull-secret-syncer-wclkd" (UID: "cb45e2aa-d5a7-47e9-828e-186e39ffd6b8") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:30:12.770832 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:12.770800 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:30:12.770985 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:12.770800 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:30:12.770985 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:12.770942 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmbbq" podUID="90ce254f-e418-4e45-ab41-427cd3a1626e" Sep 10 12:30:12.771110 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:12.770800 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:30:12.771110 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:12.770983 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5wcp" podUID="bdfce9e5-4daa-4b23-8f66-62549603ef7f" Sep 10 12:30:12.771110 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:12.771098 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wclkd" podUID="cb45e2aa-d5a7-47e9-828e-186e39ffd6b8" Sep 10 12:30:14.771007 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:14.770971 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:30:14.771462 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:14.771069 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmbbq" podUID="90ce254f-e418-4e45-ab41-427cd3a1626e" Sep 10 12:30:14.771462 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:14.771088 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:30:14.771462 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:14.771088 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:30:14.771462 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:14.771196 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wclkd" podUID="cb45e2aa-d5a7-47e9-828e-186e39ffd6b8" Sep 10 12:30:14.771462 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:14.771324 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5wcp" podUID="bdfce9e5-4daa-4b23-8f66-62549603ef7f" Sep 10 12:30:16.770616 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:16.770307 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:30:16.771645 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:16.770310 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:30:16.771645 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:16.770692 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5wcp" podUID="bdfce9e5-4daa-4b23-8f66-62549603ef7f" Sep 10 12:30:16.771645 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:16.770785 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wclkd" podUID="cb45e2aa-d5a7-47e9-828e-186e39ffd6b8" Sep 10 12:30:16.771645 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:16.770315 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:30:16.771645 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:16.770872 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmbbq" podUID="90ce254f-e418-4e45-ab41-427cd3a1626e" Sep 10 12:30:16.905100 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:16.905066 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" event={"ID":"fcfb96de-97a4-4abc-a215-32d210dda11d","Type":"ContainerStarted","Data":"9402c1ad4762986594c8e7c90b21d2a2413c6b15f51182fc0180147f08a76a87"} Sep 10 12:30:16.906800 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:16.906764 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rwdsg" event={"ID":"2d9a758b-9a83-405b-9b01-7143273959ff","Type":"ContainerStarted","Data":"956b797be789ac60c3f3fd138838560431bd8b4db232d75dbe4e4f12a0423deb"} Sep 10 12:30:16.908703 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:16.908674 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tc789" event={"ID":"76bf8bce-b7c3-4617-87e4-0be0765a14d0","Type":"ContainerStarted","Data":"bc47a5da2521d9b684b26be07922009ede307a810cf3fff3d6ad1d03d0c2336d"} Sep 10 12:30:16.912509 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:16.912479 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" event={"ID":"25672a2d-d6e3-47aa-880d-0751856a6b47","Type":"ContainerStarted","Data":"60d9e1891ae439264c2f6e74c0c788091517baac7910f8f3e33c80bc6a1442ac"} Sep 10 12:30:16.912759 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:16.912558 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" event={"ID":"25672a2d-d6e3-47aa-880d-0751856a6b47","Type":"ContainerStarted","Data":"a9339324593894f391951145f55e865ef3e69d6872d84334d00446405aaba46e"} Sep 10 12:30:16.912759 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:16.912592 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" event={"ID":"25672a2d-d6e3-47aa-880d-0751856a6b47","Type":"ContainerStarted","Data":"8f4f1dbb8d3bb054f137997bdd7025e998756a782b55b89190104f4e735fd1aa"} Sep 10 12:30:16.912759 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:16.912629 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" event={"ID":"25672a2d-d6e3-47aa-880d-0751856a6b47","Type":"ContainerStarted","Data":"091e85b4bb1ea7ac6c5e7429e9e2a75a9017d363fab90a46c8217d85552392fe"} Sep 10 12:30:16.912759 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:16.912676 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" event={"ID":"25672a2d-d6e3-47aa-880d-0751856a6b47","Type":"ContainerStarted","Data":"9a7ef4046c2289ef0f4fef9903bcb80503086cc23a0fda65eef31e75098b836f"} Sep 10 12:30:16.912759 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:16.912718 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" event={"ID":"25672a2d-d6e3-47aa-880d-0751856a6b47","Type":"ContainerStarted","Data":"072c916dc42f5fbb6f58499ec22e971544c8f8081bc6d0c99dc4144839f48110"} Sep 10 12:30:16.914404 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:16.914379 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kmxmc" event={"ID":"1019e2de-09cf-419b-a4ce-7e09921a4e04","Type":"ContainerStarted","Data":"42490dabd275df08182403f59e73d8710373ad7f20a706fe63168dee014e689d"} Sep 10 12:30:16.917291 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:16.917261 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" event={"ID":"2ed00d5c-9e03-40b5-b261-deeb8aaa46d1","Type":"ContainerStarted","Data":"b36b80a197508545e85049d1fc3e34fd74aa0c48e3eefcbab666e810ff228c51"} Sep 10 12:30:16.918978 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:16.918952 2588 generic.go:356] "Generic (PLEG): container finished" podID="334e2bed-0266-4d0f-964f-96dcb9843a25" containerID="459c2a1566f91ff27d0c687b6e43040e905a62eb719ae62fed5c4f6b3474b63b" exitCode=0 Sep 10 12:30:16.919078 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:16.919037 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-478c6" event={"ID":"334e2bed-0266-4d0f-964f-96dcb9843a25","Type":"ContainerDied","Data":"459c2a1566f91ff27d0c687b6e43040e905a62eb719ae62fed5c4f6b3474b63b"} Sep 10 12:30:16.920605 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:16.920545 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-km6jk" event={"ID":"6b43b97f-6877-402b-87be-390b1a42c4f8","Type":"ContainerStarted","Data":"82a933a606a8b3406a839ccc3e52515deb7e7b4a73b23b0d4d531011d164d363"} Sep 10 12:30:16.942927 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:16.942870 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rwdsg" podStartSLOduration=4.140828732 podStartE2EDuration="21.942847753s" totalImagesPullingTime="17.802019021s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:29:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:30:16.923060338 +0000 UTC m=+22.656655054" watchObservedRunningTime="2026-09-10 12:30:16.942847753 +0000 UTC m=+22.676442472" Sep 10 12:30:16.961962 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:16.961893 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xhmjp" podStartSLOduration=5.167854402 podStartE2EDuration="22.961867231s" totalImagesPullingTime="17.794012829s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:29:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:30:16.942825159 +0000 UTC m=+22.676419878" watchObservedRunningTime="2026-09-10 12:30:16.961867231 +0000 UTC m=+22.695461953" Sep 10 12:30:16.962158 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:16.962081 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tc789" podStartSLOduration=4.097675175 podStartE2EDuration="21.962070238s" totalImagesPullingTime="17.864395063s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:29:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:30:16.961868103 +0000 UTC m=+22.695462847" watchObservedRunningTime="2026-09-10 12:30:16.962070238 +0000 UTC m=+22.695664959" Sep 10 12:30:16.982598 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:16.982536 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-km6jk" podStartSLOduration=7.870313737 podStartE2EDuration="21.982514063s" totalImagesPullingTime="14.112200326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:29:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:30:16.98172054 +0000 UTC m=+22.715315257" watchObservedRunningTime="2026-09-10 12:30:16.982514063 +0000 UTC m=+22.716108781" Sep 10 12:30:17.115955 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:17.115893 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kmxmc" podStartSLOduration=5.323929395 podStartE2EDuration="23.115872127s" totalImagesPullingTime="17.791942732s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:29:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:30:17.115592119 +0000 UTC m=+22.849186836" watchObservedRunningTime="2026-09-10 12:30:17.115872127 +0000 UTC m=+22.849466847" Sep 10 12:30:17.388287 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:17.388258 2588 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 10 12:30:17.764536 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:17.764433 2588 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T12:30:17.388277947Z","UUID":"8ffd5917-4fd1-42df-baa5-6f331eea4955","Handler":null,"Name":"","Endpoint":""} Sep 10 12:30:17.765953 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:17.765929 2588 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 10 12:30:17.766080 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:17.765962 2588 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 10 12:30:17.925707 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:17.925666 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" event={"ID":"fcfb96de-97a4-4abc-a215-32d210dda11d","Type":"ContainerStarted","Data":"ac7ce0693c25545234fddec769590ba640995c4f2b1cb4ba3c3372a5e7147a80"} Sep 10 12:30:17.927571 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:17.927505 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bjnv5" event={"ID":"df440146-c599-4722-b96d-986b68d93e40","Type":"ContainerStarted","Data":"95d92619efa454f4757464aad662cb68ee7371313b351e03116cdcc74425b9bc"} Sep 10 12:30:17.948516 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:17.948451 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bjnv5" podStartSLOduration=6.148539263 podStartE2EDuration="23.948435385s" totalImagesPullingTime="17.799896122s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:29:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:30:17.948110774 +0000 UTC m=+23.681705493" watchObservedRunningTime="2026-09-10 12:30:17.948435385 +0000 UTC m=+23.682030102" Sep 10 12:30:18.771308 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:18.771271 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:30:18.771530 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:18.771412 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmbbq" podUID="90ce254f-e418-4e45-ab41-427cd3a1626e" Sep 10 12:30:18.771530 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:18.771271 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:30:18.771530 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:18.771271 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:30:18.771530 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:18.771517 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wclkd" podUID="cb45e2aa-d5a7-47e9-828e-186e39ffd6b8" Sep 10 12:30:18.771750 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:18.771572 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5wcp" podUID="bdfce9e5-4daa-4b23-8f66-62549603ef7f" Sep 10 12:30:18.932922 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:18.932882 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" event={"ID":"fcfb96de-97a4-4abc-a215-32d210dda11d","Type":"ContainerStarted","Data":"dbd37d730c3b3d29612a6c7f56e876024987325ed9ea7a6772a6f8d331f5e3e1"} Sep 10 12:30:18.937059 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:18.936973 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" event={"ID":"25672a2d-d6e3-47aa-880d-0751856a6b47","Type":"ContainerStarted","Data":"2245c997242e6b762b2f7b874b014e617e951422842028337e4efb778b866f53"} Sep 10 12:30:19.323191 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:19.323149 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-km6jk" Sep 10 12:30:19.323834 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:19.323809 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-km6jk" Sep 10 12:30:19.344290 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:19.344234 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5dwzp" podStartSLOduration=4.179008618 podStartE2EDuration="24.344214122s" totalImagesPullingTime="20.165205504s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:29:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:30:18.954719548 +0000 UTC m=+24.688314274" watchObservedRunningTime="2026-09-10 12:30:19.344214122 +0000 UTC m=+25.077808840" Sep 10 12:30:19.939420 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:19.939309 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-km6jk" Sep 10 12:30:19.939919 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:19.939853 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-km6jk" Sep 10 12:30:20.771348 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:20.771141 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:30:20.771348 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:20.771141 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:30:20.771457 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:20.771154 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:30:20.771457 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:20.771413 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wclkd" podUID="cb45e2aa-d5a7-47e9-828e-186e39ffd6b8" Sep 10 12:30:20.771780 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:20.771514 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmbbq" podUID="90ce254f-e418-4e45-ab41-427cd3a1626e" Sep 10 12:30:20.771780 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:20.771605 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5wcp" podUID="bdfce9e5-4daa-4b23-8f66-62549603ef7f" Sep 10 12:30:20.947017 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:20.946921 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" event={"ID":"25672a2d-d6e3-47aa-880d-0751856a6b47","Type":"ContainerStarted","Data":"6bfa10fa65ac75052a6029d09cde5e8ebe7da7dba8e428c67fe0abd60f942788"} Sep 10 12:30:20.947889 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:20.947248 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:30:20.947889 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:20.947335 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:30:20.965606 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:20.965576 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:30:21.008181 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:21.008113 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" podStartSLOduration=8.126978395 podStartE2EDuration="26.008089786s" totalImagesPullingTime="17.881111391s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:29:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:30:21.007456123 +0000 UTC m=+26.741050845" watchObservedRunningTime="2026-09-10 12:30:21.008089786 +0000 UTC m=+26.741684504" Sep 10 12:30:21.956048 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:21.956008 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:30:21.978229 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:21.978198 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:30:22.633525 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:22.633491 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p5wcp"] Sep 10 12:30:22.633727 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:22.633656 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:30:22.633813 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:22.633783 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5wcp" podUID="bdfce9e5-4daa-4b23-8f66-62549603ef7f" Sep 10 12:30:22.636828 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:22.636800 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wclkd"] Sep 10 12:30:22.636960 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:22.636911 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:30:22.637024 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:22.637003 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wclkd" podUID="cb45e2aa-d5a7-47e9-828e-186e39ffd6b8" Sep 10 12:30:22.640154 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:22.640126 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cmbbq"] Sep 10 12:30:22.640312 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:22.640273 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:30:22.640405 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:22.640384 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmbbq" podUID="90ce254f-e418-4e45-ab41-427cd3a1626e" Sep 10 12:30:22.960143 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:22.959968 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-478c6" event={"ID":"334e2bed-0266-4d0f-964f-96dcb9843a25","Type":"ContainerStarted","Data":"9a9554816cbbd376a75c38457046ac2b75f230d5075d41a02df5ac56acce8dd0"} Sep 10 12:30:23.771308 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:23.771266 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:30:23.771487 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:23.771377 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wclkd" podUID="cb45e2aa-d5a7-47e9-828e-186e39ffd6b8" Sep 10 12:30:23.964361 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:23.964323 2588 generic.go:356] "Generic (PLEG): container finished" podID="334e2bed-0266-4d0f-964f-96dcb9843a25" containerID="9a9554816cbbd376a75c38457046ac2b75f230d5075d41a02df5ac56acce8dd0" exitCode=0 Sep 10 12:30:23.964725 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:23.964405 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-478c6" event={"ID":"334e2bed-0266-4d0f-964f-96dcb9843a25","Type":"ContainerDied","Data":"9a9554816cbbd376a75c38457046ac2b75f230d5075d41a02df5ac56acce8dd0"} Sep 10 12:30:24.771252 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:24.771222 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:30:24.772699 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:24.772667 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:30:24.772860 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:24.772793 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5wcp" podUID="bdfce9e5-4daa-4b23-8f66-62549603ef7f" Sep 10 12:30:24.772954 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:24.772929 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmbbq" podUID="90ce254f-e418-4e45-ab41-427cd3a1626e" Sep 10 12:30:25.770965 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:25.770938 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:30:25.771362 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:25.771042 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wclkd" podUID="cb45e2aa-d5a7-47e9-828e-186e39ffd6b8" Sep 10 12:30:25.971583 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:25.971553 2588 generic.go:356] "Generic (PLEG): container finished" podID="334e2bed-0266-4d0f-964f-96dcb9843a25" containerID="8699016d8d6bd88997b110a41e4c66da423245c86e5068105f9834786d6c0bed" exitCode=0 Sep 10 12:30:25.971761 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:25.971590 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-478c6" event={"ID":"334e2bed-0266-4d0f-964f-96dcb9843a25","Type":"ContainerDied","Data":"8699016d8d6bd88997b110a41e4c66da423245c86e5068105f9834786d6c0bed"} Sep 10 12:30:26.771037 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:26.771002 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:30:26.771516 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:26.771006 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:30:26.771516 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:26.771139 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5wcp" podUID="bdfce9e5-4daa-4b23-8f66-62549603ef7f" Sep 10 12:30:26.771516 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:26.771238 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmbbq" podUID="90ce254f-e418-4e45-ab41-427cd3a1626e" Sep 10 12:30:27.770795 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:27.770585 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:30:27.770977 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:27.770873 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wclkd" podUID="cb45e2aa-d5a7-47e9-828e-186e39ffd6b8" Sep 10 12:30:27.979401 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:27.979361 2588 generic.go:356] "Generic (PLEG): container finished" podID="334e2bed-0266-4d0f-964f-96dcb9843a25" containerID="1b0cbf445f97a634d8f12292e15d7823152589973a3626620ef63efcb57ee3d1" exitCode=0 Sep 10 12:30:27.979812 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:27.979412 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-478c6" event={"ID":"334e2bed-0266-4d0f-964f-96dcb9843a25","Type":"ContainerDied","Data":"1b0cbf445f97a634d8f12292e15d7823152589973a3626620ef63efcb57ee3d1"} Sep 10 12:30:28.567541 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.567500 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs\") pod \"network-metrics-daemon-cmbbq\" (UID: \"90ce254f-e418-4e45-ab41-427cd3a1626e\") " pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:30:28.567793 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.567559 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqgph\" (UniqueName: \"kubernetes.io/projected/bdfce9e5-4daa-4b23-8f66-62549603ef7f-kube-api-access-jqgph\") pod \"network-check-target-p5wcp\" (UID: \"bdfce9e5-4daa-4b23-8f66-62549603ef7f\") " pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:30:28.567793 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:28.567663 2588 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:30:28.567793 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.567680 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-original-pull-secret\") pod \"global-pull-secret-syncer-wclkd\" (UID: \"cb45e2aa-d5a7-47e9-828e-186e39ffd6b8\") " pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:30:28.567793 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:28.567673 2588 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 12:30:28.567793 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:28.567755 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs podName:90ce254f-e418-4e45-ab41-427cd3a1626e nodeName:}" failed. No retries permitted until 2026-09-10 12:31:00.567717468 +0000 UTC m=+66.301312189 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs") pod "network-metrics-daemon-cmbbq" (UID: "90ce254f-e418-4e45-ab41-427cd3a1626e") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 12:30:28.567793 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:28.567763 2588 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 12:30:28.567793 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:28.567777 2588 projected.go:196] Error preparing data for projected volume kube-api-access-jqgph for pod openshift-network-diagnostics/network-check-target-p5wcp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:30:28.568055 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:28.567800 2588 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 12:30:28.568055 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:28.567828 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdfce9e5-4daa-4b23-8f66-62549603ef7f-kube-api-access-jqgph podName:bdfce9e5-4daa-4b23-8f66-62549603ef7f nodeName:}" failed. No retries permitted until 2026-09-10 12:31:00.567813391 +0000 UTC m=+66.301408087 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jqgph" (UniqueName: "kubernetes.io/projected/bdfce9e5-4daa-4b23-8f66-62549603ef7f-kube-api-access-jqgph") pod "network-check-target-p5wcp" (UID: "bdfce9e5-4daa-4b23-8f66-62549603ef7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 12:30:28.568055 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:28.567849 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-original-pull-secret podName:cb45e2aa-d5a7-47e9-828e-186e39ffd6b8 nodeName:}" failed. No retries permitted until 2026-09-10 12:31:00.567838155 +0000 UTC m=+66.301432856 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-original-pull-secret") pod "global-pull-secret-syncer-wclkd" (UID: "cb45e2aa-d5a7-47e9-828e-186e39ffd6b8") : object "kube-system"/"original-pull-secret" not registered Sep 10 12:30:28.608083 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.608058 2588 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-250.ec2.internal" event="NodeReady" Sep 10 12:30:28.608239 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.608188 2588 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 10 12:30:28.651402 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.651325 2588 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-55bf984797-4tkjd"] Sep 10 12:30:28.654281 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.654261 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:28.655648 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.655616 2588 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-948dc8f9d-px5mp"] Sep 10 12:30:28.658500 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.658476 2588 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6688db56d5-xdq6l"] Sep 10 12:30:28.658625 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.658591 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-948dc8f9d-px5mp" Sep 10 12:30:28.659286 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.659264 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 10 12:30:28.659651 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.659626 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-957dd\"" Sep 10 12:30:28.659651 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.659626 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 10 12:30:28.661811 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.661789 2588 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x"] Sep 10 12:30:28.661954 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.661919 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6688db56d5-xdq6l" Sep 10 12:30:28.664313 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.664292 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 10 12:30:28.664542 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.664522 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 10 12:30:28.664622 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.664527 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 10 12:30:28.664622 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.664584 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 10 12:30:28.664952 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.664936 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" Sep 10 12:30:28.665680 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.665628 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-sxkss\"" Sep 10 12:30:28.667535 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.666478 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 10 12:30:28.667535 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.666579 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 10 12:30:28.671539 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.671504 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 10 12:30:28.671861 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.671841 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Sep 10 12:30:28.671978 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.671960 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Sep 10 12:30:28.672044 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.671992 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Sep 10 12:30:28.672100 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.672072 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Sep 10 12:30:28.672671 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.672650 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55bf984797-4tkjd"] Sep 10 12:30:28.677279 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.677210 2588 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-b5xwv"] Sep 10 12:30:28.680286 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.680260 2588 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-f75jz"] Sep 10 12:30:28.680428 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.680408 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b5xwv" Sep 10 12:30:28.683235 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.683206 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-948dc8f9d-px5mp"] Sep 10 12:30:28.683372 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.683246 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6688db56d5-xdq6l"] Sep 10 12:30:28.683372 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.683308 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f75jz" Sep 10 12:30:28.683491 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.683478 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 10 12:30:28.683925 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.683904 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fv8rg\"" Sep 10 12:30:28.684306 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.684285 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 10 12:30:28.684412 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.684337 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 10 12:30:28.685718 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.685702 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-c86jl\"" Sep 10 12:30:28.685963 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.685946 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 10 12:30:28.686034 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.685978 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 10 12:30:28.698012 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.697984 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b5xwv"] Sep 10 12:30:28.706540 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.706508 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f75jz"] Sep 10 12:30:28.707170 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.707146 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x"] Sep 10 12:30:28.769369 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.769333 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls\") pod \"dns-default-f75jz\" (UID: \"f3aadb41-77bc-4679-9ea3-0b47e499c6e4\") " pod="openshift-dns/dns-default-f75jz" Sep 10 12:30:28.769369 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.769372 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2530f8c5-460f-4ba2-8af9-86a62eaf6361-image-registry-private-configuration\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:28.769594 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.769393 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2530f8c5-460f-4ba2-8af9-86a62eaf6361-ca-trust-extracted\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:28.769594 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.769433 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krkmg\" (UniqueName: \"kubernetes.io/projected/09a8b6b8-6084-4098-804d-260e69715516-kube-api-access-krkmg\") pod \"ingress-canary-b5xwv\" (UID: \"09a8b6b8-6084-4098-804d-260e69715516\") " pod="openshift-ingress-canary/ingress-canary-b5xwv" Sep 10 12:30:28.769594 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.769523 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bx4h\" (UniqueName: \"kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-kube-api-access-2bx4h\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:28.769594 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.769549 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4913242d-abae-4627-911b-0cc24d653a45-hub\") pod \"cluster-proxy-proxy-agent-77997bd6b8-nvd5x\" (UID: \"4913242d-abae-4627-911b-0cc24d653a45\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" Sep 10 12:30:28.769594 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.769579 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh97v\" (UniqueName: \"kubernetes.io/projected/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-kube-api-access-xh97v\") pod \"dns-default-f75jz\" (UID: \"f3aadb41-77bc-4679-9ea3-0b47e499c6e4\") " pod="openshift-dns/dns-default-f75jz" Sep 10 12:30:28.769795 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.769619 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-certificates\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:28.769795 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.769643 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2530f8c5-460f-4ba2-8af9-86a62eaf6361-installation-pull-secrets\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:28.769795 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.769671 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4913242d-abae-4627-911b-0cc24d653a45-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-77997bd6b8-nvd5x\" (UID: \"4913242d-abae-4627-911b-0cc24d653a45\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" Sep 10 12:30:28.769795 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.769693 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4913242d-abae-4627-911b-0cc24d653a45-additional-service-ca\") pod \"cluster-proxy-proxy-agent-77997bd6b8-nvd5x\" (UID: \"4913242d-abae-4627-911b-0cc24d653a45\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" Sep 10 12:30:28.769795 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.769717 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4913242d-abae-4627-911b-0cc24d653a45-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-77997bd6b8-nvd5x\" (UID: \"4913242d-abae-4627-911b-0cc24d653a45\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" Sep 10 12:30:28.769795 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.769760 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2530f8c5-460f-4ba2-8af9-86a62eaf6361-trusted-ca\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:28.769795 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.769775 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-config-volume\") pod \"dns-default-f75jz\" (UID: \"f3aadb41-77bc-4679-9ea3-0b47e499c6e4\") " pod="openshift-dns/dns-default-f75jz" Sep 10 12:30:28.769795 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.769795 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sphvn\" (UniqueName: \"kubernetes.io/projected/f1fa20aa-9454-42e8-a28e-b761d336ae57-kube-api-access-sphvn\") pod \"managed-serviceaccount-addon-agent-948dc8f9d-px5mp\" (UID: \"f1fa20aa-9454-42e8-a28e-b761d336ae57\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-948dc8f9d-px5mp" Sep 10 12:30:28.770064 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.769815 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-tmp-dir\") pod \"dns-default-f75jz\" (UID: \"f3aadb41-77bc-4679-9ea3-0b47e499c6e4\") " pod="openshift-dns/dns-default-f75jz" Sep 10 12:30:28.770064 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.769853 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/09a8b6b8-6084-4098-804d-260e69715516-cert\") pod \"ingress-canary-b5xwv\" (UID: \"09a8b6b8-6084-4098-804d-260e69715516\") " pod="openshift-ingress-canary/ingress-canary-b5xwv" Sep 10 12:30:28.770064 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.769879 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/edcf1cff-81f3-4977-8756-d37da17eba6e-klusterlet-config\") pod \"klusterlet-addon-workmgr-6688db56d5-xdq6l\" (UID: \"edcf1cff-81f3-4977-8756-d37da17eba6e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6688db56d5-xdq6l" Sep 10 12:30:28.770064 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.769895 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:28.770064 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.769924 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-bound-sa-token\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:28.770064 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.769941 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f1fa20aa-9454-42e8-a28e-b761d336ae57-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-948dc8f9d-px5mp\" (UID: \"f1fa20aa-9454-42e8-a28e-b761d336ae57\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-948dc8f9d-px5mp" Sep 10 12:30:28.770064 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.769957 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4913242d-abae-4627-911b-0cc24d653a45-ca\") pod \"cluster-proxy-proxy-agent-77997bd6b8-nvd5x\" (UID: \"4913242d-abae-4627-911b-0cc24d653a45\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" Sep 10 12:30:28.770064 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.769996 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d9m4\" (UniqueName: \"kubernetes.io/projected/4913242d-abae-4627-911b-0cc24d653a45-kube-api-access-7d9m4\") pod \"cluster-proxy-proxy-agent-77997bd6b8-nvd5x\" (UID: \"4913242d-abae-4627-911b-0cc24d653a45\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" Sep 10 12:30:28.770064 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.770013 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/edcf1cff-81f3-4977-8756-d37da17eba6e-tmp\") pod \"klusterlet-addon-workmgr-6688db56d5-xdq6l\" (UID: \"edcf1cff-81f3-4977-8756-d37da17eba6e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6688db56d5-xdq6l" Sep 10 12:30:28.770064 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.770047 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqj65\" (UniqueName: \"kubernetes.io/projected/edcf1cff-81f3-4977-8756-d37da17eba6e-kube-api-access-kqj65\") pod \"klusterlet-addon-workmgr-6688db56d5-xdq6l\" (UID: \"edcf1cff-81f3-4977-8756-d37da17eba6e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6688db56d5-xdq6l" Sep 10 12:30:28.770430 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.770413 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:30:28.770518 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.770417 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:30:28.772810 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.772768 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 12:30:28.772956 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.772842 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bqff6\"" Sep 10 12:30:28.772956 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.772909 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pkls5\"" Sep 10 12:30:28.773051 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.772958 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 12:30:28.773051 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.772958 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 12:30:28.870462 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.870413 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-certificates\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:28.870462 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.870454 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2530f8c5-460f-4ba2-8af9-86a62eaf6361-installation-pull-secrets\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:28.870685 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.870474 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4913242d-abae-4627-911b-0cc24d653a45-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-77997bd6b8-nvd5x\" (UID: \"4913242d-abae-4627-911b-0cc24d653a45\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" Sep 10 12:30:28.870685 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.870612 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4913242d-abae-4627-911b-0cc24d653a45-additional-service-ca\") pod \"cluster-proxy-proxy-agent-77997bd6b8-nvd5x\" (UID: \"4913242d-abae-4627-911b-0cc24d653a45\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" Sep 10 12:30:28.870685 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.870648 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4913242d-abae-4627-911b-0cc24d653a45-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-77997bd6b8-nvd5x\" (UID: \"4913242d-abae-4627-911b-0cc24d653a45\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" Sep 10 12:30:28.870849 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.870684 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2530f8c5-460f-4ba2-8af9-86a62eaf6361-trusted-ca\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:28.870849 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.870712 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-config-volume\") pod \"dns-default-f75jz\" (UID: \"f3aadb41-77bc-4679-9ea3-0b47e499c6e4\") " pod="openshift-dns/dns-default-f75jz" Sep 10 12:30:28.870967 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.870942 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sphvn\" (UniqueName: \"kubernetes.io/projected/f1fa20aa-9454-42e8-a28e-b761d336ae57-kube-api-access-sphvn\") pod \"managed-serviceaccount-addon-agent-948dc8f9d-px5mp\" (UID: \"f1fa20aa-9454-42e8-a28e-b761d336ae57\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-948dc8f9d-px5mp" Sep 10 12:30:28.871021 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.871002 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-tmp-dir\") pod \"dns-default-f75jz\" (UID: \"f3aadb41-77bc-4679-9ea3-0b47e499c6e4\") " pod="openshift-dns/dns-default-f75jz" Sep 10 12:30:28.871068 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.871040 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/09a8b6b8-6084-4098-804d-260e69715516-cert\") pod \"ingress-canary-b5xwv\" (UID: \"09a8b6b8-6084-4098-804d-260e69715516\") " pod="openshift-ingress-canary/ingress-canary-b5xwv" Sep 10 12:30:28.871249 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.871225 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-certificates\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:28.871327 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.871289 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/edcf1cff-81f3-4977-8756-d37da17eba6e-klusterlet-config\") pod \"klusterlet-addon-workmgr-6688db56d5-xdq6l\" (UID: \"edcf1cff-81f3-4977-8756-d37da17eba6e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6688db56d5-xdq6l" Sep 10 12:30:28.871327 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.871309 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-config-volume\") pod \"dns-default-f75jz\" (UID: \"f3aadb41-77bc-4679-9ea3-0b47e499c6e4\") " pod="openshift-dns/dns-default-f75jz" Sep 10 12:30:28.871411 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.871339 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:28.871411 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.871354 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4913242d-abae-4627-911b-0cc24d653a45-additional-service-ca\") pod \"cluster-proxy-proxy-agent-77997bd6b8-nvd5x\" (UID: \"4913242d-abae-4627-911b-0cc24d653a45\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" Sep 10 12:30:28.871411 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.871365 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-bound-sa-token\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:28.871524 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.871490 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f1fa20aa-9454-42e8-a28e-b761d336ae57-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-948dc8f9d-px5mp\" (UID: \"f1fa20aa-9454-42e8-a28e-b761d336ae57\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-948dc8f9d-px5mp" Sep 10 12:30:28.871566 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.871520 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4913242d-abae-4627-911b-0cc24d653a45-ca\") pod \"cluster-proxy-proxy-agent-77997bd6b8-nvd5x\" (UID: \"4913242d-abae-4627-911b-0cc24d653a45\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" Sep 10 12:30:28.871631 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.871614 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d9m4\" (UniqueName: \"kubernetes.io/projected/4913242d-abae-4627-911b-0cc24d653a45-kube-api-access-7d9m4\") pod \"cluster-proxy-proxy-agent-77997bd6b8-nvd5x\" (UID: \"4913242d-abae-4627-911b-0cc24d653a45\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" Sep 10 12:30:28.871679 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.871659 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/edcf1cff-81f3-4977-8756-d37da17eba6e-tmp\") pod \"klusterlet-addon-workmgr-6688db56d5-xdq6l\" (UID: \"edcf1cff-81f3-4977-8756-d37da17eba6e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6688db56d5-xdq6l" Sep 10 12:30:28.872016 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.871929 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqj65\" (UniqueName: \"kubernetes.io/projected/edcf1cff-81f3-4977-8756-d37da17eba6e-kube-api-access-kqj65\") pod \"klusterlet-addon-workmgr-6688db56d5-xdq6l\" (UID: \"edcf1cff-81f3-4977-8756-d37da17eba6e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6688db56d5-xdq6l" Sep 10 12:30:28.872016 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.871966 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls\") pod \"dns-default-f75jz\" (UID: \"f3aadb41-77bc-4679-9ea3-0b47e499c6e4\") " pod="openshift-dns/dns-default-f75jz" Sep 10 12:30:28.872016 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.871996 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2530f8c5-460f-4ba2-8af9-86a62eaf6361-image-registry-private-configuration\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:28.872186 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.872019 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2530f8c5-460f-4ba2-8af9-86a62eaf6361-ca-trust-extracted\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:28.872186 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.872043 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krkmg\" (UniqueName: \"kubernetes.io/projected/09a8b6b8-6084-4098-804d-260e69715516-kube-api-access-krkmg\") pod \"ingress-canary-b5xwv\" (UID: \"09a8b6b8-6084-4098-804d-260e69715516\") " pod="openshift-ingress-canary/ingress-canary-b5xwv" Sep 10 12:30:28.872186 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.872089 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bx4h\" (UniqueName: \"kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-kube-api-access-2bx4h\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:28.872186 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.872111 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4913242d-abae-4627-911b-0cc24d653a45-hub\") pod \"cluster-proxy-proxy-agent-77997bd6b8-nvd5x\" (UID: \"4913242d-abae-4627-911b-0cc24d653a45\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" Sep 10 12:30:28.872186 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.872133 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh97v\" (UniqueName: \"kubernetes.io/projected/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-kube-api-access-xh97v\") pod \"dns-default-f75jz\" (UID: \"f3aadb41-77bc-4679-9ea3-0b47e499c6e4\") " pod="openshift-dns/dns-default-f75jz" Sep 10 12:30:28.872394 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.871491 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-tmp-dir\") pod \"dns-default-f75jz\" (UID: \"f3aadb41-77bc-4679-9ea3-0b47e499c6e4\") " pod="openshift-dns/dns-default-f75jz" Sep 10 12:30:28.872513 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.872476 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2530f8c5-460f-4ba2-8af9-86a62eaf6361-trusted-ca\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:28.872513 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:28.872496 2588 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 12:30:28.872644 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:28.872582 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls podName:f3aadb41-77bc-4679-9ea3-0b47e499c6e4 nodeName:}" failed. No retries permitted until 2026-09-10 12:30:29.372562798 +0000 UTC m=+35.106157578 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls") pod "dns-default-f75jz" (UID: "f3aadb41-77bc-4679-9ea3-0b47e499c6e4") : secret "dns-default-metrics-tls" not found Sep 10 12:30:28.873231 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.873201 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2530f8c5-460f-4ba2-8af9-86a62eaf6361-ca-trust-extracted\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:28.873699 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:28.873679 2588 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 12:30:28.873797 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:28.873703 2588 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55bf984797-4tkjd: secret "image-registry-tls" not found Sep 10 12:30:28.873797 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:28.873793 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls podName:2530f8c5-460f-4ba2-8af9-86a62eaf6361 nodeName:}" failed. No retries permitted until 2026-09-10 12:30:29.373774869 +0000 UTC m=+35.107369583 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls") pod "image-registry-55bf984797-4tkjd" (UID: "2530f8c5-460f-4ba2-8af9-86a62eaf6361") : secret "image-registry-tls" not found Sep 10 12:30:28.873994 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.873936 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/edcf1cff-81f3-4977-8756-d37da17eba6e-tmp\") pod \"klusterlet-addon-workmgr-6688db56d5-xdq6l\" (UID: \"edcf1cff-81f3-4977-8756-d37da17eba6e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6688db56d5-xdq6l" Sep 10 12:30:28.875891 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.875866 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2530f8c5-460f-4ba2-8af9-86a62eaf6361-installation-pull-secrets\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:28.875998 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.875890 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/09a8b6b8-6084-4098-804d-260e69715516-cert\") pod \"ingress-canary-b5xwv\" (UID: \"09a8b6b8-6084-4098-804d-260e69715516\") " pod="openshift-ingress-canary/ingress-canary-b5xwv" Sep 10 12:30:28.876356 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.876304 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4913242d-abae-4627-911b-0cc24d653a45-ca\") pod \"cluster-proxy-proxy-agent-77997bd6b8-nvd5x\" (UID: \"4913242d-abae-4627-911b-0cc24d653a45\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" Sep 10 12:30:28.876356 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.876337 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4913242d-abae-4627-911b-0cc24d653a45-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-77997bd6b8-nvd5x\" (UID: \"4913242d-abae-4627-911b-0cc24d653a45\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" Sep 10 12:30:28.876504 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.876477 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4913242d-abae-4627-911b-0cc24d653a45-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-77997bd6b8-nvd5x\" (UID: \"4913242d-abae-4627-911b-0cc24d653a45\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" Sep 10 12:30:28.876723 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.876707 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/edcf1cff-81f3-4977-8756-d37da17eba6e-klusterlet-config\") pod \"klusterlet-addon-workmgr-6688db56d5-xdq6l\" (UID: \"edcf1cff-81f3-4977-8756-d37da17eba6e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6688db56d5-xdq6l" Sep 10 12:30:28.876780 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.876717 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2530f8c5-460f-4ba2-8af9-86a62eaf6361-image-registry-private-configuration\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:28.876780 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.876767 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f1fa20aa-9454-42e8-a28e-b761d336ae57-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-948dc8f9d-px5mp\" (UID: \"f1fa20aa-9454-42e8-a28e-b761d336ae57\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-948dc8f9d-px5mp" Sep 10 12:30:28.877583 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.877561 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4913242d-abae-4627-911b-0cc24d653a45-hub\") pod \"cluster-proxy-proxy-agent-77997bd6b8-nvd5x\" (UID: \"4913242d-abae-4627-911b-0cc24d653a45\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" Sep 10 12:30:28.883804 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.883777 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh97v\" (UniqueName: \"kubernetes.io/projected/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-kube-api-access-xh97v\") pod \"dns-default-f75jz\" (UID: \"f3aadb41-77bc-4679-9ea3-0b47e499c6e4\") " pod="openshift-dns/dns-default-f75jz" Sep 10 12:30:28.884159 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.884132 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sphvn\" (UniqueName: \"kubernetes.io/projected/f1fa20aa-9454-42e8-a28e-b761d336ae57-kube-api-access-sphvn\") pod \"managed-serviceaccount-addon-agent-948dc8f9d-px5mp\" (UID: \"f1fa20aa-9454-42e8-a28e-b761d336ae57\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-948dc8f9d-px5mp" Sep 10 12:30:28.884260 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.884161 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d9m4\" (UniqueName: \"kubernetes.io/projected/4913242d-abae-4627-911b-0cc24d653a45-kube-api-access-7d9m4\") pod \"cluster-proxy-proxy-agent-77997bd6b8-nvd5x\" (UID: \"4913242d-abae-4627-911b-0cc24d653a45\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" Sep 10 12:30:28.885000 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.884949 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bx4h\" (UniqueName: \"kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-kube-api-access-2bx4h\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:28.885115 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.885034 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-bound-sa-token\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:28.886276 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.886251 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krkmg\" (UniqueName: \"kubernetes.io/projected/09a8b6b8-6084-4098-804d-260e69715516-kube-api-access-krkmg\") pod \"ingress-canary-b5xwv\" (UID: \"09a8b6b8-6084-4098-804d-260e69715516\") " pod="openshift-ingress-canary/ingress-canary-b5xwv" Sep 10 12:30:28.886701 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.886685 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqj65\" (UniqueName: \"kubernetes.io/projected/edcf1cff-81f3-4977-8756-d37da17eba6e-kube-api-access-kqj65\") pod \"klusterlet-addon-workmgr-6688db56d5-xdq6l\" (UID: \"edcf1cff-81f3-4977-8756-d37da17eba6e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6688db56d5-xdq6l" Sep 10 12:30:28.985051 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.984952 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-948dc8f9d-px5mp" Sep 10 12:30:28.993123 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.993090 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6688db56d5-xdq6l" Sep 10 12:30:28.999510 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:28.999478 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" Sep 10 12:30:29.015678 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:29.015613 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b5xwv" Sep 10 12:30:29.298527 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:29.298409 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6688db56d5-xdq6l"] Sep 10 12:30:29.302574 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:30:29.302538 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedcf1cff_81f3_4977_8756_d37da17eba6e.slice/crio-6a5796f1d01ff281b4bb2673c3f4031cdf3f8940ba18cbe870b48525bbb29eb2 WatchSource:0}: Error finding container 6a5796f1d01ff281b4bb2673c3f4031cdf3f8940ba18cbe870b48525bbb29eb2: Status 404 returned error can't find the container with id 6a5796f1d01ff281b4bb2673c3f4031cdf3f8940ba18cbe870b48525bbb29eb2 Sep 10 12:30:29.304129 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:29.304102 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-948dc8f9d-px5mp"] Sep 10 12:30:29.307115 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:29.307074 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x"] Sep 10 12:30:29.309927 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:30:29.309887 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1fa20aa_9454_42e8_a28e_b761d336ae57.slice/crio-4b23ba34f1bf424dd89cd3a67e4a34df0a0ce89ab8a06e39c54b21ef2a3891dd WatchSource:0}: Error finding container 4b23ba34f1bf424dd89cd3a67e4a34df0a0ce89ab8a06e39c54b21ef2a3891dd: Status 404 returned error can't find the container with id 4b23ba34f1bf424dd89cd3a67e4a34df0a0ce89ab8a06e39c54b21ef2a3891dd Sep 10 12:30:29.311588 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:30:29.311563 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4913242d_abae_4627_911b_0cc24d653a45.slice/crio-6f766cac01e96b205800d7cc46dd2804b5b446f285affc4fc04156dddaa3bb89 WatchSource:0}: Error finding container 6f766cac01e96b205800d7cc46dd2804b5b446f285affc4fc04156dddaa3bb89: Status 404 returned error can't find the container with id 6f766cac01e96b205800d7cc46dd2804b5b446f285affc4fc04156dddaa3bb89 Sep 10 12:30:29.325596 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:29.325551 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b5xwv"] Sep 10 12:30:29.329340 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:30:29.329304 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09a8b6b8_6084_4098_804d_260e69715516.slice/crio-83f5a023993578933f5550434715596457c3dd997482002140c5ade647307a91 WatchSource:0}: Error finding container 83f5a023993578933f5550434715596457c3dd997482002140c5ade647307a91: Status 404 returned error can't find the container with id 83f5a023993578933f5550434715596457c3dd997482002140c5ade647307a91 Sep 10 12:30:29.375565 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:29.375526 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls\") pod \"dns-default-f75jz\" (UID: \"f3aadb41-77bc-4679-9ea3-0b47e499c6e4\") " pod="openshift-dns/dns-default-f75jz" Sep 10 12:30:29.375814 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:29.375643 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:29.375814 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:29.375677 2588 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 12:30:29.375814 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:29.375770 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls podName:f3aadb41-77bc-4679-9ea3-0b47e499c6e4 nodeName:}" failed. No retries permitted until 2026-09-10 12:30:30.375731203 +0000 UTC m=+36.109325898 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls") pod "dns-default-f75jz" (UID: "f3aadb41-77bc-4679-9ea3-0b47e499c6e4") : secret "dns-default-metrics-tls" not found Sep 10 12:30:29.375814 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:29.375777 2588 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 12:30:29.375814 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:29.375797 2588 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55bf984797-4tkjd: secret "image-registry-tls" not found Sep 10 12:30:29.376025 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:29.375850 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls podName:2530f8c5-460f-4ba2-8af9-86a62eaf6361 nodeName:}" failed. No retries permitted until 2026-09-10 12:30:30.375835151 +0000 UTC m=+36.109429861 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls") pod "image-registry-55bf984797-4tkjd" (UID: "2530f8c5-460f-4ba2-8af9-86a62eaf6361") : secret "image-registry-tls" not found Sep 10 12:30:29.770901 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:29.770862 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:30:29.773675 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:29.773649 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 12:30:29.986668 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:29.986624 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b5xwv" event={"ID":"09a8b6b8-6084-4098-804d-260e69715516","Type":"ContainerStarted","Data":"83f5a023993578933f5550434715596457c3dd997482002140c5ade647307a91"} Sep 10 12:30:29.988003 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:29.987970 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" event={"ID":"4913242d-abae-4627-911b-0cc24d653a45","Type":"ContainerStarted","Data":"6f766cac01e96b205800d7cc46dd2804b5b446f285affc4fc04156dddaa3bb89"} Sep 10 12:30:29.989282 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:29.989253 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6688db56d5-xdq6l" event={"ID":"edcf1cff-81f3-4977-8756-d37da17eba6e","Type":"ContainerStarted","Data":"6a5796f1d01ff281b4bb2673c3f4031cdf3f8940ba18cbe870b48525bbb29eb2"} Sep 10 12:30:29.990604 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:29.990581 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-948dc8f9d-px5mp" event={"ID":"f1fa20aa-9454-42e8-a28e-b761d336ae57","Type":"ContainerStarted","Data":"4b23ba34f1bf424dd89cd3a67e4a34df0a0ce89ab8a06e39c54b21ef2a3891dd"} Sep 10 12:30:30.385819 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:30.385779 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls\") pod \"dns-default-f75jz\" (UID: \"f3aadb41-77bc-4679-9ea3-0b47e499c6e4\") " pod="openshift-dns/dns-default-f75jz" Sep 10 12:30:30.386044 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:30.385925 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:30.386115 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:30.386080 2588 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 12:30:30.386115 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:30.386095 2588 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55bf984797-4tkjd: secret "image-registry-tls" not found Sep 10 12:30:30.386214 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:30.386156 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls podName:2530f8c5-460f-4ba2-8af9-86a62eaf6361 nodeName:}" failed. No retries permitted until 2026-09-10 12:30:32.386137324 +0000 UTC m=+38.119732044 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls") pod "image-registry-55bf984797-4tkjd" (UID: "2530f8c5-460f-4ba2-8af9-86a62eaf6361") : secret "image-registry-tls" not found Sep 10 12:30:30.386597 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:30.386572 2588 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 12:30:30.386682 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:30.386637 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls podName:f3aadb41-77bc-4679-9ea3-0b47e499c6e4 nodeName:}" failed. No retries permitted until 2026-09-10 12:30:32.386618942 +0000 UTC m=+38.120213641 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls") pod "dns-default-f75jz" (UID: "f3aadb41-77bc-4679-9ea3-0b47e499c6e4") : secret "dns-default-metrics-tls" not found Sep 10 12:30:32.405434 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:32.404882 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:32.405434 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:32.405168 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls\") pod \"dns-default-f75jz\" (UID: \"f3aadb41-77bc-4679-9ea3-0b47e499c6e4\") " pod="openshift-dns/dns-default-f75jz" Sep 10 12:30:32.405434 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:32.405051 2588 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 12:30:32.405434 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:32.405214 2588 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55bf984797-4tkjd: secret "image-registry-tls" not found Sep 10 12:30:32.405434 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:32.405289 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls podName:2530f8c5-460f-4ba2-8af9-86a62eaf6361 nodeName:}" failed. No retries permitted until 2026-09-10 12:30:36.405268746 +0000 UTC m=+42.138863446 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls") pod "image-registry-55bf984797-4tkjd" (UID: "2530f8c5-460f-4ba2-8af9-86a62eaf6361") : secret "image-registry-tls" not found Sep 10 12:30:32.405434 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:32.405366 2588 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 12:30:32.405434 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:32.405405 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls podName:f3aadb41-77bc-4679-9ea3-0b47e499c6e4 nodeName:}" failed. No retries permitted until 2026-09-10 12:30:36.405392938 +0000 UTC m=+42.138987649 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls") pod "dns-default-f75jz" (UID: "f3aadb41-77bc-4679-9ea3-0b47e499c6e4") : secret "dns-default-metrics-tls" not found Sep 10 12:30:33.664482 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:33.664442 2588 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-km6jk_6b43b97f-6877-402b-87be-390b1a42c4f8/konnectivity-agent/0.log" limit=1024 Sep 10 12:30:33.774061 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:33.774019 2588 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-km6jk_6b43b97f-6877-402b-87be-390b1a42c4f8/konnectivity-agent/0.log" limit=1024 Sep 10 12:30:34.690922 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:34.690881 2588 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-km6jk_6b43b97f-6877-402b-87be-390b1a42c4f8/konnectivity-agent/0.log" limit=1024 Sep 10 12:30:34.792990 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:34.792952 2588 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-km6jk_6b43b97f-6877-402b-87be-390b1a42c4f8/konnectivity-agent/0.log" limit=1024 Sep 10 12:30:34.941832 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:34.941750 2588 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-km6jk_6b43b97f-6877-402b-87be-390b1a42c4f8/konnectivity-agent/0.log" limit=1024 Sep 10 12:30:35.064297 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:35.064265 2588 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-km6jk_6b43b97f-6877-402b-87be-390b1a42c4f8/konnectivity-agent/0.log" limit=1024 Sep 10 12:30:35.163696 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:35.163653 2588 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-km6jk_6b43b97f-6877-402b-87be-390b1a42c4f8/konnectivity-agent/0.log" limit=1024 Sep 10 12:30:35.462751 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:35.462707 2588 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-km6jk_6b43b97f-6877-402b-87be-390b1a42c4f8/konnectivity-agent/0.log" limit=1024 Sep 10 12:30:35.534418 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:35.534375 2588 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-km6jk_6b43b97f-6877-402b-87be-390b1a42c4f8/konnectivity-agent/0.log" limit=1024 Sep 10 12:30:35.866791 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:35.866755 2588 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-km6jk_6b43b97f-6877-402b-87be-390b1a42c4f8/konnectivity-agent/0.log" limit=1024 Sep 10 12:30:35.971500 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:35.971464 2588 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-km6jk_6b43b97f-6877-402b-87be-390b1a42c4f8/konnectivity-agent/0.log" limit=1024 Sep 10 12:30:36.439987 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:36.439936 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:36.440202 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:36.440018 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls\") pod \"dns-default-f75jz\" (UID: \"f3aadb41-77bc-4679-9ea3-0b47e499c6e4\") " pod="openshift-dns/dns-default-f75jz" Sep 10 12:30:36.440202 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:36.440107 2588 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 12:30:36.440202 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:36.440129 2588 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55bf984797-4tkjd: secret "image-registry-tls" not found Sep 10 12:30:36.440202 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:36.440192 2588 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 12:30:36.440379 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:36.440197 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls podName:2530f8c5-460f-4ba2-8af9-86a62eaf6361 nodeName:}" failed. No retries permitted until 2026-09-10 12:30:44.440176882 +0000 UTC m=+50.173771577 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls") pod "image-registry-55bf984797-4tkjd" (UID: "2530f8c5-460f-4ba2-8af9-86a62eaf6361") : secret "image-registry-tls" not found Sep 10 12:30:36.440379 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:36.440271 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls podName:f3aadb41-77bc-4679-9ea3-0b47e499c6e4 nodeName:}" failed. No retries permitted until 2026-09-10 12:30:44.440252416 +0000 UTC m=+50.173847111 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls") pod "dns-default-f75jz" (UID: "f3aadb41-77bc-4679-9ea3-0b47e499c6e4") : secret "dns-default-metrics-tls" not found Sep 10 12:30:36.656436 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:36.656388 2588 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-km6jk_6b43b97f-6877-402b-87be-390b1a42c4f8/konnectivity-agent/0.log" limit=1024 Sep 10 12:30:36.732459 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:36.732375 2588 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-km6jk_6b43b97f-6877-402b-87be-390b1a42c4f8/konnectivity-agent/0.log" limit=1024 Sep 10 12:30:37.063779 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:37.063724 2588 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-km6jk_6b43b97f-6877-402b-87be-390b1a42c4f8/konnectivity-agent/0.log" limit=1024 Sep 10 12:30:37.165653 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:37.165618 2588 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-km6jk_6b43b97f-6877-402b-87be-390b1a42c4f8/konnectivity-agent/0.log" limit=1024 Sep 10 12:30:41.037005 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:41.036962 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b5xwv" event={"ID":"09a8b6b8-6084-4098-804d-260e69715516","Type":"ContainerStarted","Data":"616f3e4592526a261b775236f70274685d8add3f64df8ca0904eea896ae890e4"} Sep 10 12:30:41.039803 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:41.039769 2588 generic.go:356] "Generic (PLEG): container finished" podID="334e2bed-0266-4d0f-964f-96dcb9843a25" containerID="349bd1d2a3fe0294012546fdd798d343b0f76832890d8f209aa7307edfcc61b5" exitCode=0 Sep 10 12:30:41.039963 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:41.039849 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-478c6" event={"ID":"334e2bed-0266-4d0f-964f-96dcb9843a25","Type":"ContainerDied","Data":"349bd1d2a3fe0294012546fdd798d343b0f76832890d8f209aa7307edfcc61b5"} Sep 10 12:30:41.042217 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:41.042178 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" event={"ID":"4913242d-abae-4627-911b-0cc24d653a45","Type":"ContainerStarted","Data":"2686f466f087745d8862047543be5ce7b3ac521ce10cf1bf4699b150bce097e6"} Sep 10 12:30:41.042365 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:41.042224 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" event={"ID":"4913242d-abae-4627-911b-0cc24d653a45","Type":"ContainerStarted","Data":"2d4d1ca638cc6031126ea6da47cdb0597286a24f000e0b754e64a69f391789aa"} Sep 10 12:30:41.042365 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:41.042243 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" event={"ID":"4913242d-abae-4627-911b-0cc24d653a45","Type":"ContainerStarted","Data":"e8708746033162e60b4a61735e3ea58befd0e56779c91c928da91e22bb3c39e5"} Sep 10 12:30:41.043653 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:41.043619 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6688db56d5-xdq6l" event={"ID":"edcf1cff-81f3-4977-8756-d37da17eba6e","Type":"ContainerStarted","Data":"812a2e91b42e0bcb0669aa0fc78d8b1b5ed6f9c176f04c50e5fa24ae2226a05d"} Sep 10 12:30:41.043797 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:41.043768 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6688db56d5-xdq6l" Sep 10 12:30:41.045279 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:41.045244 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-948dc8f9d-px5mp" event={"ID":"f1fa20aa-9454-42e8-a28e-b761d336ae57","Type":"ContainerStarted","Data":"37307ae2cd239b389e329dea05e4e52480e02475c41ba3e4eacd8dc57cc13fc9"} Sep 10 12:30:41.045928 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:41.045897 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6688db56d5-xdq6l" Sep 10 12:30:41.054431 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:41.054383 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-b5xwv" podStartSLOduration=2.288657114 podStartE2EDuration="13.054366701s" totalImagesPullingTime="10.765709587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:30:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:30:41.054031096 +0000 UTC m=+46.787625805" watchObservedRunningTime="2026-09-10 12:30:41.054366701 +0000 UTC m=+46.787961414" Sep 10 12:30:41.101549 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:41.101486 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6688db56d5-xdq6l" podStartSLOduration=32.759299705 podStartE2EDuration="44.101465579s" totalImagesPullingTime="11.342165874s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:29:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:30:41.100628085 +0000 UTC m=+46.834222804" watchObservedRunningTime="2026-09-10 12:30:41.101465579 +0000 UTC m=+46.835060293" Sep 10 12:30:41.135378 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:41.135313 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" podStartSLOduration=33.352245977 podStartE2EDuration="44.135288175s" totalImagesPullingTime="10.783042198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:29:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:30:41.133872296 +0000 UTC m=+46.867467016" watchObservedRunningTime="2026-09-10 12:30:41.135288175 +0000 UTC m=+46.868882894" Sep 10 12:30:41.159581 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:41.159523 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-948dc8f9d-px5mp" podStartSLOduration=33.375091505 podStartE2EDuration="44.159504872s" totalImagesPullingTime="10.784413367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:29:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:30:41.158639613 +0000 UTC m=+46.892234329" watchObservedRunningTime="2026-09-10 12:30:41.159504872 +0000 UTC m=+46.893099590" Sep 10 12:30:42.051089 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:42.051050 2588 generic.go:356] "Generic (PLEG): container finished" podID="334e2bed-0266-4d0f-964f-96dcb9843a25" containerID="f37d6dbf47ae19c5bcc6e3424ee080ff7ebd57298dab6c89cbcb41b04752f3f2" exitCode=0 Sep 10 12:30:42.051589 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:42.051165 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-478c6" event={"ID":"334e2bed-0266-4d0f-964f-96dcb9843a25","Type":"ContainerDied","Data":"f37d6dbf47ae19c5bcc6e3424ee080ff7ebd57298dab6c89cbcb41b04752f3f2"} Sep 10 12:30:43.058445 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:43.058404 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-478c6" event={"ID":"334e2bed-0266-4d0f-964f-96dcb9843a25","Type":"ContainerStarted","Data":"7512a2388e92723e2ed482b76e53de861d4027731726cadfa065d87a1c81019f"} Sep 10 12:30:43.084956 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:43.084891 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-478c6" podStartSLOduration=10.694788551 podStartE2EDuration="49.084870501s" totalImagesPullingTime="38.39008195s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:29:54 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:30:43.08439399 +0000 UTC m=+48.817988708" watchObservedRunningTime="2026-09-10 12:30:43.084870501 +0000 UTC m=+48.818465219" Sep 10 12:30:44.505353 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:44.505308 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:30:44.505353 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:44.505365 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls\") pod \"dns-default-f75jz\" (UID: \"f3aadb41-77bc-4679-9ea3-0b47e499c6e4\") " pod="openshift-dns/dns-default-f75jz" Sep 10 12:30:44.505815 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:44.505494 2588 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 12:30:44.505815 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:44.505517 2588 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55bf984797-4tkjd: secret "image-registry-tls" not found Sep 10 12:30:44.505815 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:44.505566 2588 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 12:30:44.505815 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:44.505588 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls podName:2530f8c5-460f-4ba2-8af9-86a62eaf6361 nodeName:}" failed. No retries permitted until 2026-09-10 12:31:00.50556804 +0000 UTC m=+66.239162765 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls") pod "image-registry-55bf984797-4tkjd" (UID: "2530f8c5-460f-4ba2-8af9-86a62eaf6361") : secret "image-registry-tls" not found Sep 10 12:30:44.505815 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:30:44.505615 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls podName:f3aadb41-77bc-4679-9ea3-0b47e499c6e4 nodeName:}" failed. No retries permitted until 2026-09-10 12:31:00.505602916 +0000 UTC m=+66.239197635 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls") pod "dns-default-f75jz" (UID: "f3aadb41-77bc-4679-9ea3-0b47e499c6e4") : secret "dns-default-metrics-tls" not found Sep 10 12:30:53.981237 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:30:53.981206 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b47ts" Sep 10 12:31:00.528807 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:00.528729 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls\") pod \"dns-default-f75jz\" (UID: \"f3aadb41-77bc-4679-9ea3-0b47e499c6e4\") " pod="openshift-dns/dns-default-f75jz" Sep 10 12:31:00.529370 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:00.528852 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:31:00.529370 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:00.528938 2588 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 12:31:00.529370 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:00.528938 2588 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 12:31:00.529370 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:00.529031 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls podName:f3aadb41-77bc-4679-9ea3-0b47e499c6e4 nodeName:}" failed. No retries permitted until 2026-09-10 12:31:32.529008026 +0000 UTC m=+98.262602739 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls") pod "dns-default-f75jz" (UID: "f3aadb41-77bc-4679-9ea3-0b47e499c6e4") : secret "dns-default-metrics-tls" not found Sep 10 12:31:00.529370 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:00.528949 2588 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55bf984797-4tkjd: secret "image-registry-tls" not found Sep 10 12:31:00.529370 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:00.529077 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls podName:2530f8c5-460f-4ba2-8af9-86a62eaf6361 nodeName:}" failed. No retries permitted until 2026-09-10 12:31:32.52906591 +0000 UTC m=+98.262660604 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls") pod "image-registry-55bf984797-4tkjd" (UID: "2530f8c5-460f-4ba2-8af9-86a62eaf6361") : secret "image-registry-tls" not found Sep 10 12:31:00.629777 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:00.629714 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs\") pod \"network-metrics-daemon-cmbbq\" (UID: \"90ce254f-e418-4e45-ab41-427cd3a1626e\") " pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:31:00.629967 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:00.629809 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqgph\" (UniqueName: \"kubernetes.io/projected/bdfce9e5-4daa-4b23-8f66-62549603ef7f-kube-api-access-jqgph\") pod \"network-check-target-p5wcp\" (UID: \"bdfce9e5-4daa-4b23-8f66-62549603ef7f\") " pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:31:00.629967 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:00.629832 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-original-pull-secret\") pod \"global-pull-secret-syncer-wclkd\" (UID: \"cb45e2aa-d5a7-47e9-828e-186e39ffd6b8\") " pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:31:00.633834 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:00.633811 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 12:31:00.633911 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:00.633888 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 12:31:00.633999 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:00.633983 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 12:31:00.640864 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:00.640834 2588 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 10 12:31:00.640956 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:00.640936 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs podName:90ce254f-e418-4e45-ab41-427cd3a1626e nodeName:}" failed. No retries permitted until 2026-09-10 12:32:04.640912683 +0000 UTC m=+130.374507393 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs") pod "network-metrics-daemon-cmbbq" (UID: "90ce254f-e418-4e45-ab41-427cd3a1626e") : secret "metrics-daemon-secret" not found Sep 10 12:31:00.643241 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:00.643220 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 12:31:00.644011 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:00.643989 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb45e2aa-d5a7-47e9-828e-186e39ffd6b8-original-pull-secret\") pod \"global-pull-secret-syncer-wclkd\" (UID: \"cb45e2aa-d5a7-47e9-828e-186e39ffd6b8\") " pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:31:00.653336 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:00.653305 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqgph\" (UniqueName: \"kubernetes.io/projected/bdfce9e5-4daa-4b23-8f66-62549603ef7f-kube-api-access-jqgph\") pod \"network-check-target-p5wcp\" (UID: \"bdfce9e5-4daa-4b23-8f66-62549603ef7f\") " pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:31:00.683404 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:00.683366 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wclkd" Sep 10 12:31:00.888857 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:00.888760 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pkls5\"" Sep 10 12:31:00.896955 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:00.896925 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:31:00.958102 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:00.958068 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wclkd"] Sep 10 12:31:00.962317 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:31:00.962287 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb45e2aa_d5a7_47e9_828e_186e39ffd6b8.slice/crio-4ae5ebd25fb65f0ab29a4eb56addf203e453f893f27358d5aa93a47e79f2c304 WatchSource:0}: Error finding container 4ae5ebd25fb65f0ab29a4eb56addf203e453f893f27358d5aa93a47e79f2c304: Status 404 returned error can't find the container with id 4ae5ebd25fb65f0ab29a4eb56addf203e453f893f27358d5aa93a47e79f2c304 Sep 10 12:31:01.115150 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:01.115096 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wclkd" event={"ID":"cb45e2aa-d5a7-47e9-828e-186e39ffd6b8","Type":"ContainerStarted","Data":"4ae5ebd25fb65f0ab29a4eb56addf203e453f893f27358d5aa93a47e79f2c304"} Sep 10 12:31:01.168264 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:01.168177 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p5wcp"] Sep 10 12:31:01.171149 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:31:01.171120 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdfce9e5_4daa_4b23_8f66_62549603ef7f.slice/crio-828e2e676eebae1b16a8be839fc1a0890530500a1e0f3609d7e43e8adc4163eb WatchSource:0}: Error finding container 828e2e676eebae1b16a8be839fc1a0890530500a1e0f3609d7e43e8adc4163eb: Status 404 returned error can't find the container with id 828e2e676eebae1b16a8be839fc1a0890530500a1e0f3609d7e43e8adc4163eb Sep 10 12:31:02.119676 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:02.119634 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p5wcp" event={"ID":"bdfce9e5-4daa-4b23-8f66-62549603ef7f","Type":"ContainerStarted","Data":"828e2e676eebae1b16a8be839fc1a0890530500a1e0f3609d7e43e8adc4163eb"} Sep 10 12:31:07.143022 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:07.142975 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p5wcp" event={"ID":"bdfce9e5-4daa-4b23-8f66-62549603ef7f","Type":"ContainerStarted","Data":"37f2a7b44ec99e72f404cd1f400bbcba780e4036cbcfcbfd16c4a2a1a9aacb82"} Sep 10 12:31:07.143494 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:07.143094 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:31:07.144751 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:07.144705 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wclkd" event={"ID":"cb45e2aa-d5a7-47e9-828e-186e39ffd6b8","Type":"ContainerStarted","Data":"13d79b6f0f1ffe8ce6df6f5108fbfb8e3b2a6ec30aedf8b7e284527ba3180fb0"} Sep 10 12:31:07.160302 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:07.160232 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-p5wcp" podStartSLOduration=66.381220128 podStartE2EDuration="1m12.160207445s" totalImagesPullingTime="5.778987317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:29:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:31:07.159209018 +0000 UTC m=+72.892803737" watchObservedRunningTime="2026-09-10 12:31:07.160207445 +0000 UTC m=+72.893802162" Sep 10 12:31:32.584937 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:32.584890 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls\") pod \"dns-default-f75jz\" (UID: \"f3aadb41-77bc-4679-9ea3-0b47e499c6e4\") " pod="openshift-dns/dns-default-f75jz" Sep 10 12:31:32.585420 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:32.584988 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls\") pod \"image-registry-55bf984797-4tkjd\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:31:32.585420 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:32.585017 2588 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 12:31:32.585420 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:32.585103 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls podName:f3aadb41-77bc-4679-9ea3-0b47e499c6e4 nodeName:}" failed. No retries permitted until 2026-09-10 12:32:36.585085522 +0000 UTC m=+162.318680221 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls") pod "dns-default-f75jz" (UID: "f3aadb41-77bc-4679-9ea3-0b47e499c6e4") : secret "dns-default-metrics-tls" not found Sep 10 12:31:32.585420 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:32.585115 2588 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 12:31:32.585420 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:32.585130 2588 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55bf984797-4tkjd: secret "image-registry-tls" not found Sep 10 12:31:32.585420 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:32.585195 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls podName:2530f8c5-460f-4ba2-8af9-86a62eaf6361 nodeName:}" failed. No retries permitted until 2026-09-10 12:32:36.585178153 +0000 UTC m=+162.318772852 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls") pod "image-registry-55bf984797-4tkjd" (UID: "2530f8c5-460f-4ba2-8af9-86a62eaf6361") : secret "image-registry-tls" not found Sep 10 12:31:34.517650 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:34.517589 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wclkd" podStartSLOduration=93.518611216 podStartE2EDuration="1m39.517569647s" totalImagesPullingTime="5.998958431s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:29:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:31:07.175967926 +0000 UTC m=+72.909562655" watchObservedRunningTime="2026-09-10 12:31:34.517569647 +0000 UTC m=+100.251164482" Sep 10 12:31:34.518094 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:34.517988 2588 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-bvfvx"] Sep 10 12:31:34.521935 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:34.521914 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-bvfvx" Sep 10 12:31:34.525971 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:34.525945 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-cxsfq\"" Sep 10 12:31:34.557122 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:34.557081 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-bvfvx"] Sep 10 12:31:34.599156 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:34.599116 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhzzp\" (UniqueName: \"kubernetes.io/projected/a0e08fb9-ba6f-4d67-8335-aeef23ffe1d6-kube-api-access-qhzzp\") pod \"network-check-source-685456655f-bvfvx\" (UID: \"a0e08fb9-ba6f-4d67-8335-aeef23ffe1d6\") " pod="openshift-network-diagnostics/network-check-source-685456655f-bvfvx" Sep 10 12:31:34.699567 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:34.699525 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhzzp\" (UniqueName: \"kubernetes.io/projected/a0e08fb9-ba6f-4d67-8335-aeef23ffe1d6-kube-api-access-qhzzp\") pod \"network-check-source-685456655f-bvfvx\" (UID: \"a0e08fb9-ba6f-4d67-8335-aeef23ffe1d6\") " pod="openshift-network-diagnostics/network-check-source-685456655f-bvfvx" Sep 10 12:31:34.715905 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:34.715866 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhzzp\" (UniqueName: \"kubernetes.io/projected/a0e08fb9-ba6f-4d67-8335-aeef23ffe1d6-kube-api-access-qhzzp\") pod \"network-check-source-685456655f-bvfvx\" (UID: \"a0e08fb9-ba6f-4d67-8335-aeef23ffe1d6\") " pod="openshift-network-diagnostics/network-check-source-685456655f-bvfvx" Sep 10 12:31:34.830816 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:34.830700 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-bvfvx" Sep 10 12:31:35.109577 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:35.109485 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-bvfvx"] Sep 10 12:31:35.114706 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:31:35.114661 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0e08fb9_ba6f_4d67_8335_aeef23ffe1d6.slice/crio-8e3f14d3b002b07a8c5ccbfe7253b07c035a69b870c6fe2a8504598548db3530 WatchSource:0}: Error finding container 8e3f14d3b002b07a8c5ccbfe7253b07c035a69b870c6fe2a8504598548db3530: Status 404 returned error can't find the container with id 8e3f14d3b002b07a8c5ccbfe7253b07c035a69b870c6fe2a8504598548db3530 Sep 10 12:31:35.231787 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:35.231749 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-bvfvx" event={"ID":"a0e08fb9-ba6f-4d67-8335-aeef23ffe1d6","Type":"ContainerStarted","Data":"61b00966ab07452844f7c717b199c20b9a0abc4a2958300489b21e4577232759"} Sep 10 12:31:35.231959 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:35.231799 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-bvfvx" event={"ID":"a0e08fb9-ba6f-4d67-8335-aeef23ffe1d6","Type":"ContainerStarted","Data":"8e3f14d3b002b07a8c5ccbfe7253b07c035a69b870c6fe2a8504598548db3530"} Sep 10 12:31:35.250272 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:35.250211 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-bvfvx" podStartSLOduration=1.250190969 podStartE2EDuration="1.250190969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:31:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:31:35.249207272 +0000 UTC m=+100.982802013" watchObservedRunningTime="2026-09-10 12:31:35.250190969 +0000 UTC m=+100.983785686" Sep 10 12:31:38.150802 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:38.150772 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-p5wcp" Sep 10 12:31:39.615317 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:39.615279 2588 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-wqq6h"] Sep 10 12:31:39.647984 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:39.647948 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wqq6h"] Sep 10 12:31:39.648150 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:39.648075 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:39.650402 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:39.650375 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 10 12:31:39.650402 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:39.650399 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 10 12:31:39.650760 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:39.650723 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 10 12:31:39.650913 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:39.650900 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-r87vl\"" Sep 10 12:31:39.651003 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:39.650987 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 10 12:31:39.737864 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:39.737819 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecc55eac-e95e-4d11-9436-f465ac9c84b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:39.737864 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:39.737861 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecc55eac-e95e-4d11-9436-f465ac9c84b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:39.738105 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:39.737889 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ecc55eac-e95e-4d11-9436-f465ac9c84b3-data-volume\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:39.738105 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:39.737905 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpvbp\" (UniqueName: \"kubernetes.io/projected/ecc55eac-e95e-4d11-9436-f465ac9c84b3-kube-api-access-jpvbp\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:39.738105 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:39.738031 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ecc55eac-e95e-4d11-9436-f465ac9c84b3-crio-socket\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:39.838368 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:39.838324 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecc55eac-e95e-4d11-9436-f465ac9c84b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:39.838368 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:39.838369 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecc55eac-e95e-4d11-9436-f465ac9c84b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:39.838672 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:39.838396 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ecc55eac-e95e-4d11-9436-f465ac9c84b3-data-volume\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:39.838672 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:39.838398 2588 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 12:31:39.838672 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:39.838412 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpvbp\" (UniqueName: \"kubernetes.io/projected/ecc55eac-e95e-4d11-9436-f465ac9c84b3-kube-api-access-jpvbp\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:39.838672 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:39.838468 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ecc55eac-e95e-4d11-9436-f465ac9c84b3-kube-rbac-proxy-cm podName:ecc55eac-e95e-4d11-9436-f465ac9c84b3 nodeName:}" failed. No retries permitted until 2026-09-10 12:31:40.338449962 +0000 UTC m=+106.072044686 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ecc55eac-e95e-4d11-9436-f465ac9c84b3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wqq6h" (UID: "ecc55eac-e95e-4d11-9436-f465ac9c84b3") : configmap "kube-rbac-proxy" not found Sep 10 12:31:39.838672 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:39.838514 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ecc55eac-e95e-4d11-9436-f465ac9c84b3-crio-socket\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:39.838672 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:39.838516 2588 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 12:31:39.838672 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:39.838594 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ecc55eac-e95e-4d11-9436-f465ac9c84b3-crio-socket\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:39.838672 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:39.838613 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecc55eac-e95e-4d11-9436-f465ac9c84b3-insights-runtime-extractor-tls podName:ecc55eac-e95e-4d11-9436-f465ac9c84b3 nodeName:}" failed. No retries permitted until 2026-09-10 12:31:40.338604578 +0000 UTC m=+106.072199272 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ecc55eac-e95e-4d11-9436-f465ac9c84b3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wqq6h" (UID: "ecc55eac-e95e-4d11-9436-f465ac9c84b3") : secret "insights-runtime-extractor-tls" not found Sep 10 12:31:39.839019 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:39.838811 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ecc55eac-e95e-4d11-9436-f465ac9c84b3-data-volume\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:39.849010 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:39.848973 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpvbp\" (UniqueName: \"kubernetes.io/projected/ecc55eac-e95e-4d11-9436-f465ac9c84b3-kube-api-access-jpvbp\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:40.342420 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:40.342371 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecc55eac-e95e-4d11-9436-f465ac9c84b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:40.342420 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:40.342416 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecc55eac-e95e-4d11-9436-f465ac9c84b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:40.342654 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:40.342502 2588 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 12:31:40.342654 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:40.342523 2588 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 12:31:40.342654 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:40.342575 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecc55eac-e95e-4d11-9436-f465ac9c84b3-insights-runtime-extractor-tls podName:ecc55eac-e95e-4d11-9436-f465ac9c84b3 nodeName:}" failed. No retries permitted until 2026-09-10 12:31:41.342561382 +0000 UTC m=+107.076156078 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ecc55eac-e95e-4d11-9436-f465ac9c84b3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wqq6h" (UID: "ecc55eac-e95e-4d11-9436-f465ac9c84b3") : secret "insights-runtime-extractor-tls" not found Sep 10 12:31:40.342654 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:40.342643 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ecc55eac-e95e-4d11-9436-f465ac9c84b3-kube-rbac-proxy-cm podName:ecc55eac-e95e-4d11-9436-f465ac9c84b3 nodeName:}" failed. No retries permitted until 2026-09-10 12:31:41.342626073 +0000 UTC m=+107.076220768 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ecc55eac-e95e-4d11-9436-f465ac9c84b3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wqq6h" (UID: "ecc55eac-e95e-4d11-9436-f465ac9c84b3") : configmap "kube-rbac-proxy" not found Sep 10 12:31:41.350551 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:41.350507 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecc55eac-e95e-4d11-9436-f465ac9c84b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:41.350551 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:41.350559 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecc55eac-e95e-4d11-9436-f465ac9c84b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:41.351058 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:41.350646 2588 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 12:31:41.351058 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:41.350764 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ecc55eac-e95e-4d11-9436-f465ac9c84b3-kube-rbac-proxy-cm podName:ecc55eac-e95e-4d11-9436-f465ac9c84b3 nodeName:}" failed. No retries permitted until 2026-09-10 12:31:43.350721679 +0000 UTC m=+109.084316377 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ecc55eac-e95e-4d11-9436-f465ac9c84b3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wqq6h" (UID: "ecc55eac-e95e-4d11-9436-f465ac9c84b3") : configmap "kube-rbac-proxy" not found Sep 10 12:31:41.351058 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:41.350771 2588 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 12:31:41.351058 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:41.350840 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecc55eac-e95e-4d11-9436-f465ac9c84b3-insights-runtime-extractor-tls podName:ecc55eac-e95e-4d11-9436-f465ac9c84b3 nodeName:}" failed. No retries permitted until 2026-09-10 12:31:43.350827344 +0000 UTC m=+109.084422038 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ecc55eac-e95e-4d11-9436-f465ac9c84b3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wqq6h" (UID: "ecc55eac-e95e-4d11-9436-f465ac9c84b3") : secret "insights-runtime-extractor-tls" not found Sep 10 12:31:42.568781 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:42.568732 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rwdsg_2d9a758b-9a83-405b-9b01-7143273959ff/dns-node-resolver/0.log" Sep 10 12:31:43.169621 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:43.169586 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kmxmc_1019e2de-09cf-419b-a4ce-7e09921a4e04/node-ca/0.log" Sep 10 12:31:43.368365 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:43.368324 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecc55eac-e95e-4d11-9436-f465ac9c84b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:43.368567 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:43.368377 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecc55eac-e95e-4d11-9436-f465ac9c84b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:43.368567 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:43.368433 2588 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 12:31:43.368567 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:43.368503 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ecc55eac-e95e-4d11-9436-f465ac9c84b3-kube-rbac-proxy-cm podName:ecc55eac-e95e-4d11-9436-f465ac9c84b3 nodeName:}" failed. No retries permitted until 2026-09-10 12:31:47.368486479 +0000 UTC m=+113.102081188 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ecc55eac-e95e-4d11-9436-f465ac9c84b3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wqq6h" (UID: "ecc55eac-e95e-4d11-9436-f465ac9c84b3") : configmap "kube-rbac-proxy" not found Sep 10 12:31:43.368567 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:43.368546 2588 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 12:31:43.368776 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:43.368612 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecc55eac-e95e-4d11-9436-f465ac9c84b3-insights-runtime-extractor-tls podName:ecc55eac-e95e-4d11-9436-f465ac9c84b3 nodeName:}" failed. No retries permitted until 2026-09-10 12:31:47.368594835 +0000 UTC m=+113.102189550 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ecc55eac-e95e-4d11-9436-f465ac9c84b3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wqq6h" (UID: "ecc55eac-e95e-4d11-9436-f465ac9c84b3") : secret "insights-runtime-extractor-tls" not found Sep 10 12:31:43.968842 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:43.968814 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b5xwv_09a8b6b8-6084-4098-804d-260e69715516/serve-healthcheck-canary/0.log" Sep 10 12:31:47.396715 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:47.396665 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecc55eac-e95e-4d11-9436-f465ac9c84b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:47.396715 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:47.396710 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecc55eac-e95e-4d11-9436-f465ac9c84b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:47.397144 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:47.396809 2588 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 12:31:47.397144 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:47.396858 2588 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 12:31:47.397144 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:47.396878 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ecc55eac-e95e-4d11-9436-f465ac9c84b3-kube-rbac-proxy-cm podName:ecc55eac-e95e-4d11-9436-f465ac9c84b3 nodeName:}" failed. No retries permitted until 2026-09-10 12:31:55.396861464 +0000 UTC m=+121.130456163 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ecc55eac-e95e-4d11-9436-f465ac9c84b3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wqq6h" (UID: "ecc55eac-e95e-4d11-9436-f465ac9c84b3") : configmap "kube-rbac-proxy" not found Sep 10 12:31:47.397144 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:47.396907 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecc55eac-e95e-4d11-9436-f465ac9c84b3-insights-runtime-extractor-tls podName:ecc55eac-e95e-4d11-9436-f465ac9c84b3 nodeName:}" failed. No retries permitted until 2026-09-10 12:31:55.396895313 +0000 UTC m=+121.130490007 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ecc55eac-e95e-4d11-9436-f465ac9c84b3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wqq6h" (UID: "ecc55eac-e95e-4d11-9436-f465ac9c84b3") : secret "insights-runtime-extractor-tls" not found Sep 10 12:31:55.462533 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:55.462484 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecc55eac-e95e-4d11-9436-f465ac9c84b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:55.462533 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:55.462531 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecc55eac-e95e-4d11-9436-f465ac9c84b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:55.463006 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:55.462640 2588 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 12:31:55.463006 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:31:55.462693 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecc55eac-e95e-4d11-9436-f465ac9c84b3-insights-runtime-extractor-tls podName:ecc55eac-e95e-4d11-9436-f465ac9c84b3 nodeName:}" failed. No retries permitted until 2026-09-10 12:32:11.462678692 +0000 UTC m=+137.196273386 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ecc55eac-e95e-4d11-9436-f465ac9c84b3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wqq6h" (UID: "ecc55eac-e95e-4d11-9436-f465ac9c84b3") : secret "insights-runtime-extractor-tls" not found Sep 10 12:31:55.463084 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:55.463061 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecc55eac-e95e-4d11-9436-f465ac9c84b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:31:59.000792 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:31:59.000714 2588 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" podUID="4913242d-abae-4627-911b-0cc24d653a45" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 10 12:32:04.736938 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:04.736872 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs\") pod \"network-metrics-daemon-cmbbq\" (UID: \"90ce254f-e418-4e45-ab41-427cd3a1626e\") " pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:32:04.739306 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:04.739284 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90ce254f-e418-4e45-ab41-427cd3a1626e-metrics-certs\") pod \"network-metrics-daemon-cmbbq\" (UID: \"90ce254f-e418-4e45-ab41-427cd3a1626e\") " pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:32:04.782699 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:04.782666 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bqff6\"" Sep 10 12:32:04.790438 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:04.790410 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmbbq" Sep 10 12:32:05.064930 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:05.064893 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cmbbq"] Sep 10 12:32:05.068969 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:32:05.068938 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90ce254f_e418_4e45_ab41_427cd3a1626e.slice/crio-8ef9d96abd7b3139bc0955491a3c4793632ef4374207f13f32e29b2929de8440 WatchSource:0}: Error finding container 8ef9d96abd7b3139bc0955491a3c4793632ef4374207f13f32e29b2929de8440: Status 404 returned error can't find the container with id 8ef9d96abd7b3139bc0955491a3c4793632ef4374207f13f32e29b2929de8440 Sep 10 12:32:05.326801 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:05.326691 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cmbbq" event={"ID":"90ce254f-e418-4e45-ab41-427cd3a1626e","Type":"ContainerStarted","Data":"8ef9d96abd7b3139bc0955491a3c4793632ef4374207f13f32e29b2929de8440"} Sep 10 12:32:07.339547 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:07.339508 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cmbbq" event={"ID":"90ce254f-e418-4e45-ab41-427cd3a1626e","Type":"ContainerStarted","Data":"190769cba099001805d17c2c3b72b14182d5cd9dff39080c262d3168d3fc47a1"} Sep 10 12:32:07.339953 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:07.339590 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cmbbq" event={"ID":"90ce254f-e418-4e45-ab41-427cd3a1626e","Type":"ContainerStarted","Data":"a4073b47c1602d8cafc70e7ef066ecf2342bdde50057524f7b5b7b8e3b48d8a7"} Sep 10 12:32:07.360833 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:07.360779 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cmbbq" podStartSLOduration=131.04059944 podStartE2EDuration="2m12.360764315s" totalImagesPullingTime="1.320164875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:29:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:32:07.360519948 +0000 UTC m=+133.094114666" watchObservedRunningTime="2026-09-10 12:32:07.360764315 +0000 UTC m=+133.094359033" Sep 10 12:32:09.000947 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:09.000908 2588 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" podUID="4913242d-abae-4627-911b-0cc24d653a45" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 10 12:32:11.491347 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:11.491307 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecc55eac-e95e-4d11-9436-f465ac9c84b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:32:11.493706 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:11.493681 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecc55eac-e95e-4d11-9436-f465ac9c84b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wqq6h\" (UID: \"ecc55eac-e95e-4d11-9436-f465ac9c84b3\") " pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:32:11.760404 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:11.760371 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-r87vl\"" Sep 10 12:32:11.767732 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:11.767701 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wqq6h" Sep 10 12:32:12.042510 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:12.042406 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wqq6h"] Sep 10 12:32:12.045373 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:32:12.045348 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecc55eac_e95e_4d11_9436_f465ac9c84b3.slice/crio-99d4c27f5d6adc47cd44019b089bdce74f718ff54d834be084564e32e4f82b0e WatchSource:0}: Error finding container 99d4c27f5d6adc47cd44019b089bdce74f718ff54d834be084564e32e4f82b0e: Status 404 returned error can't find the container with id 99d4c27f5d6adc47cd44019b089bdce74f718ff54d834be084564e32e4f82b0e Sep 10 12:32:12.357286 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:12.357199 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wqq6h" event={"ID":"ecc55eac-e95e-4d11-9436-f465ac9c84b3","Type":"ContainerStarted","Data":"bdec54a6313fba8172d3bbc0995c2a93018de283d723d6c400af5a5e1342de2a"} Sep 10 12:32:12.357286 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:12.357255 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wqq6h" event={"ID":"ecc55eac-e95e-4d11-9436-f465ac9c84b3","Type":"ContainerStarted","Data":"99d4c27f5d6adc47cd44019b089bdce74f718ff54d834be084564e32e4f82b0e"} Sep 10 12:32:13.363126 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:13.363085 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wqq6h" event={"ID":"ecc55eac-e95e-4d11-9436-f465ac9c84b3","Type":"ContainerStarted","Data":"034dd472ec860fdc180498007717a215ae09c6e9488e01ecc329c892599410ed"} Sep 10 12:32:14.368472 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:14.368435 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wqq6h" event={"ID":"ecc55eac-e95e-4d11-9436-f465ac9c84b3","Type":"ContainerStarted","Data":"04ccf9a66947efa5c0525336943087d8e9077c2a3dff4454af56742cdc6ab7a4"} Sep 10 12:32:14.386794 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:14.386728 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-wqq6h" podStartSLOduration=33.390196575 podStartE2EDuration="35.386713587s" totalImagesPullingTime="1.996517012s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:31:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:32:14.385814189 +0000 UTC m=+140.119408906" watchObservedRunningTime="2026-09-10 12:32:14.386713587 +0000 UTC m=+140.120308304" Sep 10 12:32:19.001085 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.001047 2588 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" podUID="4913242d-abae-4627-911b-0cc24d653a45" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 10 12:32:19.002019 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.001122 2588 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" Sep 10 12:32:19.002019 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.001590 2588 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"2686f466f087745d8862047543be5ce7b3ac521ce10cf1bf4699b150bce097e6"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" containerMessage="Container service-proxy failed liveness probe, will be restarted" Sep 10 12:32:19.002019 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.001649 2588 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" podUID="4913242d-abae-4627-911b-0cc24d653a45" containerName="service-proxy" containerID="cri-o://2686f466f087745d8862047543be5ce7b3ac521ce10cf1bf4699b150bce097e6" gracePeriod=30 Sep 10 12:32:19.320766 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.320721 2588 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-l89fj"] Sep 10 12:32:19.323929 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.323908 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.330238 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.330206 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 10 12:32:19.330346 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.330273 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 10 12:32:19.330572 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.330553 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 10 12:32:19.330667 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.330584 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 10 12:32:19.330978 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.330961 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 10 12:32:19.331199 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.331185 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5j6g8\"" Sep 10 12:32:19.331503 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.331486 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 10 12:32:19.386919 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.386884 2588 generic.go:356] "Generic (PLEG): container finished" podID="4913242d-abae-4627-911b-0cc24d653a45" containerID="2686f466f087745d8862047543be5ce7b3ac521ce10cf1bf4699b150bce097e6" exitCode=2 Sep 10 12:32:19.387088 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.386932 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" event={"ID":"4913242d-abae-4627-911b-0cc24d653a45","Type":"ContainerDied","Data":"2686f466f087745d8862047543be5ce7b3ac521ce10cf1bf4699b150bce097e6"} Sep 10 12:32:19.387088 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.386966 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77997bd6b8-nvd5x" event={"ID":"4913242d-abae-4627-911b-0cc24d653a45","Type":"ContainerStarted","Data":"d10ecbc6a416dac37953b396f6878cca7464c29d80e22ac7e6c8e8e2e934c0e4"} Sep 10 12:32:19.457459 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.457423 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c803e13-07a0-4250-8f81-80ac7cdc47bd-metrics-client-ca\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.457663 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.457484 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4c803e13-07a0-4250-8f81-80ac7cdc47bd-node-exporter-textfile\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.457663 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.457588 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4c803e13-07a0-4250-8f81-80ac7cdc47bd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.457663 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.457618 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4c803e13-07a0-4250-8f81-80ac7cdc47bd-node-exporter-tls\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.457663 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.457658 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4c803e13-07a0-4250-8f81-80ac7cdc47bd-node-exporter-wtmp\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.457911 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.457717 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trkgf\" (UniqueName: \"kubernetes.io/projected/4c803e13-07a0-4250-8f81-80ac7cdc47bd-kube-api-access-trkgf\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.457911 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.457781 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4c803e13-07a0-4250-8f81-80ac7cdc47bd-root\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.457911 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.457823 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c803e13-07a0-4250-8f81-80ac7cdc47bd-sys\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.457911 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.457857 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4c803e13-07a0-4250-8f81-80ac7cdc47bd-node-exporter-accelerators-collector-config\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.558844 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.558789 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4c803e13-07a0-4250-8f81-80ac7cdc47bd-node-exporter-textfile\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.559041 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.558884 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4c803e13-07a0-4250-8f81-80ac7cdc47bd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.559041 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.558909 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4c803e13-07a0-4250-8f81-80ac7cdc47bd-node-exporter-tls\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.559041 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.558928 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4c803e13-07a0-4250-8f81-80ac7cdc47bd-node-exporter-wtmp\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.559041 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.558946 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trkgf\" (UniqueName: \"kubernetes.io/projected/4c803e13-07a0-4250-8f81-80ac7cdc47bd-kube-api-access-trkgf\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.559041 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.558960 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4c803e13-07a0-4250-8f81-80ac7cdc47bd-root\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.559041 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.558983 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c803e13-07a0-4250-8f81-80ac7cdc47bd-sys\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.559041 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.559002 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4c803e13-07a0-4250-8f81-80ac7cdc47bd-node-exporter-accelerators-collector-config\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.559041 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.559038 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c803e13-07a0-4250-8f81-80ac7cdc47bd-metrics-client-ca\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.559452 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:32:19.559054 2588 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 10 12:32:19.559452 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.559095 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4c803e13-07a0-4250-8f81-80ac7cdc47bd-root\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.559452 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.559099 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4c803e13-07a0-4250-8f81-80ac7cdc47bd-node-exporter-wtmp\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.559452 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:32:19.559142 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c803e13-07a0-4250-8f81-80ac7cdc47bd-node-exporter-tls podName:4c803e13-07a0-4250-8f81-80ac7cdc47bd nodeName:}" failed. No retries permitted until 2026-09-10 12:32:20.059119425 +0000 UTC m=+145.792714139 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4c803e13-07a0-4250-8f81-80ac7cdc47bd-node-exporter-tls") pod "node-exporter-l89fj" (UID: "4c803e13-07a0-4250-8f81-80ac7cdc47bd") : secret "node-exporter-tls" not found Sep 10 12:32:19.559452 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.559172 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c803e13-07a0-4250-8f81-80ac7cdc47bd-sys\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.559452 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.559214 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4c803e13-07a0-4250-8f81-80ac7cdc47bd-node-exporter-textfile\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.559669 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.559566 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c803e13-07a0-4250-8f81-80ac7cdc47bd-metrics-client-ca\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.559669 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.559631 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4c803e13-07a0-4250-8f81-80ac7cdc47bd-node-exporter-accelerators-collector-config\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.561339 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.561315 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4c803e13-07a0-4250-8f81-80ac7cdc47bd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:19.571995 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:19.571928 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trkgf\" (UniqueName: \"kubernetes.io/projected/4c803e13-07a0-4250-8f81-80ac7cdc47bd-kube-api-access-trkgf\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:20.062781 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:20.062721 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4c803e13-07a0-4250-8f81-80ac7cdc47bd-node-exporter-tls\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:20.065096 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:20.065069 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4c803e13-07a0-4250-8f81-80ac7cdc47bd-node-exporter-tls\") pod \"node-exporter-l89fj\" (UID: \"4c803e13-07a0-4250-8f81-80ac7cdc47bd\") " pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:20.233178 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:20.233132 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l89fj" Sep 10 12:32:20.244328 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:32:20.244298 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c803e13_07a0_4250_8f81_80ac7cdc47bd.slice/crio-4d223363f1e7b19c0feda60935b741e32ffc75591cf11fdbf449ba662f9946b3 WatchSource:0}: Error finding container 4d223363f1e7b19c0feda60935b741e32ffc75591cf11fdbf449ba662f9946b3: Status 404 returned error can't find the container with id 4d223363f1e7b19c0feda60935b741e32ffc75591cf11fdbf449ba662f9946b3 Sep 10 12:32:20.391351 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:20.391263 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l89fj" event={"ID":"4c803e13-07a0-4250-8f81-80ac7cdc47bd","Type":"ContainerStarted","Data":"4d223363f1e7b19c0feda60935b741e32ffc75591cf11fdbf449ba662f9946b3"} Sep 10 12:32:22.400749 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:22.400715 2588 generic.go:356] "Generic (PLEG): container finished" podID="4c803e13-07a0-4250-8f81-80ac7cdc47bd" containerID="798fa3286dffa4eaf410cb7e5fa1c5d21097362a4b33b00bdb641774c5871bd7" exitCode=0 Sep 10 12:32:22.401181 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:22.400832 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l89fj" event={"ID":"4c803e13-07a0-4250-8f81-80ac7cdc47bd","Type":"ContainerDied","Data":"798fa3286dffa4eaf410cb7e5fa1c5d21097362a4b33b00bdb641774c5871bd7"} Sep 10 12:32:23.409883 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:23.409843 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l89fj" event={"ID":"4c803e13-07a0-4250-8f81-80ac7cdc47bd","Type":"ContainerStarted","Data":"ec8dab19d6b76bfc2145c61b8ab531f9d59db31463d5516c5cde329cd1bf435b"} Sep 10 12:32:23.410283 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:23.409930 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l89fj" event={"ID":"4c803e13-07a0-4250-8f81-80ac7cdc47bd","Type":"ContainerStarted","Data":"3aec06531b909d71de82e7c04831bb127371c6a36d0aca324948bfa4902f8fae"} Sep 10 12:32:23.447187 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:23.447126 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-l89fj" podStartSLOduration=3.149498137 podStartE2EDuration="4.447103807s" totalImagesPullingTime="1.232693552s" totalInitContainerRuntime="64.912118ms" isStatefulPod=true podCreationTimestamp="2026-09-10 12:32:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:32:23.446828521 +0000 UTC m=+149.180423244" watchObservedRunningTime="2026-09-10 12:32:23.447103807 +0000 UTC m=+149.180698525" Sep 10 12:32:29.891099 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:29.891064 2588 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-55bf984797-4tkjd"] Sep 10 12:32:29.891459 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:32:29.891279 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-55bf984797-4tkjd" podUID="2530f8c5-460f-4ba2-8af9-86a62eaf6361" Sep 10 12:32:30.432495 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:30.432461 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:32:30.439038 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:30.439014 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:32:30.545354 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:30.545316 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2530f8c5-460f-4ba2-8af9-86a62eaf6361-ca-trust-extracted\") pod \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " Sep 10 12:32:30.545354 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:30.545364 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-certificates\") pod \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " Sep 10 12:32:30.545588 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:30.545394 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2530f8c5-460f-4ba2-8af9-86a62eaf6361-image-registry-private-configuration\") pod \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " Sep 10 12:32:30.545588 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:30.545424 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2530f8c5-460f-4ba2-8af9-86a62eaf6361-installation-pull-secrets\") pod \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " Sep 10 12:32:30.545677 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:30.545588 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bx4h\" (UniqueName: \"kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-kube-api-access-2bx4h\") pod \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " Sep 10 12:32:30.545677 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:30.545636 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-bound-sa-token\") pod \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " Sep 10 12:32:30.545677 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:30.545671 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2530f8c5-460f-4ba2-8af9-86a62eaf6361-trusted-ca\") pod \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\" (UID: \"2530f8c5-460f-4ba2-8af9-86a62eaf6361\") " Sep 10 12:32:30.545871 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:30.545701 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2530f8c5-460f-4ba2-8af9-86a62eaf6361" (UID: "2530f8c5-460f-4ba2-8af9-86a62eaf6361"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:32:30.545937 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:30.545925 2588 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-certificates\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:32:30.546072 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:30.546042 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2530f8c5-460f-4ba2-8af9-86a62eaf6361-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2530f8c5-460f-4ba2-8af9-86a62eaf6361" (UID: "2530f8c5-460f-4ba2-8af9-86a62eaf6361"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:32:30.546189 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:30.546157 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2530f8c5-460f-4ba2-8af9-86a62eaf6361-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2530f8c5-460f-4ba2-8af9-86a62eaf6361" (UID: "2530f8c5-460f-4ba2-8af9-86a62eaf6361"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:32:30.547967 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:30.547936 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-kube-api-access-2bx4h" (OuterVolumeSpecName: "kube-api-access-2bx4h") pod "2530f8c5-460f-4ba2-8af9-86a62eaf6361" (UID: "2530f8c5-460f-4ba2-8af9-86a62eaf6361"). InnerVolumeSpecName "kube-api-access-2bx4h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:32:30.548076 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:30.547942 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2530f8c5-460f-4ba2-8af9-86a62eaf6361-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2530f8c5-460f-4ba2-8af9-86a62eaf6361" (UID: "2530f8c5-460f-4ba2-8af9-86a62eaf6361"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:32:30.548240 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:30.548219 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2530f8c5-460f-4ba2-8af9-86a62eaf6361" (UID: "2530f8c5-460f-4ba2-8af9-86a62eaf6361"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:32:30.548280 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:30.548255 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2530f8c5-460f-4ba2-8af9-86a62eaf6361-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2530f8c5-460f-4ba2-8af9-86a62eaf6361" (UID: "2530f8c5-460f-4ba2-8af9-86a62eaf6361"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:32:30.647170 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:30.647132 2588 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2530f8c5-460f-4ba2-8af9-86a62eaf6361-ca-trust-extracted\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:32:30.647170 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:30.647162 2588 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2530f8c5-460f-4ba2-8af9-86a62eaf6361-image-registry-private-configuration\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:32:30.647170 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:30.647172 2588 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2530f8c5-460f-4ba2-8af9-86a62eaf6361-installation-pull-secrets\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:32:30.647170 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:30.647180 2588 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bx4h\" (UniqueName: \"kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-kube-api-access-2bx4h\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:32:30.647417 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:30.647191 2588 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-bound-sa-token\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:32:30.647417 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:30.647200 2588 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2530f8c5-460f-4ba2-8af9-86a62eaf6361-trusted-ca\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:32:31.435263 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:31.435232 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55bf984797-4tkjd" Sep 10 12:32:31.481494 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:31.481464 2588 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-55bf984797-4tkjd"] Sep 10 12:32:31.487588 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:31.487561 2588 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-55bf984797-4tkjd"] Sep 10 12:32:31.653860 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:31.653813 2588 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2530f8c5-460f-4ba2-8af9-86a62eaf6361-registry-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:32:31.720902 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:32:31.720818 2588 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-f75jz" podUID="f3aadb41-77bc-4679-9ea3-0b47e499c6e4" Sep 10 12:32:32.438626 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:32.438543 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f75jz" Sep 10 12:32:32.775239 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:32.775206 2588 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2530f8c5-460f-4ba2-8af9-86a62eaf6361" path="/var/lib/kubelet/pods/2530f8c5-460f-4ba2-8af9-86a62eaf6361/volumes" Sep 10 12:32:36.591284 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:36.591235 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls\") pod \"dns-default-f75jz\" (UID: \"f3aadb41-77bc-4679-9ea3-0b47e499c6e4\") " pod="openshift-dns/dns-default-f75jz" Sep 10 12:32:36.593793 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:36.593768 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3aadb41-77bc-4679-9ea3-0b47e499c6e4-metrics-tls\") pod \"dns-default-f75jz\" (UID: \"f3aadb41-77bc-4679-9ea3-0b47e499c6e4\") " pod="openshift-dns/dns-default-f75jz" Sep 10 12:32:36.641563 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:36.641526 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-c86jl\"" Sep 10 12:32:36.649812 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:36.649773 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f75jz" Sep 10 12:32:36.929359 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:36.929333 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f75jz"] Sep 10 12:32:36.931970 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:32:36.931943 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3aadb41_77bc_4679_9ea3_0b47e499c6e4.slice/crio-5ea8e894b112db199b3f7e04b1d97095b2e10abd2a6ed5e74aaec7f60206b4ea WatchSource:0}: Error finding container 5ea8e894b112db199b3f7e04b1d97095b2e10abd2a6ed5e74aaec7f60206b4ea: Status 404 returned error can't find the container with id 5ea8e894b112db199b3f7e04b1d97095b2e10abd2a6ed5e74aaec7f60206b4ea Sep 10 12:32:37.455154 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:37.455120 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f75jz" event={"ID":"f3aadb41-77bc-4679-9ea3-0b47e499c6e4","Type":"ContainerStarted","Data":"5ea8e894b112db199b3f7e04b1d97095b2e10abd2a6ed5e74aaec7f60206b4ea"} Sep 10 12:32:39.464229 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:39.464190 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f75jz" event={"ID":"f3aadb41-77bc-4679-9ea3-0b47e499c6e4","Type":"ContainerStarted","Data":"d417dfa55a419257b8c5b2a3a8fa2afd4504f1129048233e81f841e01156b563"} Sep 10 12:32:39.464622 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:39.464245 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f75jz" event={"ID":"f3aadb41-77bc-4679-9ea3-0b47e499c6e4","Type":"ContainerStarted","Data":"10afeace04ab5baf1f8f878118c231598078ed739232855fece1ee2ae9549f27"} Sep 10 12:32:39.464622 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:39.464277 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-f75jz" Sep 10 12:32:39.485141 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:39.485086 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-f75jz" podStartSLOduration=129.720637365 podStartE2EDuration="2m11.485065s" totalImagesPullingTime="1.764427635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:30:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:32:39.482873592 +0000 UTC m=+165.216468310" watchObservedRunningTime="2026-09-10 12:32:39.485065 +0000 UTC m=+165.218659719" Sep 10 12:32:49.470054 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:32:49.469972 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-f75jz" Sep 10 12:34:54.707356 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:34:54.707326 2588 kubelet.go:1648] "Image garbage collection succeeded" Sep 10 12:37:05.862040 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:37:05.862005 2588 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-c49dfcc48-rvsfh"] Sep 10 12:37:05.864911 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:37:05.864894 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-c49dfcc48-rvsfh" Sep 10 12:37:05.867490 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:37:05.867464 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-pdq2p\"" Sep 10 12:37:05.868183 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:37:05.868164 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 10 12:37:05.868183 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:37:05.868174 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 10 12:37:05.953214 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:37:05.953183 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-c49dfcc48-rvsfh"] Sep 10 12:37:05.959167 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:37:05.959130 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nll98\" (UniqueName: \"kubernetes.io/projected/a5cd184a-a6eb-4eb8-a98e-cc6188d5e5c3-kube-api-access-nll98\") pod \"feast-operator-controller-manager-c49dfcc48-rvsfh\" (UID: \"a5cd184a-a6eb-4eb8-a98e-cc6188d5e5c3\") " pod="feast-operator-system/feast-operator-controller-manager-c49dfcc48-rvsfh" Sep 10 12:37:06.060067 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:37:06.060026 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nll98\" (UniqueName: \"kubernetes.io/projected/a5cd184a-a6eb-4eb8-a98e-cc6188d5e5c3-kube-api-access-nll98\") pod \"feast-operator-controller-manager-c49dfcc48-rvsfh\" (UID: \"a5cd184a-a6eb-4eb8-a98e-cc6188d5e5c3\") " pod="feast-operator-system/feast-operator-controller-manager-c49dfcc48-rvsfh" Sep 10 12:37:06.071289 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:37:06.071256 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nll98\" (UniqueName: \"kubernetes.io/projected/a5cd184a-a6eb-4eb8-a98e-cc6188d5e5c3-kube-api-access-nll98\") pod \"feast-operator-controller-manager-c49dfcc48-rvsfh\" (UID: \"a5cd184a-a6eb-4eb8-a98e-cc6188d5e5c3\") " pod="feast-operator-system/feast-operator-controller-manager-c49dfcc48-rvsfh" Sep 10 12:37:06.174353 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:37:06.174259 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-c49dfcc48-rvsfh" Sep 10 12:37:06.458710 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:37:06.458625 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-c49dfcc48-rvsfh"] Sep 10 12:37:06.461508 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:37:06.461477 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5cd184a_a6eb_4eb8_a98e_cc6188d5e5c3.slice/crio-e6d40964cbac447cab52ca433223b6fdc6ae84750a6c17fd95433f7c8d967580 WatchSource:0}: Error finding container e6d40964cbac447cab52ca433223b6fdc6ae84750a6c17fd95433f7c8d967580: Status 404 returned error can't find the container with id e6d40964cbac447cab52ca433223b6fdc6ae84750a6c17fd95433f7c8d967580 Sep 10 12:37:06.463228 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:37:06.463200 2588 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 12:37:07.338292 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:37:07.338257 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-c49dfcc48-rvsfh" event={"ID":"a5cd184a-a6eb-4eb8-a98e-cc6188d5e5c3","Type":"ContainerStarted","Data":"e6d40964cbac447cab52ca433223b6fdc6ae84750a6c17fd95433f7c8d967580"} Sep 10 12:37:09.346870 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:37:09.346834 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-c49dfcc48-rvsfh" event={"ID":"a5cd184a-a6eb-4eb8-a98e-cc6188d5e5c3","Type":"ContainerStarted","Data":"9b8bb477d1f54d1b897e2ece070943b982152b249fe7724936b56107e967bbd2"} Sep 10 12:37:09.347323 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:37:09.346968 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-c49dfcc48-rvsfh" Sep 10 12:37:09.378812 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:37:09.378767 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-c49dfcc48-rvsfh" podStartSLOduration=2.012013846 podStartE2EDuration="4.378749409s" totalImagesPullingTime="2.366735563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:37:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:37:09.378267513 +0000 UTC m=+435.111862227" watchObservedRunningTime="2026-09-10 12:37:09.378749409 +0000 UTC m=+435.112344117" Sep 10 12:37:20.352881 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:37:20.352811 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-c49dfcc48-rvsfh" Sep 10 12:44:02.602807 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:02.602775 2588 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-test-apply-64p8m"] Sep 10 12:44:02.605745 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:02.605722 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-64p8m" Sep 10 12:44:02.608210 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:02.608188 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 10 12:44:02.608304 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:02.608213 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-cronjob-dockercfg-5kktn\"" Sep 10 12:44:02.608910 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:02.608896 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 10 12:44:02.609021 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:02.609003 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 10 12:44:02.613789 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:02.613770 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-64p8m"] Sep 10 12:44:02.620087 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:02.620064 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4fdk\" (UniqueName: \"kubernetes.io/projected/27a0e485-6483-48ec-914d-52f8bc0961e6-kube-api-access-x4fdk\") pod \"feast-test-apply-64p8m\" (UID: \"27a0e485-6483-48ec-914d-52f8bc0961e6\") " pod="test-ns-feast/feast-test-apply-64p8m" Sep 10 12:44:02.721038 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:02.721000 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4fdk\" (UniqueName: \"kubernetes.io/projected/27a0e485-6483-48ec-914d-52f8bc0961e6-kube-api-access-x4fdk\") pod \"feast-test-apply-64p8m\" (UID: \"27a0e485-6483-48ec-914d-52f8bc0961e6\") " pod="test-ns-feast/feast-test-apply-64p8m" Sep 10 12:44:02.730000 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:02.729971 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4fdk\" (UniqueName: \"kubernetes.io/projected/27a0e485-6483-48ec-914d-52f8bc0961e6-kube-api-access-x4fdk\") pod \"feast-test-apply-64p8m\" (UID: \"27a0e485-6483-48ec-914d-52f8bc0961e6\") " pod="test-ns-feast/feast-test-apply-64p8m" Sep 10 12:44:02.914331 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:02.914243 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-64p8m" Sep 10 12:44:03.185565 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:03.185470 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-64p8m"] Sep 10 12:44:03.192070 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:44:03.192030 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod27a0e485_6483_48ec_914d_52f8bc0961e6.slice/crio-ec25ed9d681b9ce73a7fa7a0447c86e32a914e5740300ba65bbd14149c380e37 WatchSource:0}: Error finding container ec25ed9d681b9ce73a7fa7a0447c86e32a914e5740300ba65bbd14149c380e37: Status 404 returned error can't find the container with id ec25ed9d681b9ce73a7fa7a0447c86e32a914e5740300ba65bbd14149c380e37 Sep 10 12:44:03.194110 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:03.194092 2588 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 12:44:03.697819 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:03.697777 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-64p8m" event={"ID":"27a0e485-6483-48ec-914d-52f8bc0961e6","Type":"ContainerStarted","Data":"ec25ed9d681b9ce73a7fa7a0447c86e32a914e5740300ba65bbd14149c380e37"} Sep 10 12:44:08.716700 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:08.716606 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-64p8m" event={"ID":"27a0e485-6483-48ec-914d-52f8bc0961e6","Type":"ContainerStarted","Data":"6a0c9e322808b1a545b81561f69ff3e30ea07bdc6feb2d318dbd3ce338b72922"} Sep 10 12:44:15.741334 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:15.741298 2588 generic.go:356] "Generic (PLEG): container finished" podID="27a0e485-6483-48ec-914d-52f8bc0961e6" containerID="6a0c9e322808b1a545b81561f69ff3e30ea07bdc6feb2d318dbd3ce338b72922" exitCode=0 Sep 10 12:44:15.741688 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:15.741369 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-64p8m" event={"ID":"27a0e485-6483-48ec-914d-52f8bc0961e6","Type":"ContainerDied","Data":"6a0c9e322808b1a545b81561f69ff3e30ea07bdc6feb2d318dbd3ce338b72922"} Sep 10 12:44:16.751906 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:16.751858 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-64p8m" event={"ID":"27a0e485-6483-48ec-914d-52f8bc0961e6","Type":"ContainerStarted","Data":"3ca1ee7a5ba7c848eb8e1b15f57860c04f982b0674d45ab4d0d5ce73bfc04207"} Sep 10 12:44:16.771719 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:16.771650 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-test-apply-64p8m" podStartSLOduration=3.279242217 podStartE2EDuration="14.771627101s" totalImagesPullingTime="5.149420412s" totalInitContainerRuntime="6.342964472s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:44:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:44:16.769519438 +0000 UTC m=+862.503114164" watchObservedRunningTime="2026-09-10 12:44:16.771627101 +0000 UTC m=+862.505221817" Sep 10 12:44:25.781194 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:25.781160 2588 generic.go:356] "Generic (PLEG): container finished" podID="27a0e485-6483-48ec-914d-52f8bc0961e6" containerID="3ca1ee7a5ba7c848eb8e1b15f57860c04f982b0674d45ab4d0d5ce73bfc04207" exitCode=0 Sep 10 12:44:25.781567 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:25.781231 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-64p8m" event={"ID":"27a0e485-6483-48ec-914d-52f8bc0961e6","Type":"ContainerDied","Data":"3ca1ee7a5ba7c848eb8e1b15f57860c04f982b0674d45ab4d0d5ce73bfc04207"} Sep 10 12:44:26.929638 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:26.929611 2588 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-64p8m" Sep 10 12:44:26.984323 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:26.984280 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4fdk\" (UniqueName: \"kubernetes.io/projected/27a0e485-6483-48ec-914d-52f8bc0961e6-kube-api-access-x4fdk\") pod \"27a0e485-6483-48ec-914d-52f8bc0961e6\" (UID: \"27a0e485-6483-48ec-914d-52f8bc0961e6\") " Sep 10 12:44:26.986540 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:26.986497 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27a0e485-6483-48ec-914d-52f8bc0961e6-kube-api-access-x4fdk" (OuterVolumeSpecName: "kube-api-access-x4fdk") pod "27a0e485-6483-48ec-914d-52f8bc0961e6" (UID: "27a0e485-6483-48ec-914d-52f8bc0961e6"). InnerVolumeSpecName "kube-api-access-x4fdk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:44:27.085182 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:27.085099 2588 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4fdk\" (UniqueName: \"kubernetes.io/projected/27a0e485-6483-48ec-914d-52f8bc0961e6-kube-api-access-x4fdk\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:44:27.789139 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:27.789103 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-64p8m" event={"ID":"27a0e485-6483-48ec-914d-52f8bc0961e6","Type":"ContainerDied","Data":"ec25ed9d681b9ce73a7fa7a0447c86e32a914e5740300ba65bbd14149c380e37"} Sep 10 12:44:27.789139 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:27.789144 2588 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-64p8m" Sep 10 12:44:27.789342 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:27.789147 2588 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec25ed9d681b9ce73a7fa7a0447c86e32a914e5740300ba65bbd14149c380e37" Sep 10 12:44:27.976323 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:27.976294 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-64p8m_27a0e485-6483-48ec-914d-52f8bc0961e6/feast-0/0.log" Sep 10 12:44:27.983907 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:44:27.983884 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-64p8m_27a0e485-6483-48ec-914d-52f8bc0961e6/feast-1/0.log" Sep 10 12:45:04.602864 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:45:04.602826 2588 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-test-apply-64p8m"] Sep 10 12:45:04.609977 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:45:04.609947 2588 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-test-apply-64p8m"] Sep 10 12:45:04.774975 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:45:04.774942 2588 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27a0e485-6483-48ec-914d-52f8bc0961e6" path="/var/lib/kubelet/pods/27a0e485-6483-48ec-914d-52f8bc0961e6/volumes" Sep 10 12:50:05.539979 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:05.539945 2588 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-c49dfcc48-rvsfh"] Sep 10 12:50:05.540473 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:05.540157 2588 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-c49dfcc48-rvsfh" podUID="a5cd184a-a6eb-4eb8-a98e-cc6188d5e5c3" containerName="manager" containerID="cri-o://9b8bb477d1f54d1b897e2ece070943b982152b249fe7724936b56107e967bbd2" gracePeriod=10 Sep 10 12:50:05.799057 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:05.799019 2588 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-c49dfcc48-rvsfh" Sep 10 12:50:05.863793 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:05.863760 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nll98\" (UniqueName: \"kubernetes.io/projected/a5cd184a-a6eb-4eb8-a98e-cc6188d5e5c3-kube-api-access-nll98\") pod \"a5cd184a-a6eb-4eb8-a98e-cc6188d5e5c3\" (UID: \"a5cd184a-a6eb-4eb8-a98e-cc6188d5e5c3\") " Sep 10 12:50:05.867291 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:05.867246 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5cd184a-a6eb-4eb8-a98e-cc6188d5e5c3-kube-api-access-nll98" (OuterVolumeSpecName: "kube-api-access-nll98") pod "a5cd184a-a6eb-4eb8-a98e-cc6188d5e5c3" (UID: "a5cd184a-a6eb-4eb8-a98e-cc6188d5e5c3"). InnerVolumeSpecName "kube-api-access-nll98". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:50:05.914980 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:05.914947 2588 generic.go:356] "Generic (PLEG): container finished" podID="a5cd184a-a6eb-4eb8-a98e-cc6188d5e5c3" containerID="9b8bb477d1f54d1b897e2ece070943b982152b249fe7724936b56107e967bbd2" exitCode=0 Sep 10 12:50:05.915141 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:05.914989 2588 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-c49dfcc48-rvsfh" Sep 10 12:50:05.915141 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:05.914990 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-c49dfcc48-rvsfh" event={"ID":"a5cd184a-a6eb-4eb8-a98e-cc6188d5e5c3","Type":"ContainerDied","Data":"9b8bb477d1f54d1b897e2ece070943b982152b249fe7724936b56107e967bbd2"} Sep 10 12:50:05.915141 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:05.915107 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-c49dfcc48-rvsfh" event={"ID":"a5cd184a-a6eb-4eb8-a98e-cc6188d5e5c3","Type":"ContainerDied","Data":"e6d40964cbac447cab52ca433223b6fdc6ae84750a6c17fd95433f7c8d967580"} Sep 10 12:50:05.915141 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:05.915137 2588 scope.go:117] "RemoveContainer" containerID="9b8bb477d1f54d1b897e2ece070943b982152b249fe7724936b56107e967bbd2" Sep 10 12:50:05.923062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:05.923039 2588 scope.go:117] "RemoveContainer" containerID="9b8bb477d1f54d1b897e2ece070943b982152b249fe7724936b56107e967bbd2" Sep 10 12:50:05.923417 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:50:05.923393 2588 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b8bb477d1f54d1b897e2ece070943b982152b249fe7724936b56107e967bbd2\": container with ID starting with 9b8bb477d1f54d1b897e2ece070943b982152b249fe7724936b56107e967bbd2 not found: ID does not exist" containerID="9b8bb477d1f54d1b897e2ece070943b982152b249fe7724936b56107e967bbd2" Sep 10 12:50:05.923509 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:05.923426 2588 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b8bb477d1f54d1b897e2ece070943b982152b249fe7724936b56107e967bbd2"} err="failed to get container status \"9b8bb477d1f54d1b897e2ece070943b982152b249fe7724936b56107e967bbd2\": rpc error: code = NotFound desc = could not find container \"9b8bb477d1f54d1b897e2ece070943b982152b249fe7724936b56107e967bbd2\": container with ID starting with 9b8bb477d1f54d1b897e2ece070943b982152b249fe7724936b56107e967bbd2 not found: ID does not exist" Sep 10 12:50:05.944853 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:05.944816 2588 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-c49dfcc48-rvsfh"] Sep 10 12:50:05.947103 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:05.947079 2588 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-c49dfcc48-rvsfh"] Sep 10 12:50:05.964688 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:05.964650 2588 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nll98\" (UniqueName: \"kubernetes.io/projected/a5cd184a-a6eb-4eb8-a98e-cc6188d5e5c3-kube-api-access-nll98\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:50:06.775204 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:06.775166 2588 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5cd184a-a6eb-4eb8-a98e-cc6188d5e5c3" path="/var/lib/kubelet/pods/a5cd184a-a6eb-4eb8-a98e-cc6188d5e5c3/volumes" Sep 10 12:50:33.781961 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.781919 2588 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l"] Sep 10 12:50:33.782358 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.782178 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27a0e485-6483-48ec-914d-52f8bc0961e6" containerName="feast-0" Sep 10 12:50:33.782358 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.782191 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="27a0e485-6483-48ec-914d-52f8bc0961e6" containerName="feast-0" Sep 10 12:50:33.782358 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.782205 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27a0e485-6483-48ec-914d-52f8bc0961e6" containerName="feast-1" Sep 10 12:50:33.782358 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.782211 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="27a0e485-6483-48ec-914d-52f8bc0961e6" containerName="feast-1" Sep 10 12:50:33.782358 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.782224 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5cd184a-a6eb-4eb8-a98e-cc6188d5e5c3" containerName="manager" Sep 10 12:50:33.782358 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.782229 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5cd184a-a6eb-4eb8-a98e-cc6188d5e5c3" containerName="manager" Sep 10 12:50:33.782358 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.782266 2588 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5cd184a-a6eb-4eb8-a98e-cc6188d5e5c3" containerName="manager" Sep 10 12:50:33.782358 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.782274 2588 memory_manager.go:368] "RemoveStaleState removing state" podUID="27a0e485-6483-48ec-914d-52f8bc0961e6" containerName="feast-1" Sep 10 12:50:33.785346 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.785326 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:50:33.787693 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.787664 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 10 12:50:33.787836 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.787673 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 10 12:50:33.787836 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.787673 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 10 12:50:33.787946 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.787832 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 10 12:50:33.788018 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.788002 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 10 12:50:33.788067 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.788026 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 10 12:50:33.788067 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.788041 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-66cjx\"" Sep 10 12:50:33.802218 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.802180 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l"] Sep 10 12:50:33.825794 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.825762 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6zw5l\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:50:33.825973 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.825801 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bnp7\" (UniqueName: \"kubernetes.io/projected/b8b34da7-9e61-4720-8e7c-95435ddc8413-kube-api-access-8bnp7\") pod \"feast-simple-feast-setup-57576b8fcf-6zw5l\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:50:33.825973 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.825863 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6zw5l\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:50:33.825973 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.825889 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b8b34da7-9e61-4720-8e7c-95435ddc8413-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-6zw5l\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:50:33.825973 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.825942 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6zw5l\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:50:33.826132 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.825991 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6zw5l\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:50:33.926328 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.926289 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6zw5l\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:50:33.926328 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.926331 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bnp7\" (UniqueName: \"kubernetes.io/projected/b8b34da7-9e61-4720-8e7c-95435ddc8413-kube-api-access-8bnp7\") pod \"feast-simple-feast-setup-57576b8fcf-6zw5l\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:50:33.926575 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.926371 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6zw5l\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:50:33.926575 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.926395 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b8b34da7-9e61-4720-8e7c-95435ddc8413-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-6zw5l\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:50:33.926575 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.926412 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6zw5l\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:50:33.926575 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.926428 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6zw5l\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:50:33.926575 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:50:33.926445 2588 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 10 12:50:33.926575 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:50:33.926520 2588 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Sep 10 12:50:33.926575 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:50:33.926543 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-offline-tls podName:b8b34da7-9e61-4720-8e7c-95435ddc8413 nodeName:}" failed. No retries permitted until 2026-09-10 12:50:34.426518971 +0000 UTC m=+1240.160113680 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-offline-tls") pod "feast-simple-feast-setup-57576b8fcf-6zw5l" (UID: "b8b34da7-9e61-4720-8e7c-95435ddc8413") : secret "feast-simple-feast-setup-offline-tls" not found Sep 10 12:50:33.926575 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:50:33.926580 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-online-tls podName:b8b34da7-9e61-4720-8e7c-95435ddc8413 nodeName:}" failed. No retries permitted until 2026-09-10 12:50:34.426562115 +0000 UTC m=+1240.160156831 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-online-tls") pod "feast-simple-feast-setup-57576b8fcf-6zw5l" (UID: "b8b34da7-9e61-4720-8e7c-95435ddc8413") : secret "feast-simple-feast-setup-online-tls" not found Sep 10 12:50:33.926970 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.926915 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b8b34da7-9e61-4720-8e7c-95435ddc8413-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-6zw5l\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:50:33.929009 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.928982 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6zw5l\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:50:33.929109 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.929020 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6zw5l\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:50:33.935527 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:33.935495 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bnp7\" (UniqueName: \"kubernetes.io/projected/b8b34da7-9e61-4720-8e7c-95435ddc8413-kube-api-access-8bnp7\") pod \"feast-simple-feast-setup-57576b8fcf-6zw5l\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:50:34.430615 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:34.430576 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6zw5l\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:50:34.430615 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:34.430623 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6zw5l\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:50:34.433116 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:34.433081 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6zw5l\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:50:34.433240 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:34.433087 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-6zw5l\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:50:34.694407 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:34.694309 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:50:34.977819 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:34.977713 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l"] Sep 10 12:50:34.981464 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:50:34.981437 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8b34da7_9e61_4720_8e7c_95435ddc8413.slice/crio-4a509cdc45fcb14c0c1a7765e68f8df49c59a2bc51f8c401854007e738b772c8 WatchSource:0}: Error finding container 4a509cdc45fcb14c0c1a7765e68f8df49c59a2bc51f8c401854007e738b772c8: Status 404 returned error can't find the container with id 4a509cdc45fcb14c0c1a7765e68f8df49c59a2bc51f8c401854007e738b772c8 Sep 10 12:50:34.983234 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:34.983219 2588 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 12:50:35.009123 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:35.009084 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" event={"ID":"b8b34da7-9e61-4720-8e7c-95435ddc8413","Type":"ContainerStarted","Data":"4a509cdc45fcb14c0c1a7765e68f8df49c59a2bc51f8c401854007e738b772c8"} Sep 10 12:50:50.076377 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:50.076340 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" event={"ID":"b8b34da7-9e61-4720-8e7c-95435ddc8413","Type":"ContainerStarted","Data":"23e8bafee51628c0c70445e95df65271db4f2235ae779222d857ac0bb9316972"} Sep 10 12:50:54.785618 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:54.785588 2588 scope.go:117] "RemoveContainer" containerID="3ca1ee7a5ba7c848eb8e1b15f57860c04f982b0674d45ab4d0d5ce73bfc04207" Sep 10 12:50:54.792974 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:54.792952 2588 scope.go:117] "RemoveContainer" containerID="6a0c9e322808b1a545b81561f69ff3e30ea07bdc6feb2d318dbd3ce338b72922" Sep 10 12:50:55.096366 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:55.096286 2588 generic.go:356] "Generic (PLEG): container finished" podID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerID="23e8bafee51628c0c70445e95df65271db4f2235ae779222d857ac0bb9316972" exitCode=0 Sep 10 12:50:55.096366 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:55.096359 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" event={"ID":"b8b34da7-9e61-4720-8e7c-95435ddc8413","Type":"ContainerDied","Data":"23e8bafee51628c0c70445e95df65271db4f2235ae779222d857ac0bb9316972"} Sep 10 12:50:56.102089 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:50:56.102048 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" event={"ID":"b8b34da7-9e61-4720-8e7c-95435ddc8413","Type":"ContainerStarted","Data":"da57be0d9f6d8af4288d3f47e2e5a4e25f0bf016f5e11773bb15c6fdb6915de6"} Sep 10 12:51:01.121384 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:01.121354 2588 generic.go:356] "Generic (PLEG): container finished" podID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerID="da57be0d9f6d8af4288d3f47e2e5a4e25f0bf016f5e11773bb15c6fdb6915de6" exitCode=0 Sep 10 12:51:01.121802 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:01.121430 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" event={"ID":"b8b34da7-9e61-4720-8e7c-95435ddc8413","Type":"ContainerDied","Data":"da57be0d9f6d8af4288d3f47e2e5a4e25f0bf016f5e11773bb15c6fdb6915de6"} Sep 10 12:51:02.139231 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:02.139175 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" event={"ID":"b8b34da7-9e61-4720-8e7c-95435ddc8413","Type":"ContainerStarted","Data":"80e6f6fb1be14cfeaed705bdc0696610c47bf84c2e7660d04cf55a7ad62a224e"} Sep 10 12:51:02.139710 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:02.139251 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" event={"ID":"b8b34da7-9e61-4720-8e7c-95435ddc8413","Type":"ContainerStarted","Data":"12144d58e1632a618ebd1ba90e7c5f5bdaeaa5307f423a1a3d887f7d674b30de"} Sep 10 12:51:02.139710 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:02.139271 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" event={"ID":"b8b34da7-9e61-4720-8e7c-95435ddc8413","Type":"ContainerStarted","Data":"7784a8127996616f57c6375bcf48bd5e92ebea17d9b98a639e28cd077f1d5bed"} Sep 10 12:51:02.139710 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:02.139289 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" event={"ID":"b8b34da7-9e61-4720-8e7c-95435ddc8413","Type":"ContainerStarted","Data":"823f2ded32c03d82ec410a1334a21e6a62d826708b6649a455d5a097e4e3f13b"} Sep 10 12:51:02.181431 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:02.181364 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" podStartSLOduration=15.06127629 podStartE2EDuration="29.181341569s" totalImagesPullingTime="14.120065279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:50:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:51:02.180036961 +0000 UTC m=+1267.913631678" watchObservedRunningTime="2026-09-10 12:51:02.181341569 +0000 UTC m=+1267.914936289" Sep 10 12:51:04.694676 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:04.694627 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:51:04.695180 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:04.694722 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:51:04.695180 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:04.694759 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:51:04.695180 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:04.694777 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:51:04.695655 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:04.695634 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:51:04.695723 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:04.695672 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:51:04.695723 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:04.695691 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:51:04.695723 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:04.695708 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:51:04.697077 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:04.697043 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.16:8443: connect: connection refused" Sep 10 12:51:04.697077 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:04.697059 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="online" probeResult="failure" output="Get \"https://10.132.0.16:6567/health\": dial tcp 10.132.0.16:6567: connect: connection refused" Sep 10 12:51:04.697228 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:04.697130 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.16:8816: connect: connection refused" Sep 10 12:51:04.697391 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:04.697367 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.16:6571: connect: connection refused" Sep 10 12:51:07.695779 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:07.695654 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.16:6571: connect: connection refused" Sep 10 12:51:07.695779 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:07.695682 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="online" probeResult="failure" output="Get \"https://10.132.0.16:6567/health\": dial tcp 10.132.0.16:6567: connect: connection refused" Sep 10 12:51:07.695779 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:07.695699 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.16:8443: connect: connection refused" Sep 10 12:51:07.696410 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:07.696187 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.16:8816: connect: connection refused" Sep 10 12:51:10.695865 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:10.695829 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:51:10.696247 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:10.695982 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:51:10.696320 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:10.696299 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:51:10.696868 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:10.696846 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:51:10.696982 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:10.696900 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:51:10.696982 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:10.696945 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:51:10.701354 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:10.701333 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:51:11.185123 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:51:11.185089 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:52:14.070969 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.070931 2588 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l"] Sep 10 12:52:14.075196 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.071175 2588 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="registry" containerID="cri-o://823f2ded32c03d82ec410a1334a21e6a62d826708b6649a455d5a097e4e3f13b" gracePeriod=30 Sep 10 12:52:14.075196 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.071279 2588 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="ui" containerID="cri-o://80e6f6fb1be14cfeaed705bdc0696610c47bf84c2e7660d04cf55a7ad62a224e" gracePeriod=30 Sep 10 12:52:14.075196 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.071295 2588 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="online" containerID="cri-o://7784a8127996616f57c6375bcf48bd5e92ebea17d9b98a639e28cd077f1d5bed" gracePeriod=30 Sep 10 12:52:14.075196 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.071279 2588 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="offline" containerID="cri-o://12144d58e1632a618ebd1ba90e7c5f5bdaeaa5307f423a1a3d887f7d674b30de" gracePeriod=30 Sep 10 12:52:14.496805 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.496687 2588 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8"] Sep 10 12:52:14.500195 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.500166 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:14.503140 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.503111 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-hhd5z\"" Sep 10 12:52:14.515031 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.514995 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8"] Sep 10 12:52:14.583525 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.583465 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nwgw8\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:14.583766 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.583570 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl54v\" (UniqueName: \"kubernetes.io/projected/b4d3563e-48cb-4472-9c7b-62af5a61633a-kube-api-access-jl54v\") pod \"feast-simple-feast-setup-57576b8fcf-nwgw8\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:14.583766 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.583602 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b4d3563e-48cb-4472-9c7b-62af5a61633a-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-nwgw8\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:14.583766 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.583650 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nwgw8\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:14.583766 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.583698 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nwgw8\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:14.583766 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.583759 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nwgw8\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:14.685108 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.685057 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl54v\" (UniqueName: \"kubernetes.io/projected/b4d3563e-48cb-4472-9c7b-62af5a61633a-kube-api-access-jl54v\") pod \"feast-simple-feast-setup-57576b8fcf-nwgw8\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:14.685108 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.685098 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b4d3563e-48cb-4472-9c7b-62af5a61633a-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-nwgw8\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:14.685369 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.685199 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nwgw8\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:14.685369 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.685220 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nwgw8\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:14.685369 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.685244 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nwgw8\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:14.685369 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.685263 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nwgw8\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:14.685369 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:52:14.685350 2588 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Sep 10 12:52:14.685619 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:52:14.685419 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-online-tls podName:b4d3563e-48cb-4472-9c7b-62af5a61633a nodeName:}" failed. No retries permitted until 2026-09-10 12:52:15.185398632 +0000 UTC m=+1340.918993337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-online-tls") pod "feast-simple-feast-setup-57576b8fcf-nwgw8" (UID: "b4d3563e-48cb-4472-9c7b-62af5a61633a") : secret "feast-simple-feast-setup-online-tls" not found Sep 10 12:52:14.685802 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.685777 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b4d3563e-48cb-4472-9c7b-62af5a61633a-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-nwgw8\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:14.688196 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.688163 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nwgw8\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:14.688457 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.688436 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nwgw8\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:14.688528 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.688440 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nwgw8\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:14.696611 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:14.696576 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl54v\" (UniqueName: \"kubernetes.io/projected/b4d3563e-48cb-4472-9c7b-62af5a61633a-kube-api-access-jl54v\") pod \"feast-simple-feast-setup-57576b8fcf-nwgw8\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:15.189372 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:15.189329 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nwgw8\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:15.191719 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:15.191697 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-nwgw8\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:15.406502 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:15.406470 2588 generic.go:356] "Generic (PLEG): container finished" podID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerID="80e6f6fb1be14cfeaed705bdc0696610c47bf84c2e7660d04cf55a7ad62a224e" exitCode=0 Sep 10 12:52:15.406502 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:15.406494 2588 generic.go:356] "Generic (PLEG): container finished" podID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerID="7784a8127996616f57c6375bcf48bd5e92ebea17d9b98a639e28cd077f1d5bed" exitCode=0 Sep 10 12:52:15.406697 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:15.406544 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" event={"ID":"b8b34da7-9e61-4720-8e7c-95435ddc8413","Type":"ContainerDied","Data":"80e6f6fb1be14cfeaed705bdc0696610c47bf84c2e7660d04cf55a7ad62a224e"} Sep 10 12:52:15.406697 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:15.406582 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" event={"ID":"b8b34da7-9e61-4720-8e7c-95435ddc8413","Type":"ContainerDied","Data":"7784a8127996616f57c6375bcf48bd5e92ebea17d9b98a639e28cd077f1d5bed"} Sep 10 12:52:15.409694 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:15.409675 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:15.691978 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:15.691189 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8"] Sep 10 12:52:16.412047 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:16.412015 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" event={"ID":"b4d3563e-48cb-4472-9c7b-62af5a61633a","Type":"ContainerStarted","Data":"387d1e14d4e48880eebc2f332a72d55584eaf8f57bf40ca9962996bc5c8f003d"} Sep 10 12:52:16.412429 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:16.412065 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" event={"ID":"b4d3563e-48cb-4472-9c7b-62af5a61633a","Type":"ContainerStarted","Data":"e185767b5bf728f2dc2cb88e8b383ff955801e5c69841610725bd656c6756955"} Sep 10 12:52:20.696857 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:20.696810 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.16:8443: connect: connection refused" Sep 10 12:52:21.182373 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:21.182330 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="online" probeResult="failure" output="Get \"https://10.132.0.16:6567/health\": dial tcp 10.132.0.16:6567: connect: connection refused" Sep 10 12:52:21.432591 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:21.432517 2588 generic.go:356] "Generic (PLEG): container finished" podID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerID="387d1e14d4e48880eebc2f332a72d55584eaf8f57bf40ca9962996bc5c8f003d" exitCode=0 Sep 10 12:52:21.432591 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:21.432555 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" event={"ID":"b4d3563e-48cb-4472-9c7b-62af5a61633a","Type":"ContainerDied","Data":"387d1e14d4e48880eebc2f332a72d55584eaf8f57bf40ca9962996bc5c8f003d"} Sep 10 12:52:22.438978 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:22.438943 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" event={"ID":"b4d3563e-48cb-4472-9c7b-62af5a61633a","Type":"ContainerStarted","Data":"cefc553a4af88969554b6bd9cec9409399072c26f88979d5b91ac4c8a1aeca3b"} Sep 10 12:52:27.461402 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:27.461372 2588 generic.go:356] "Generic (PLEG): container finished" podID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerID="cefc553a4af88969554b6bd9cec9409399072c26f88979d5b91ac4c8a1aeca3b" exitCode=0 Sep 10 12:52:27.461719 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:27.461419 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" event={"ID":"b4d3563e-48cb-4472-9c7b-62af5a61633a","Type":"ContainerDied","Data":"cefc553a4af88969554b6bd9cec9409399072c26f88979d5b91ac4c8a1aeca3b"} Sep 10 12:52:28.471191 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:28.471145 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" event={"ID":"b4d3563e-48cb-4472-9c7b-62af5a61633a","Type":"ContainerStarted","Data":"45d0e421a850e5fb1c33821b574add5fbc0c6e5e373fa71d7c939eabead93bd8"} Sep 10 12:52:28.471654 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:28.471216 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" event={"ID":"b4d3563e-48cb-4472-9c7b-62af5a61633a","Type":"ContainerStarted","Data":"5e468e18afba646a937810950471b76b681287adc917c4db94f654ec444c8138"} Sep 10 12:52:28.471654 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:28.471239 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" event={"ID":"b4d3563e-48cb-4472-9c7b-62af5a61633a","Type":"ContainerStarted","Data":"0c61dcec4f9a2c4356ed67b2620cd0b7cedfbc6c2237ab28b396ea859ff8a583"} Sep 10 12:52:28.471654 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:28.471258 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" event={"ID":"b4d3563e-48cb-4472-9c7b-62af5a61633a","Type":"ContainerStarted","Data":"88f152c04e78476a11e4b12182fe9adf2fd7615f9af39d63e396dfe085444b93"} Sep 10 12:52:28.515385 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:28.515310 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" podStartSLOduration=14.515284954 podStartE2EDuration="14.515284954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:52:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:52:28.509726977 +0000 UTC m=+1354.243321726" watchObservedRunningTime="2026-09-10 12:52:28.515284954 +0000 UTC m=+1354.248879678" Sep 10 12:52:30.410036 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:30.409996 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:30.410586 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:30.410551 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:30.410586 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:30.410589 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:30.410801 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:30.410608 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:30.412047 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:30.412000 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.17:8816: connect: connection refused" Sep 10 12:52:30.412047 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:30.412026 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.17:6571: connect: connection refused" Sep 10 12:52:30.412231 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:30.412026 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.17:8443: connect: connection refused" Sep 10 12:52:30.412231 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:30.412084 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="online" probeResult="failure" output="Get \"https://10.132.0.17:6567/health\": dial tcp 10.132.0.17:6567: connect: connection refused" Sep 10 12:52:30.697119 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:30.697015 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.16:8443: connect: connection refused" Sep 10 12:52:31.182792 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:31.182726 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="online" probeResult="failure" output="Get \"https://10.132.0.16:6567/health\": dial tcp 10.132.0.16:6567: connect: connection refused" Sep 10 12:52:33.410854 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:33.410805 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.17:6571: connect: connection refused" Sep 10 12:52:33.410854 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:33.410850 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="online" probeResult="failure" output="Get \"https://10.132.0.17:6567/health\": dial tcp 10.132.0.17:6567: connect: connection refused" Sep 10 12:52:33.411349 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:33.410805 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.17:8443: connect: connection refused" Sep 10 12:52:33.411349 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:33.410808 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.17:8816: connect: connection refused" Sep 10 12:52:35.410513 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:35.410463 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:35.411011 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:35.410546 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:35.411011 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:35.410568 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:35.411130 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:35.411086 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:36.410978 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:36.410945 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:36.411441 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:36.411145 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:36.411441 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:36.411357 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:36.416217 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:36.416197 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:36.508933 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:36.508898 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:36.509179 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:36.509156 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:36.509532 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:36.509513 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:36.511730 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:36.511714 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:52:40.696870 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:40.696828 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.16:8443: connect: connection refused" Sep 10 12:52:40.697340 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:40.696957 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:52:41.182015 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:41.181964 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="online" probeResult="failure" output="Get \"https://10.132.0.16:6567/health\": dial tcp 10.132.0.16:6567: connect: connection refused" Sep 10 12:52:41.182156 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:41.182110 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:52:44.542270 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:44.542237 2588 generic.go:356] "Generic (PLEG): container finished" podID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerID="12144d58e1632a618ebd1ba90e7c5f5bdaeaa5307f423a1a3d887f7d674b30de" exitCode=137 Sep 10 12:52:44.542270 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:44.542262 2588 generic.go:356] "Generic (PLEG): container finished" podID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerID="823f2ded32c03d82ec410a1334a21e6a62d826708b6649a455d5a097e4e3f13b" exitCode=137 Sep 10 12:52:44.542669 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:44.542307 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" event={"ID":"b8b34da7-9e61-4720-8e7c-95435ddc8413","Type":"ContainerDied","Data":"12144d58e1632a618ebd1ba90e7c5f5bdaeaa5307f423a1a3d887f7d674b30de"} Sep 10 12:52:44.542669 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:44.542342 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" event={"ID":"b8b34da7-9e61-4720-8e7c-95435ddc8413","Type":"ContainerDied","Data":"823f2ded32c03d82ec410a1334a21e6a62d826708b6649a455d5a097e4e3f13b"} Sep 10 12:52:44.718685 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:44.718655 2588 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:52:44.814820 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:44.814706 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-ui-tls\") pod \"b8b34da7-9e61-4720-8e7c-95435ddc8413\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " Sep 10 12:52:44.814820 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:44.814793 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b8b34da7-9e61-4720-8e7c-95435ddc8413-feast-data\") pod \"b8b34da7-9e61-4720-8e7c-95435ddc8413\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " Sep 10 12:52:44.814820 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:44.814811 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-online-tls\") pod \"b8b34da7-9e61-4720-8e7c-95435ddc8413\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " Sep 10 12:52:44.815078 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:44.814831 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-registry-tls\") pod \"b8b34da7-9e61-4720-8e7c-95435ddc8413\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " Sep 10 12:52:44.815078 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:44.814867 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-offline-tls\") pod \"b8b34da7-9e61-4720-8e7c-95435ddc8413\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " Sep 10 12:52:44.815078 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:44.814883 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bnp7\" (UniqueName: \"kubernetes.io/projected/b8b34da7-9e61-4720-8e7c-95435ddc8413-kube-api-access-8bnp7\") pod \"b8b34da7-9e61-4720-8e7c-95435ddc8413\" (UID: \"b8b34da7-9e61-4720-8e7c-95435ddc8413\") " Sep 10 12:52:44.815438 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:44.815404 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8b34da7-9e61-4720-8e7c-95435ddc8413-feast-data" (OuterVolumeSpecName: "feast-data") pod "b8b34da7-9e61-4720-8e7c-95435ddc8413" (UID: "b8b34da7-9e61-4720-8e7c-95435ddc8413"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:52:44.817303 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:44.817265 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "b8b34da7-9e61-4720-8e7c-95435ddc8413" (UID: "b8b34da7-9e61-4720-8e7c-95435ddc8413"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:52:44.817303 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:44.817292 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-online-tls" (OuterVolumeSpecName: "online-tls") pod "b8b34da7-9e61-4720-8e7c-95435ddc8413" (UID: "b8b34da7-9e61-4720-8e7c-95435ddc8413"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:52:44.817471 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:44.817389 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b8b34da7-9e61-4720-8e7c-95435ddc8413" (UID: "b8b34da7-9e61-4720-8e7c-95435ddc8413"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:52:44.817471 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:44.817411 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "b8b34da7-9e61-4720-8e7c-95435ddc8413" (UID: "b8b34da7-9e61-4720-8e7c-95435ddc8413"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:52:44.817471 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:44.817430 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8b34da7-9e61-4720-8e7c-95435ddc8413-kube-api-access-8bnp7" (OuterVolumeSpecName: "kube-api-access-8bnp7") pod "b8b34da7-9e61-4720-8e7c-95435ddc8413" (UID: "b8b34da7-9e61-4720-8e7c-95435ddc8413"). InnerVolumeSpecName "kube-api-access-8bnp7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:52:44.916031 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:44.915986 2588 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b8b34da7-9e61-4720-8e7c-95435ddc8413-feast-data\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:52:44.916031 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:44.916021 2588 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-online-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:52:44.916031 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:44.916030 2588 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-registry-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:52:44.916031 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:44.916038 2588 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-offline-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:52:44.916291 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:44.916048 2588 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bnp7\" (UniqueName: \"kubernetes.io/projected/b8b34da7-9e61-4720-8e7c-95435ddc8413-kube-api-access-8bnp7\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:52:44.916291 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:44.916058 2588 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b8b34da7-9e61-4720-8e7c-95435ddc8413-ui-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:52:45.548323 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:45.548288 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" event={"ID":"b8b34da7-9e61-4720-8e7c-95435ddc8413","Type":"ContainerDied","Data":"4a509cdc45fcb14c0c1a7765e68f8df49c59a2bc51f8c401854007e738b772c8"} Sep 10 12:52:45.548323 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:45.548323 2588 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l" Sep 10 12:52:45.548910 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:45.548347 2588 scope.go:117] "RemoveContainer" containerID="80e6f6fb1be14cfeaed705bdc0696610c47bf84c2e7660d04cf55a7ad62a224e" Sep 10 12:52:45.557147 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:45.557129 2588 scope.go:117] "RemoveContainer" containerID="12144d58e1632a618ebd1ba90e7c5f5bdaeaa5307f423a1a3d887f7d674b30de" Sep 10 12:52:45.565136 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:45.565113 2588 scope.go:117] "RemoveContainer" containerID="7784a8127996616f57c6375bcf48bd5e92ebea17d9b98a639e28cd077f1d5bed" Sep 10 12:52:45.570715 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:45.570684 2588 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l"] Sep 10 12:52:45.574118 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:45.574093 2588 scope.go:117] "RemoveContainer" containerID="823f2ded32c03d82ec410a1334a21e6a62d826708b6649a455d5a097e4e3f13b" Sep 10 12:52:45.577226 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:45.577199 2588 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-6zw5l"] Sep 10 12:52:45.582531 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:45.582511 2588 scope.go:117] "RemoveContainer" containerID="da57be0d9f6d8af4288d3f47e2e5a4e25f0bf016f5e11773bb15c6fdb6915de6" Sep 10 12:52:45.590496 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:45.590476 2588 scope.go:117] "RemoveContainer" containerID="23e8bafee51628c0c70445e95df65271db4f2235ae779222d857ac0bb9316972" Sep 10 12:52:46.774614 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:52:46.774583 2588 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" path="/var/lib/kubelet/pods/b8b34da7-9e61-4720-8e7c-95435ddc8413/volumes" Sep 10 12:53:39.495818 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.495778 2588 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj"] Sep 10 12:53:39.496407 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.496125 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="feast-apply" Sep 10 12:53:39.496407 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.496143 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="feast-apply" Sep 10 12:53:39.496407 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.496162 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="feast-init" Sep 10 12:53:39.496407 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.496170 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="feast-init" Sep 10 12:53:39.496407 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.496181 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="online" Sep 10 12:53:39.496407 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.496190 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="online" Sep 10 12:53:39.496407 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.496199 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="offline" Sep 10 12:53:39.496407 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.496206 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="offline" Sep 10 12:53:39.496407 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.496217 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="registry" Sep 10 12:53:39.496407 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.496224 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="registry" Sep 10 12:53:39.496407 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.496241 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="ui" Sep 10 12:53:39.496407 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.496249 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="ui" Sep 10 12:53:39.496407 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.496318 2588 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="offline" Sep 10 12:53:39.496407 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.496327 2588 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="online" Sep 10 12:53:39.496407 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.496337 2588 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="ui" Sep 10 12:53:39.496407 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.496347 2588 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8b34da7-9e61-4720-8e7c-95435ddc8413" containerName="registry" Sep 10 12:53:39.499862 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.499834 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:53:39.502152 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.502128 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 10 12:53:39.502302 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.502175 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 10 12:53:39.502302 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.502224 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-nvztk\"" Sep 10 12:53:39.502302 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.502175 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 10 12:53:39.502746 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.502719 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 10 12:53:39.502819 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.502809 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 10 12:53:39.502876 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.502833 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 10 12:53:39.509185 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.509156 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0409065d-2410-4df3-862f-feb0ca1beb59-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-56shj\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:53:39.509271 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.509194 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2sv9\" (UniqueName: \"kubernetes.io/projected/0409065d-2410-4df3-862f-feb0ca1beb59-kube-api-access-q2sv9\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-56shj\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:53:39.509271 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.509246 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0409065d-2410-4df3-862f-feb0ca1beb59-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-56shj\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:53:39.509271 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.509268 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0409065d-2410-4df3-862f-feb0ca1beb59-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-56shj\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:53:39.509370 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.509288 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0409065d-2410-4df3-862f-feb0ca1beb59-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-56shj\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:53:39.509370 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.509322 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0409065d-2410-4df3-862f-feb0ca1beb59-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-56shj\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:53:39.511938 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.511911 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj"] Sep 10 12:53:39.610227 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.610184 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0409065d-2410-4df3-862f-feb0ca1beb59-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-56shj\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:53:39.610227 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.610223 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0409065d-2410-4df3-862f-feb0ca1beb59-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-56shj\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:53:39.610486 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.610249 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0409065d-2410-4df3-862f-feb0ca1beb59-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-56shj\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:53:39.610486 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.610297 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0409065d-2410-4df3-862f-feb0ca1beb59-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-56shj\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:53:39.610486 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.610321 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2sv9\" (UniqueName: \"kubernetes.io/projected/0409065d-2410-4df3-862f-feb0ca1beb59-kube-api-access-q2sv9\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-56shj\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:53:39.610486 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.610355 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0409065d-2410-4df3-862f-feb0ca1beb59-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-56shj\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:53:39.610486 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:53:39.610423 2588 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found Sep 10 12:53:39.610729 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:53:39.610501 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0409065d-2410-4df3-862f-feb0ca1beb59-offline-tls podName:0409065d-2410-4df3-862f-feb0ca1beb59 nodeName:}" failed. No retries permitted until 2026-09-10 12:53:40.110482609 +0000 UTC m=+1425.844077324 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/0409065d-2410-4df3-862f-feb0ca1beb59-offline-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-56shj" (UID: "0409065d-2410-4df3-862f-feb0ca1beb59") : secret "feast-simple-feast-remote-setup-offline-tls" not found Sep 10 12:53:39.610729 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.610686 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0409065d-2410-4df3-862f-feb0ca1beb59-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-56shj\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:53:39.610971 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.610950 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0409065d-2410-4df3-862f-feb0ca1beb59-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-56shj\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:53:39.612951 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.612931 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0409065d-2410-4df3-862f-feb0ca1beb59-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-56shj\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:53:39.613062 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.612974 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0409065d-2410-4df3-862f-feb0ca1beb59-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-56shj\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:53:39.619481 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:39.619447 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2sv9\" (UniqueName: \"kubernetes.io/projected/0409065d-2410-4df3-862f-feb0ca1beb59-kube-api-access-q2sv9\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-56shj\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:53:40.115041 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:40.114985 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0409065d-2410-4df3-862f-feb0ca1beb59-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-56shj\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:53:40.117401 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:40.117378 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0409065d-2410-4df3-862f-feb0ca1beb59-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-56shj\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:53:40.410033 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:40.409925 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:53:40.696821 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:40.696799 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj"] Sep 10 12:53:40.699199 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:53:40.699160 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0409065d_2410_4df3_862f_feb0ca1beb59.slice/crio-b8e5f5b9809d975f4090e7bf521f8a78c844c0ea7811d026fae728585bfe3ea4 WatchSource:0}: Error finding container b8e5f5b9809d975f4090e7bf521f8a78c844c0ea7811d026fae728585bfe3ea4: Status 404 returned error can't find the container with id b8e5f5b9809d975f4090e7bf521f8a78c844c0ea7811d026fae728585bfe3ea4 Sep 10 12:53:40.737536 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:40.737500 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" event={"ID":"0409065d-2410-4df3-862f-feb0ca1beb59","Type":"ContainerStarted","Data":"b8e5f5b9809d975f4090e7bf521f8a78c844c0ea7811d026fae728585bfe3ea4"} Sep 10 12:53:41.743246 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:41.743205 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" event={"ID":"0409065d-2410-4df3-862f-feb0ca1beb59","Type":"ContainerStarted","Data":"fb4306f7dc77af3cef83026c534c0283c1ddeea30b68c8ffa165ae4cfa6fd1db"} Sep 10 12:53:46.762610 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:46.762532 2588 generic.go:356] "Generic (PLEG): container finished" podID="0409065d-2410-4df3-862f-feb0ca1beb59" containerID="fb4306f7dc77af3cef83026c534c0283c1ddeea30b68c8ffa165ae4cfa6fd1db" exitCode=0 Sep 10 12:53:46.762610 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:46.762593 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" event={"ID":"0409065d-2410-4df3-862f-feb0ca1beb59","Type":"ContainerDied","Data":"fb4306f7dc77af3cef83026c534c0283c1ddeea30b68c8ffa165ae4cfa6fd1db"} Sep 10 12:53:47.768296 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:53:47.768257 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" event={"ID":"0409065d-2410-4df3-862f-feb0ca1beb59","Type":"ContainerStarted","Data":"f8cf4f02f0d5c8ef9d5e10b64dbf3ee42c007b58451ff1e0f969b49e1287b99d"} Sep 10 12:54:11.853319 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:11.853286 2588 generic.go:356] "Generic (PLEG): container finished" podID="0409065d-2410-4df3-862f-feb0ca1beb59" containerID="f8cf4f02f0d5c8ef9d5e10b64dbf3ee42c007b58451ff1e0f969b49e1287b99d" exitCode=0 Sep 10 12:54:11.853771 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:11.853354 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" event={"ID":"0409065d-2410-4df3-862f-feb0ca1beb59","Type":"ContainerDied","Data":"f8cf4f02f0d5c8ef9d5e10b64dbf3ee42c007b58451ff1e0f969b49e1287b99d"} Sep 10 12:54:12.861276 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:12.861242 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" event={"ID":"0409065d-2410-4df3-862f-feb0ca1beb59","Type":"ContainerStarted","Data":"b1709711bdbbcf368b93fd9eea73dc48e72a664e08e22ce6bb3a37a0c9eff3b4"} Sep 10 12:54:12.861751 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:12.861293 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" event={"ID":"0409065d-2410-4df3-862f-feb0ca1beb59","Type":"ContainerStarted","Data":"c1dd1f8a2d861c6edee579ecc40b578b8e66cd6588a2b50f6f5f065673e6c318"} Sep 10 12:54:12.861751 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:12.861310 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" event={"ID":"0409065d-2410-4df3-862f-feb0ca1beb59","Type":"ContainerStarted","Data":"a92486ce7a32393869c4dab56e793971dd9afb184a9f182d3f8a543c1724924f"} Sep 10 12:54:12.892943 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:12.892857 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podStartSLOduration=33.892841282 podStartE2EDuration="33.892841282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:53:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:54:12.891351058 +0000 UTC m=+1458.624945775" watchObservedRunningTime="2026-09-10 12:54:12.892841282 +0000 UTC m=+1458.626435996" Sep 10 12:54:13.410340 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:13.410299 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:54:13.410542 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:13.410363 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:54:13.410542 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:13.410383 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:54:13.412056 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:13.412009 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.18:8816: connect: connection refused" Sep 10 12:54:13.412056 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:13.412041 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.18:8443: connect: connection refused" Sep 10 12:54:13.412266 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:13.412025 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": dial tcp 10.132.0.18:6567: connect: connection refused" Sep 10 12:54:16.410975 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:16.410938 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": dial tcp 10.132.0.18:6567: connect: connection refused" Sep 10 12:54:16.411450 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:16.410939 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.18:8816: connect: connection refused" Sep 10 12:54:16.411450 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:16.410939 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.18:8443: connect: connection refused" Sep 10 12:54:19.410674 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:19.410626 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.18:8816: connect: connection refused" Sep 10 12:54:19.411077 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:19.410678 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": dial tcp 10.132.0.18:6567: connect: connection refused" Sep 10 12:54:19.411077 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:19.410626 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.18:8443: connect: connection refused" Sep 10 12:54:20.410267 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:20.410227 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:54:20.410419 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:20.410282 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:54:20.410419 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:20.410295 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:54:22.411858 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:22.411827 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:54:22.412335 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:22.411960 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:54:22.412573 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:22.412555 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:54:22.412751 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:22.412725 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:54:23.414891 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:23.414850 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 10 12:54:26.414904 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:26.414859 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 10 12:54:29.414896 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:29.414855 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 10 12:54:32.415894 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:32.415851 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 10 12:54:35.413926 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:35.413884 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 10 12:54:38.305387 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:38.305351 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:54:39.240284 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:54:39.240251 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:55:21.415960 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:21.415867 2588 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 10 12:55:29.310899 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:29.310855 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 10 12:55:41.869358 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:41.869317 2588 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj"] Sep 10 12:55:41.869891 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:41.869609 2588 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="online" containerID="cri-o://a92486ce7a32393869c4dab56e793971dd9afb184a9f182d3f8a543c1724924f" gracePeriod=30 Sep 10 12:55:41.869891 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:41.869641 2588 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="ui" containerID="cri-o://b1709711bdbbcf368b93fd9eea73dc48e72a664e08e22ce6bb3a37a0c9eff3b4" gracePeriod=30 Sep 10 12:55:41.869891 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:41.869682 2588 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="offline" containerID="cri-o://c1dd1f8a2d861c6edee579ecc40b578b8e66cd6588a2b50f6f5f065673e6c318" gracePeriod=30 Sep 10 12:55:42.053180 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:42.053148 2588 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8"] Sep 10 12:55:42.053540 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:42.053386 2588 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="registry" containerID="cri-o://88f152c04e78476a11e4b12182fe9adf2fd7615f9af39d63e396dfe085444b93" gracePeriod=30 Sep 10 12:55:42.053540 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:42.053454 2588 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="offline" containerID="cri-o://5e468e18afba646a937810950471b76b681287adc917c4db94f654ec444c8138" gracePeriod=30 Sep 10 12:55:42.053540 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:42.053445 2588 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="online" containerID="cri-o://0c61dcec4f9a2c4356ed67b2620cd0b7cedfbc6c2237ab28b396ea859ff8a583" gracePeriod=30 Sep 10 12:55:42.053892 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:42.053648 2588 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="ui" containerID="cri-o://45d0e421a850e5fb1c33821b574add5fbc0c6e5e373fa71d7c939eabead93bd8" gracePeriod=30 Sep 10 12:55:42.412834 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:42.412786 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.18:8443: connect: connection refused" Sep 10 12:55:42.731127 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:55:42.731095 2588 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4d3563e_48cb_4472_9c7b_62af5a61633a.slice/crio-45d0e421a850e5fb1c33821b574add5fbc0c6e5e373fa71d7c939eabead93bd8.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0409065d_2410_4df3_862f_feb0ca1beb59.slice/crio-conmon-a92486ce7a32393869c4dab56e793971dd9afb184a9f182d3f8a543c1724924f.scope\": RecentStats: unable to find data in memory cache]" Sep 10 12:55:43.200095 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:43.200058 2588 generic.go:356] "Generic (PLEG): container finished" podID="0409065d-2410-4df3-862f-feb0ca1beb59" containerID="b1709711bdbbcf368b93fd9eea73dc48e72a664e08e22ce6bb3a37a0c9eff3b4" exitCode=0 Sep 10 12:55:43.200095 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:43.200087 2588 generic.go:356] "Generic (PLEG): container finished" podID="0409065d-2410-4df3-862f-feb0ca1beb59" containerID="a92486ce7a32393869c4dab56e793971dd9afb184a9f182d3f8a543c1724924f" exitCode=0 Sep 10 12:55:43.200573 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:43.200131 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" event={"ID":"0409065d-2410-4df3-862f-feb0ca1beb59","Type":"ContainerDied","Data":"b1709711bdbbcf368b93fd9eea73dc48e72a664e08e22ce6bb3a37a0c9eff3b4"} Sep 10 12:55:43.200573 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:43.200177 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" event={"ID":"0409065d-2410-4df3-862f-feb0ca1beb59","Type":"ContainerDied","Data":"a92486ce7a32393869c4dab56e793971dd9afb184a9f182d3f8a543c1724924f"} Sep 10 12:55:43.203158 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:43.203133 2588 generic.go:356] "Generic (PLEG): container finished" podID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerID="45d0e421a850e5fb1c33821b574add5fbc0c6e5e373fa71d7c939eabead93bd8" exitCode=0 Sep 10 12:55:43.203158 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:43.203153 2588 generic.go:356] "Generic (PLEG): container finished" podID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerID="0c61dcec4f9a2c4356ed67b2620cd0b7cedfbc6c2237ab28b396ea859ff8a583" exitCode=0 Sep 10 12:55:43.203322 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:43.203176 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" event={"ID":"b4d3563e-48cb-4472-9c7b-62af5a61633a","Type":"ContainerDied","Data":"45d0e421a850e5fb1c33821b574add5fbc0c6e5e373fa71d7c939eabead93bd8"} Sep 10 12:55:43.203322 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:43.203204 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" event={"ID":"b4d3563e-48cb-4472-9c7b-62af5a61633a","Type":"ContainerDied","Data":"0c61dcec4f9a2c4356ed67b2620cd0b7cedfbc6c2237ab28b396ea859ff8a583"} Sep 10 12:55:46.508952 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:46.508913 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="online" probeResult="failure" output="Get \"https://10.132.0.17:6567/health\": dial tcp 10.132.0.17:6567: connect: connection refused" Sep 10 12:55:46.509383 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:46.508910 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.17:8443: connect: connection refused" Sep 10 12:55:48.306241 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:48.306190 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": dial tcp 10.132.0.18:6567: connect: connection refused" Sep 10 12:55:52.412802 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:52.412757 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.18:8443: connect: connection refused" Sep 10 12:55:56.509218 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:56.509169 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.17:8443: connect: connection refused" Sep 10 12:55:56.509776 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:56.509227 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="online" probeResult="failure" output="Get \"https://10.132.0.17:6567/health\": dial tcp 10.132.0.17:6567: connect: connection refused" Sep 10 12:55:58.306212 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:55:58.306160 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": dial tcp 10.132.0.18:6567: connect: connection refused" Sep 10 12:56:02.412706 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:02.412654 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.18:8443: connect: connection refused" Sep 10 12:56:02.413201 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:02.412814 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:56:06.508979 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:06.508930 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.17:8443: connect: connection refused" Sep 10 12:56:06.509440 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:06.508936 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="online" probeResult="failure" output="Get \"https://10.132.0.17:6567/health\": dial tcp 10.132.0.17:6567: connect: connection refused" Sep 10 12:56:06.513508 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:06.509586 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:56:06.513508 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:06.509769 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:56:08.306582 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:08.306535 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": dial tcp 10.132.0.18:6567: connect: connection refused" Sep 10 12:56:08.307143 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:08.306656 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:56:12.055596 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.055570 2588 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:56:12.148437 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.148368 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0409065d-2410-4df3-862f-feb0ca1beb59-registry-tls\") pod \"0409065d-2410-4df3-862f-feb0ca1beb59\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " Sep 10 12:56:12.148437 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.148416 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2sv9\" (UniqueName: \"kubernetes.io/projected/0409065d-2410-4df3-862f-feb0ca1beb59-kube-api-access-q2sv9\") pod \"0409065d-2410-4df3-862f-feb0ca1beb59\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " Sep 10 12:56:12.148437 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.148436 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0409065d-2410-4df3-862f-feb0ca1beb59-offline-tls\") pod \"0409065d-2410-4df3-862f-feb0ca1beb59\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " Sep 10 12:56:12.148699 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.148465 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0409065d-2410-4df3-862f-feb0ca1beb59-feast-data\") pod \"0409065d-2410-4df3-862f-feb0ca1beb59\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " Sep 10 12:56:12.148699 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.148594 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0409065d-2410-4df3-862f-feb0ca1beb59-ui-tls\") pod \"0409065d-2410-4df3-862f-feb0ca1beb59\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " Sep 10 12:56:12.148699 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.148629 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0409065d-2410-4df3-862f-feb0ca1beb59-online-tls\") pod \"0409065d-2410-4df3-862f-feb0ca1beb59\" (UID: \"0409065d-2410-4df3-862f-feb0ca1beb59\") " Sep 10 12:56:12.148867 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.148835 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0409065d-2410-4df3-862f-feb0ca1beb59-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0409065d-2410-4df3-862f-feb0ca1beb59" (UID: "0409065d-2410-4df3-862f-feb0ca1beb59"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 12:56:12.149010 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.148983 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0409065d-2410-4df3-862f-feb0ca1beb59-feast-data" (OuterVolumeSpecName: "feast-data") pod "0409065d-2410-4df3-862f-feb0ca1beb59" (UID: "0409065d-2410-4df3-862f-feb0ca1beb59"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:56:12.150674 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.150644 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0409065d-2410-4df3-862f-feb0ca1beb59-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "0409065d-2410-4df3-862f-feb0ca1beb59" (UID: "0409065d-2410-4df3-862f-feb0ca1beb59"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:56:12.150825 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.150694 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0409065d-2410-4df3-862f-feb0ca1beb59-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "0409065d-2410-4df3-862f-feb0ca1beb59" (UID: "0409065d-2410-4df3-862f-feb0ca1beb59"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:56:12.150825 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.150721 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0409065d-2410-4df3-862f-feb0ca1beb59-kube-api-access-q2sv9" (OuterVolumeSpecName: "kube-api-access-q2sv9") pod "0409065d-2410-4df3-862f-feb0ca1beb59" (UID: "0409065d-2410-4df3-862f-feb0ca1beb59"). InnerVolumeSpecName "kube-api-access-q2sv9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:56:12.150825 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.150724 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0409065d-2410-4df3-862f-feb0ca1beb59-online-tls" (OuterVolumeSpecName: "online-tls") pod "0409065d-2410-4df3-862f-feb0ca1beb59" (UID: "0409065d-2410-4df3-862f-feb0ca1beb59"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:56:12.249343 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.249306 2588 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0409065d-2410-4df3-862f-feb0ca1beb59-registry-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:56:12.249343 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.249336 2588 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2sv9\" (UniqueName: \"kubernetes.io/projected/0409065d-2410-4df3-862f-feb0ca1beb59-kube-api-access-q2sv9\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:56:12.249343 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.249345 2588 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0409065d-2410-4df3-862f-feb0ca1beb59-offline-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:56:12.249343 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.249354 2588 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0409065d-2410-4df3-862f-feb0ca1beb59-feast-data\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:56:12.249584 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.249362 2588 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0409065d-2410-4df3-862f-feb0ca1beb59-ui-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:56:12.249584 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.249369 2588 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0409065d-2410-4df3-862f-feb0ca1beb59-online-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:56:12.306331 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.306295 2588 generic.go:356] "Generic (PLEG): container finished" podID="0409065d-2410-4df3-862f-feb0ca1beb59" containerID="c1dd1f8a2d861c6edee579ecc40b578b8e66cd6588a2b50f6f5f065673e6c318" exitCode=137 Sep 10 12:56:12.306500 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.306356 2588 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" Sep 10 12:56:12.306500 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.306371 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" event={"ID":"0409065d-2410-4df3-862f-feb0ca1beb59","Type":"ContainerDied","Data":"c1dd1f8a2d861c6edee579ecc40b578b8e66cd6588a2b50f6f5f065673e6c318"} Sep 10 12:56:12.306500 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.306408 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj" event={"ID":"0409065d-2410-4df3-862f-feb0ca1beb59","Type":"ContainerDied","Data":"b8e5f5b9809d975f4090e7bf521f8a78c844c0ea7811d026fae728585bfe3ea4"} Sep 10 12:56:12.306500 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.306426 2588 scope.go:117] "RemoveContainer" containerID="b1709711bdbbcf368b93fd9eea73dc48e72a664e08e22ce6bb3a37a0c9eff3b4" Sep 10 12:56:12.309929 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.309908 2588 generic.go:356] "Generic (PLEG): container finished" podID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerID="5e468e18afba646a937810950471b76b681287adc917c4db94f654ec444c8138" exitCode=137 Sep 10 12:56:12.309929 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.309926 2588 generic.go:356] "Generic (PLEG): container finished" podID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerID="88f152c04e78476a11e4b12182fe9adf2fd7615f9af39d63e396dfe085444b93" exitCode=137 Sep 10 12:56:12.310098 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.309954 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" event={"ID":"b4d3563e-48cb-4472-9c7b-62af5a61633a","Type":"ContainerDied","Data":"5e468e18afba646a937810950471b76b681287adc917c4db94f654ec444c8138"} Sep 10 12:56:12.310098 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.309981 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" event={"ID":"b4d3563e-48cb-4472-9c7b-62af5a61633a","Type":"ContainerDied","Data":"88f152c04e78476a11e4b12182fe9adf2fd7615f9af39d63e396dfe085444b93"} Sep 10 12:56:12.316107 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.316088 2588 scope.go:117] "RemoveContainer" containerID="c1dd1f8a2d861c6edee579ecc40b578b8e66cd6588a2b50f6f5f065673e6c318" Sep 10 12:56:12.323522 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.323503 2588 scope.go:117] "RemoveContainer" containerID="a92486ce7a32393869c4dab56e793971dd9afb184a9f182d3f8a543c1724924f" Sep 10 12:56:12.330412 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.330392 2588 scope.go:117] "RemoveContainer" containerID="f8cf4f02f0d5c8ef9d5e10b64dbf3ee42c007b58451ff1e0f969b49e1287b99d" Sep 10 12:56:12.333287 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.333266 2588 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj"] Sep 10 12:56:12.337911 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.337889 2588 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-56shj"] Sep 10 12:56:12.338453 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.338439 2588 scope.go:117] "RemoveContainer" containerID="fb4306f7dc77af3cef83026c534c0283c1ddeea30b68c8ffa165ae4cfa6fd1db" Sep 10 12:56:12.347216 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.347198 2588 scope.go:117] "RemoveContainer" containerID="b1709711bdbbcf368b93fd9eea73dc48e72a664e08e22ce6bb3a37a0c9eff3b4" Sep 10 12:56:12.347578 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:56:12.347559 2588 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1709711bdbbcf368b93fd9eea73dc48e72a664e08e22ce6bb3a37a0c9eff3b4\": container with ID starting with b1709711bdbbcf368b93fd9eea73dc48e72a664e08e22ce6bb3a37a0c9eff3b4 not found: ID does not exist" containerID="b1709711bdbbcf368b93fd9eea73dc48e72a664e08e22ce6bb3a37a0c9eff3b4" Sep 10 12:56:12.347631 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.347589 2588 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1709711bdbbcf368b93fd9eea73dc48e72a664e08e22ce6bb3a37a0c9eff3b4"} err="failed to get container status \"b1709711bdbbcf368b93fd9eea73dc48e72a664e08e22ce6bb3a37a0c9eff3b4\": rpc error: code = NotFound desc = could not find container \"b1709711bdbbcf368b93fd9eea73dc48e72a664e08e22ce6bb3a37a0c9eff3b4\": container with ID starting with b1709711bdbbcf368b93fd9eea73dc48e72a664e08e22ce6bb3a37a0c9eff3b4 not found: ID does not exist" Sep 10 12:56:12.347631 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.347608 2588 scope.go:117] "RemoveContainer" containerID="c1dd1f8a2d861c6edee579ecc40b578b8e66cd6588a2b50f6f5f065673e6c318" Sep 10 12:56:12.347903 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:56:12.347887 2588 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1dd1f8a2d861c6edee579ecc40b578b8e66cd6588a2b50f6f5f065673e6c318\": container with ID starting with c1dd1f8a2d861c6edee579ecc40b578b8e66cd6588a2b50f6f5f065673e6c318 not found: ID does not exist" containerID="c1dd1f8a2d861c6edee579ecc40b578b8e66cd6588a2b50f6f5f065673e6c318" Sep 10 12:56:12.347948 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.347907 2588 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1dd1f8a2d861c6edee579ecc40b578b8e66cd6588a2b50f6f5f065673e6c318"} err="failed to get container status \"c1dd1f8a2d861c6edee579ecc40b578b8e66cd6588a2b50f6f5f065673e6c318\": rpc error: code = NotFound desc = could not find container \"c1dd1f8a2d861c6edee579ecc40b578b8e66cd6588a2b50f6f5f065673e6c318\": container with ID starting with c1dd1f8a2d861c6edee579ecc40b578b8e66cd6588a2b50f6f5f065673e6c318 not found: ID does not exist" Sep 10 12:56:12.347948 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.347920 2588 scope.go:117] "RemoveContainer" containerID="a92486ce7a32393869c4dab56e793971dd9afb184a9f182d3f8a543c1724924f" Sep 10 12:56:12.348162 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:56:12.348144 2588 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a92486ce7a32393869c4dab56e793971dd9afb184a9f182d3f8a543c1724924f\": container with ID starting with a92486ce7a32393869c4dab56e793971dd9afb184a9f182d3f8a543c1724924f not found: ID does not exist" containerID="a92486ce7a32393869c4dab56e793971dd9afb184a9f182d3f8a543c1724924f" Sep 10 12:56:12.348218 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.348165 2588 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a92486ce7a32393869c4dab56e793971dd9afb184a9f182d3f8a543c1724924f"} err="failed to get container status \"a92486ce7a32393869c4dab56e793971dd9afb184a9f182d3f8a543c1724924f\": rpc error: code = NotFound desc = could not find container \"a92486ce7a32393869c4dab56e793971dd9afb184a9f182d3f8a543c1724924f\": container with ID starting with a92486ce7a32393869c4dab56e793971dd9afb184a9f182d3f8a543c1724924f not found: ID does not exist" Sep 10 12:56:12.348218 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.348176 2588 scope.go:117] "RemoveContainer" containerID="f8cf4f02f0d5c8ef9d5e10b64dbf3ee42c007b58451ff1e0f969b49e1287b99d" Sep 10 12:56:12.348362 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:56:12.348349 2588 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8cf4f02f0d5c8ef9d5e10b64dbf3ee42c007b58451ff1e0f969b49e1287b99d\": container with ID starting with f8cf4f02f0d5c8ef9d5e10b64dbf3ee42c007b58451ff1e0f969b49e1287b99d not found: ID does not exist" containerID="f8cf4f02f0d5c8ef9d5e10b64dbf3ee42c007b58451ff1e0f969b49e1287b99d" Sep 10 12:56:12.348402 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.348364 2588 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8cf4f02f0d5c8ef9d5e10b64dbf3ee42c007b58451ff1e0f969b49e1287b99d"} err="failed to get container status \"f8cf4f02f0d5c8ef9d5e10b64dbf3ee42c007b58451ff1e0f969b49e1287b99d\": rpc error: code = NotFound desc = could not find container \"f8cf4f02f0d5c8ef9d5e10b64dbf3ee42c007b58451ff1e0f969b49e1287b99d\": container with ID starting with f8cf4f02f0d5c8ef9d5e10b64dbf3ee42c007b58451ff1e0f969b49e1287b99d not found: ID does not exist" Sep 10 12:56:12.348402 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.348375 2588 scope.go:117] "RemoveContainer" containerID="fb4306f7dc77af3cef83026c534c0283c1ddeea30b68c8ffa165ae4cfa6fd1db" Sep 10 12:56:12.348576 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:56:12.348563 2588 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb4306f7dc77af3cef83026c534c0283c1ddeea30b68c8ffa165ae4cfa6fd1db\": container with ID starting with fb4306f7dc77af3cef83026c534c0283c1ddeea30b68c8ffa165ae4cfa6fd1db not found: ID does not exist" containerID="fb4306f7dc77af3cef83026c534c0283c1ddeea30b68c8ffa165ae4cfa6fd1db" Sep 10 12:56:12.348615 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.348579 2588 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb4306f7dc77af3cef83026c534c0283c1ddeea30b68c8ffa165ae4cfa6fd1db"} err="failed to get container status \"fb4306f7dc77af3cef83026c534c0283c1ddeea30b68c8ffa165ae4cfa6fd1db\": rpc error: code = NotFound desc = could not find container \"fb4306f7dc77af3cef83026c534c0283c1ddeea30b68c8ffa165ae4cfa6fd1db\": container with ID starting with fb4306f7dc77af3cef83026c534c0283c1ddeea30b68c8ffa165ae4cfa6fd1db not found: ID does not exist" Sep 10 12:56:12.705060 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.705031 2588 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:56:12.775450 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.775416 2588 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" path="/var/lib/kubelet/pods/0409065d-2410-4df3-862f-feb0ca1beb59/volumes" Sep 10 12:56:12.852530 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.852441 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jl54v\" (UniqueName: \"kubernetes.io/projected/b4d3563e-48cb-4472-9c7b-62af5a61633a-kube-api-access-jl54v\") pod \"b4d3563e-48cb-4472-9c7b-62af5a61633a\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " Sep 10 12:56:12.852530 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.852480 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-online-tls\") pod \"b4d3563e-48cb-4472-9c7b-62af5a61633a\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " Sep 10 12:56:12.852530 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.852530 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-offline-tls\") pod \"b4d3563e-48cb-4472-9c7b-62af5a61633a\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " Sep 10 12:56:12.852841 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.852686 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-ui-tls\") pod \"b4d3563e-48cb-4472-9c7b-62af5a61633a\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " Sep 10 12:56:12.852841 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.852725 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b4d3563e-48cb-4472-9c7b-62af5a61633a-feast-data\") pod \"b4d3563e-48cb-4472-9c7b-62af5a61633a\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " Sep 10 12:56:12.852841 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.852801 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-registry-tls\") pod \"b4d3563e-48cb-4472-9c7b-62af5a61633a\" (UID: \"b4d3563e-48cb-4472-9c7b-62af5a61633a\") " Sep 10 12:56:12.853403 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.853372 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4d3563e-48cb-4472-9c7b-62af5a61633a-feast-data" (OuterVolumeSpecName: "feast-data") pod "b4d3563e-48cb-4472-9c7b-62af5a61633a" (UID: "b4d3563e-48cb-4472-9c7b-62af5a61633a"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:56:12.854778 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.854712 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4d3563e-48cb-4472-9c7b-62af5a61633a-kube-api-access-jl54v" (OuterVolumeSpecName: "kube-api-access-jl54v") pod "b4d3563e-48cb-4472-9c7b-62af5a61633a" (UID: "b4d3563e-48cb-4472-9c7b-62af5a61633a"). InnerVolumeSpecName "kube-api-access-jl54v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:56:12.854778 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.854764 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "b4d3563e-48cb-4472-9c7b-62af5a61633a" (UID: "b4d3563e-48cb-4472-9c7b-62af5a61633a"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:56:12.854894 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.854789 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b4d3563e-48cb-4472-9c7b-62af5a61633a" (UID: "b4d3563e-48cb-4472-9c7b-62af5a61633a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:56:12.855025 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.855002 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-online-tls" (OuterVolumeSpecName: "online-tls") pod "b4d3563e-48cb-4472-9c7b-62af5a61633a" (UID: "b4d3563e-48cb-4472-9c7b-62af5a61633a"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:56:12.855025 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.855014 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "b4d3563e-48cb-4472-9c7b-62af5a61633a" (UID: "b4d3563e-48cb-4472-9c7b-62af5a61633a"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:56:12.953787 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.953752 2588 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-ui-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:56:12.953787 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.953781 2588 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/b4d3563e-48cb-4472-9c7b-62af5a61633a-feast-data\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:56:12.953787 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.953790 2588 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-registry-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:56:12.953999 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.953799 2588 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jl54v\" (UniqueName: \"kubernetes.io/projected/b4d3563e-48cb-4472-9c7b-62af5a61633a-kube-api-access-jl54v\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:56:12.953999 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.953808 2588 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-online-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:56:12.953999 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:12.953816 2588 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/b4d3563e-48cb-4472-9c7b-62af5a61633a-offline-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:56:13.316929 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:13.316897 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" event={"ID":"b4d3563e-48cb-4472-9c7b-62af5a61633a","Type":"ContainerDied","Data":"e185767b5bf728f2dc2cb88e8b383ff955801e5c69841610725bd656c6756955"} Sep 10 12:56:13.316929 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:13.316930 2588 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8" Sep 10 12:56:13.317446 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:13.316949 2588 scope.go:117] "RemoveContainer" containerID="45d0e421a850e5fb1c33821b574add5fbc0c6e5e373fa71d7c939eabead93bd8" Sep 10 12:56:13.324530 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:13.324510 2588 scope.go:117] "RemoveContainer" containerID="5e468e18afba646a937810950471b76b681287adc917c4db94f654ec444c8138" Sep 10 12:56:13.332526 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:13.332509 2588 scope.go:117] "RemoveContainer" containerID="0c61dcec4f9a2c4356ed67b2620cd0b7cedfbc6c2237ab28b396ea859ff8a583" Sep 10 12:56:13.339483 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:13.339463 2588 scope.go:117] "RemoveContainer" containerID="88f152c04e78476a11e4b12182fe9adf2fd7615f9af39d63e396dfe085444b93" Sep 10 12:56:13.341441 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:13.341418 2588 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8"] Sep 10 12:56:13.347541 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:13.347518 2588 scope.go:117] "RemoveContainer" containerID="cefc553a4af88969554b6bd9cec9409399072c26f88979d5b91ac4c8a1aeca3b" Sep 10 12:56:13.348271 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:13.348250 2588 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-nwgw8"] Sep 10 12:56:13.354765 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:13.354747 2588 scope.go:117] "RemoveContainer" containerID="387d1e14d4e48880eebc2f332a72d55584eaf8f57bf40ca9962996bc5c8f003d" Sep 10 12:56:14.775123 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:14.775089 2588 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" path="/var/lib/kubelet/pods/b4d3563e-48cb-4472-9c7b-62af5a61633a/volumes" Sep 10 12:56:53.570066 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.569981 2588 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph"] Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570233 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="feast-apply" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570245 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="feast-apply" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570254 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="online" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570260 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="online" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570266 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="offline" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570271 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="offline" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570277 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="feast-init" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570282 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="feast-init" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570288 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="registry" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570293 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="registry" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570298 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="offline" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570302 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="offline" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570312 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="feast-init" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570317 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="feast-init" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570325 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="feast-apply" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570331 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="feast-apply" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570341 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="online" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570347 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="online" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570352 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="ui" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570357 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="ui" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570362 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="ui" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570367 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="ui" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570401 2588 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="ui" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570408 2588 memory_manager.go:368] "RemoveStaleState removing state" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="offline" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570413 2588 memory_manager.go:368] "RemoveStaleState removing state" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="ui" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570418 2588 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="offline" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570424 2588 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="online" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570428 2588 memory_manager.go:368] "RemoveStaleState removing state" podUID="0409065d-2410-4df3-862f-feb0ca1beb59" containerName="online" Sep 10 12:56:53.570492 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.570433 2588 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4d3563e-48cb-4472-9c7b-62af5a61633a" containerName="registry" Sep 10 12:56:53.573517 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.573497 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:56:53.579262 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.579235 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 10 12:56:53.579262 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.579253 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 10 12:56:53.579448 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.579253 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 10 12:56:53.579847 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.579503 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 10 12:56:53.579847 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.579706 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 10 12:56:53.579847 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.579768 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 10 12:56:53.586558 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.586542 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-4v997\"" Sep 10 12:56:53.606771 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.606728 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph"] Sep 10 12:56:53.639703 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.639667 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7db58a8-4084-477e-ad7e-d90055e326d1-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-l6zph\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:56:53.639703 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.639702 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l6zph\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:56:53.639926 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.639727 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l6zph\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:56:53.639926 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.639773 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l6zph\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:56:53.639926 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.639790 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg7dz\" (UniqueName: \"kubernetes.io/projected/e7db58a8-4084-477e-ad7e-d90055e326d1-kube-api-access-sg7dz\") pod \"feast-simple-feast-setup-57576b8fcf-l6zph\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:56:53.639926 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.639807 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l6zph\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:56:53.740826 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.740789 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7db58a8-4084-477e-ad7e-d90055e326d1-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-l6zph\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:56:53.741019 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.740831 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l6zph\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:56:53.741019 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.740864 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l6zph\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:56:53.741019 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.740889 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l6zph\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:56:53.741019 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.740911 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg7dz\" (UniqueName: \"kubernetes.io/projected/e7db58a8-4084-477e-ad7e-d90055e326d1-kube-api-access-sg7dz\") pod \"feast-simple-feast-setup-57576b8fcf-l6zph\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:56:53.741019 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.740936 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l6zph\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:56:53.741279 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:56:53.741067 2588 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Sep 10 12:56:53.741279 ip-10-0-135-250 kubenswrapper[2588]: E0910 12:56:53.741147 2588 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-online-tls podName:e7db58a8-4084-477e-ad7e-d90055e326d1 nodeName:}" failed. No retries permitted until 2026-09-10 12:56:54.241125095 +0000 UTC m=+1619.974719794 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-online-tls") pod "feast-simple-feast-setup-57576b8fcf-l6zph" (UID: "e7db58a8-4084-477e-ad7e-d90055e326d1") : secret "feast-simple-feast-setup-online-tls" not found Sep 10 12:56:53.741279 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.741220 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7db58a8-4084-477e-ad7e-d90055e326d1-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-l6zph\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:56:53.743369 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.743341 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l6zph\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:56:53.743485 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.743468 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l6zph\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:56:53.743543 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.743521 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l6zph\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:56:53.752342 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:53.752316 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg7dz\" (UniqueName: \"kubernetes.io/projected/e7db58a8-4084-477e-ad7e-d90055e326d1-kube-api-access-sg7dz\") pod \"feast-simple-feast-setup-57576b8fcf-l6zph\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:56:54.244904 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:54.244864 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l6zph\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:56:54.247218 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:54.247198 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-l6zph\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:56:54.482527 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:54.482482 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:56:54.769090 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:54.769055 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph"] Sep 10 12:56:54.772486 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:56:54.772459 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7db58a8_4084_477e_ad7e_d90055e326d1.slice/crio-671bc2a67600f1746dd74b446856038e0718d3854652c0336174662f61b1d05f WatchSource:0}: Error finding container 671bc2a67600f1746dd74b446856038e0718d3854652c0336174662f61b1d05f: Status 404 returned error can't find the container with id 671bc2a67600f1746dd74b446856038e0718d3854652c0336174662f61b1d05f Sep 10 12:56:55.454407 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:55.454373 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" event={"ID":"e7db58a8-4084-477e-ad7e-d90055e326d1","Type":"ContainerStarted","Data":"4812ea6d36a9817772a7b534fbe7d3ff15599f3443a14ff5e4d1c74d6e3834de"} Sep 10 12:56:55.454407 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:56:55.454415 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" event={"ID":"e7db58a8-4084-477e-ad7e-d90055e326d1","Type":"ContainerStarted","Data":"671bc2a67600f1746dd74b446856038e0718d3854652c0336174662f61b1d05f"} Sep 10 12:57:00.474854 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:00.474813 2588 generic.go:356] "Generic (PLEG): container finished" podID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerID="4812ea6d36a9817772a7b534fbe7d3ff15599f3443a14ff5e4d1c74d6e3834de" exitCode=0 Sep 10 12:57:00.475279 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:00.474883 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" event={"ID":"e7db58a8-4084-477e-ad7e-d90055e326d1","Type":"ContainerDied","Data":"4812ea6d36a9817772a7b534fbe7d3ff15599f3443a14ff5e4d1c74d6e3834de"} Sep 10 12:57:01.479964 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:01.479916 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" event={"ID":"e7db58a8-4084-477e-ad7e-d90055e326d1","Type":"ContainerStarted","Data":"336168667a6da921f54dcbbda18a4c3325f4222fc88a1f56fbb17c0088d6b586"} Sep 10 12:57:07.502907 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:07.502874 2588 generic.go:356] "Generic (PLEG): container finished" podID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerID="336168667a6da921f54dcbbda18a4c3325f4222fc88a1f56fbb17c0088d6b586" exitCode=0 Sep 10 12:57:07.503278 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:07.502955 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" event={"ID":"e7db58a8-4084-477e-ad7e-d90055e326d1","Type":"ContainerDied","Data":"336168667a6da921f54dcbbda18a4c3325f4222fc88a1f56fbb17c0088d6b586"} Sep 10 12:57:08.511909 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:08.511868 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" event={"ID":"e7db58a8-4084-477e-ad7e-d90055e326d1","Type":"ContainerStarted","Data":"786d96ccf415e607a5c63c90ad6375f5ecd873272b4feaa247fe74c053d78ef0"} Sep 10 12:57:08.512392 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:08.511932 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" event={"ID":"e7db58a8-4084-477e-ad7e-d90055e326d1","Type":"ContainerStarted","Data":"043af474655df25da6e00d18fe493d7e2602a4e5908ddcc9295f2568a5d9c3a7"} Sep 10 12:57:08.512392 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:08.511952 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" event={"ID":"e7db58a8-4084-477e-ad7e-d90055e326d1","Type":"ContainerStarted","Data":"d3b21bef5504b0a0d6646012eb3a36561f8c46b6e7f00b3c528750fb4b1fb9c5"} Sep 10 12:57:08.512392 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:08.511968 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" event={"ID":"e7db58a8-4084-477e-ad7e-d90055e326d1","Type":"ContainerStarted","Data":"369c47f93d74634cc9d7a84d335c055bac8e95d4c2e43d6e05ccbc82fac313d6"} Sep 10 12:57:08.540005 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:08.539937 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" podStartSLOduration=15.539915957 podStartE2EDuration="15.539915957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:56:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:57:08.537721429 +0000 UTC m=+1634.271316149" watchObservedRunningTime="2026-09-10 12:57:08.539915957 +0000 UTC m=+1634.273510676" Sep 10 12:57:09.483205 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:09.483162 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:57:09.483401 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:09.483232 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:57:09.483401 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:09.483255 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:57:09.483401 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:09.483273 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:57:09.485396 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:09.485354 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.19:8443: connect: connection refused" Sep 10 12:57:09.485666 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:09.485385 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.19:6571: connect: connection refused" Sep 10 12:57:09.485783 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:09.485391 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.19:8816: connect: connection refused" Sep 10 12:57:09.485783 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:09.485410 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="online" probeResult="failure" output="Get \"https://10.132.0.19:6567/health\": dial tcp 10.132.0.19:6567: connect: connection refused" Sep 10 12:57:12.483007 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:12.482954 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.19:8443: connect: connection refused" Sep 10 12:57:12.483585 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:12.482966 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.19:6571: connect: connection refused" Sep 10 12:57:12.483585 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:12.482975 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.19:8816: connect: connection refused" Sep 10 12:57:12.483729 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:12.483647 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="online" probeResult="failure" output="Get \"https://10.132.0.19:6567/health\": dial tcp 10.132.0.19:6567: connect: connection refused" Sep 10 12:57:14.483378 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:14.483330 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:57:14.485858 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:14.483409 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:57:14.485858 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:14.483435 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:57:14.485858 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:14.483887 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:57:15.483906 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:15.483864 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.19:6571: connect: connection refused" Sep 10 12:57:15.484381 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:15.483863 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.19:8443: connect: connection refused" Sep 10 12:57:15.484493 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:15.483886 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="online" probeResult="failure" output="Get \"https://10.132.0.19:6567/health\": dial tcp 10.132.0.19:6567: connect: connection refused" Sep 10 12:57:15.484894 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:15.484875 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:57:15.547509 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:15.547476 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:57:18.484066 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:18.484032 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:57:18.484529 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:18.484159 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:57:18.484776 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:18.484758 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:57:18.484873 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:18.484854 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:57:18.489148 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:18.489125 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:57:18.559298 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:57:18.559270 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:58:19.557383 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:19.557297 2588 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph"] Sep 10 12:58:19.557979 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:19.557602 2588 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="registry" containerID="cri-o://369c47f93d74634cc9d7a84d335c055bac8e95d4c2e43d6e05ccbc82fac313d6" gracePeriod=30 Sep 10 12:58:19.557979 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:19.557647 2588 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="offline" containerID="cri-o://043af474655df25da6e00d18fe493d7e2602a4e5908ddcc9295f2568a5d9c3a7" gracePeriod=30 Sep 10 12:58:19.557979 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:19.557671 2588 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="online" containerID="cri-o://d3b21bef5504b0a0d6646012eb3a36561f8c46b6e7f00b3c528750fb4b1fb9c5" gracePeriod=30 Sep 10 12:58:19.557979 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:19.557672 2588 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="ui" containerID="cri-o://786d96ccf415e607a5c63c90ad6375f5ecd873272b4feaa247fe74c053d78ef0" gracePeriod=30 Sep 10 12:58:20.108421 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.108383 2588 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b"] Sep 10 12:58:20.112712 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.112689 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:20.115277 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.115250 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-hj4bw\"" Sep 10 12:58:20.134000 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.133956 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b"] Sep 10 12:58:20.165915 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.165868 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-fmn7b\" (UID: \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:20.166115 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.165920 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtmm9\" (UniqueName: \"kubernetes.io/projected/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-kube-api-access-rtmm9\") pod \"feast-simple-feast-setup-57576b8fcf-fmn7b\" (UID: \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:20.166115 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.165956 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fmn7b\" (UID: \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:20.166115 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.165986 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fmn7b\" (UID: \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:20.166115 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.166096 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fmn7b\" (UID: \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:20.166279 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.166140 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fmn7b\" (UID: \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:20.266799 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.266762 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-fmn7b\" (UID: \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:20.266992 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.266812 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtmm9\" (UniqueName: \"kubernetes.io/projected/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-kube-api-access-rtmm9\") pod \"feast-simple-feast-setup-57576b8fcf-fmn7b\" (UID: \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:20.266992 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.266839 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fmn7b\" (UID: \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:20.266992 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.266867 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fmn7b\" (UID: \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:20.266992 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.266904 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fmn7b\" (UID: \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:20.266992 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.266944 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fmn7b\" (UID: \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:20.267249 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.267178 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-fmn7b\" (UID: \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:20.269384 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.269350 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fmn7b\" (UID: \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:20.269558 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.269532 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fmn7b\" (UID: \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:20.269860 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.269837 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fmn7b\" (UID: \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:20.270031 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.270011 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-fmn7b\" (UID: \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:20.282724 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.282697 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtmm9\" (UniqueName: \"kubernetes.io/projected/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-kube-api-access-rtmm9\") pod \"feast-simple-feast-setup-57576b8fcf-fmn7b\" (UID: \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:20.422527 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.422485 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:20.701307 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.701283 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b"] Sep 10 12:58:20.703449 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:58:20.703417 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ef6e76b_dcd0_49dd_8afe_a2e6911d2d05.slice/crio-e03ca3b9c629c9d208d87a749fb6850167c3a2185f2c5811df7be455ad434cb4 WatchSource:0}: Error finding container e03ca3b9c629c9d208d87a749fb6850167c3a2185f2c5811df7be455ad434cb4: Status 404 returned error can't find the container with id e03ca3b9c629c9d208d87a749fb6850167c3a2185f2c5811df7be455ad434cb4 Sep 10 12:58:20.774591 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.774552 2588 generic.go:356] "Generic (PLEG): container finished" podID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerID="786d96ccf415e607a5c63c90ad6375f5ecd873272b4feaa247fe74c053d78ef0" exitCode=0 Sep 10 12:58:20.774591 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.774590 2588 generic.go:356] "Generic (PLEG): container finished" podID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerID="d3b21bef5504b0a0d6646012eb3a36561f8c46b6e7f00b3c528750fb4b1fb9c5" exitCode=0 Sep 10 12:58:20.774881 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.774857 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" event={"ID":"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05","Type":"ContainerStarted","Data":"597afeacdeccf4abb7977a50597358c23354f8751ace25a48c7f84a96b369395"} Sep 10 12:58:20.774923 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.774900 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" event={"ID":"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05","Type":"ContainerStarted","Data":"e03ca3b9c629c9d208d87a749fb6850167c3a2185f2c5811df7be455ad434cb4"} Sep 10 12:58:20.774923 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.774914 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" event={"ID":"e7db58a8-4084-477e-ad7e-d90055e326d1","Type":"ContainerDied","Data":"786d96ccf415e607a5c63c90ad6375f5ecd873272b4feaa247fe74c053d78ef0"} Sep 10 12:58:20.775005 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:20.774932 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" event={"ID":"e7db58a8-4084-477e-ad7e-d90055e326d1","Type":"ContainerDied","Data":"d3b21bef5504b0a0d6646012eb3a36561f8c46b6e7f00b3c528750fb4b1fb9c5"} Sep 10 12:58:25.794495 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:25.794458 2588 generic.go:356] "Generic (PLEG): container finished" podID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerID="597afeacdeccf4abb7977a50597358c23354f8751ace25a48c7f84a96b369395" exitCode=0 Sep 10 12:58:25.794876 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:25.794533 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" event={"ID":"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05","Type":"ContainerDied","Data":"597afeacdeccf4abb7977a50597358c23354f8751ace25a48c7f84a96b369395"} Sep 10 12:58:26.799829 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:26.799795 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" event={"ID":"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05","Type":"ContainerStarted","Data":"a5824ce190e8807d04a1950f1aa0b6f566fa94d9e794059aee4b91799f42ed64"} Sep 10 12:58:28.484999 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:28.484965 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.19:8443: connect: connection refused" Sep 10 12:58:28.556916 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:28.556875 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="online" probeResult="failure" output="Get \"https://10.132.0.19:6567/health\": dial tcp 10.132.0.19:6567: connect: connection refused" Sep 10 12:58:32.823191 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:32.823158 2588 generic.go:356] "Generic (PLEG): container finished" podID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerID="a5824ce190e8807d04a1950f1aa0b6f566fa94d9e794059aee4b91799f42ed64" exitCode=0 Sep 10 12:58:32.823652 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:32.823232 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" event={"ID":"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05","Type":"ContainerDied","Data":"a5824ce190e8807d04a1950f1aa0b6f566fa94d9e794059aee4b91799f42ed64"} Sep 10 12:58:33.833044 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:33.833004 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" event={"ID":"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05","Type":"ContainerStarted","Data":"7fccdc2e242db81e4d8345a5f04aaba9221d0919510a119336e4f5be9e01a606"} Sep 10 12:58:33.833513 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:33.833065 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" event={"ID":"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05","Type":"ContainerStarted","Data":"4117b49761c0a6be585532cc6efe638e1575f360481ef829da3ca2a54c4c45dc"} Sep 10 12:58:33.833513 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:33.833087 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" event={"ID":"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05","Type":"ContainerStarted","Data":"d4d535d5374b43bc2d1bcc586a3f7ba1625024283cc2177de7dab01eea6ad85a"} Sep 10 12:58:33.833513 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:33.833107 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" event={"ID":"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05","Type":"ContainerStarted","Data":"86a2edc6e23f52a0e001048692d855678871c60792afa43eb0a4f69b952cefde"} Sep 10 12:58:33.861385 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:33.861321 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" podStartSLOduration=13.861298672 podStartE2EDuration="13.861298672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:58:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:58:33.858600755 +0000 UTC m=+1719.592195507" watchObservedRunningTime="2026-09-10 12:58:33.861298672 +0000 UTC m=+1719.594893390" Sep 10 12:58:35.422959 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:35.422924 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:35.423441 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:35.422989 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:35.423441 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:35.423009 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:35.423441 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:35.423026 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:35.425125 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:35.425081 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.20:8816: connect: connection refused" Sep 10 12:58:35.425273 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:35.425136 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.20:8443: connect: connection refused" Sep 10 12:58:35.425273 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:35.425083 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.20:6571: connect: connection refused" Sep 10 12:58:35.425273 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:35.425096 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="online" probeResult="failure" output="Get \"https://10.132.0.20:6567/health\": dial tcp 10.132.0.20:6567: connect: connection refused" Sep 10 12:58:38.423226 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:38.423174 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.20:8443: connect: connection refused" Sep 10 12:58:38.423856 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:38.423175 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.20:6571: connect: connection refused" Sep 10 12:58:38.424061 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:38.423227 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="online" probeResult="failure" output="Get \"https://10.132.0.20:6567/health\": dial tcp 10.132.0.20:6567: connect: connection refused" Sep 10 12:58:38.424215 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:38.423245 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.20:8816: connect: connection refused" Sep 10 12:58:38.485277 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:38.485210 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.19:8443: connect: connection refused" Sep 10 12:58:38.556452 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:38.556406 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="online" probeResult="failure" output="Get \"https://10.132.0.19:6567/health\": dial tcp 10.132.0.19:6567: connect: connection refused" Sep 10 12:58:40.423246 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:40.423200 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:40.423769 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:40.423271 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:40.423769 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:40.423292 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:40.423769 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:40.423309 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:41.422897 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:41.422852 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.20:8443: connect: connection refused" Sep 10 12:58:41.423788 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:41.423762 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:41.424170 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:41.423959 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:41.424569 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:41.424548 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:41.424802 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:41.424782 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:41.429362 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:41.429339 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:41.873469 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:41.873437 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:44.423411 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:44.423382 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:44.881046 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:44.881018 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 12:58:48.484860 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:48.484816 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.19:8443: connect: connection refused" Sep 10 12:58:48.485317 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:48.484963 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:58:48.556818 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:48.556772 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="online" probeResult="failure" output="Get \"https://10.132.0.19:6567/health\": dial tcp 10.132.0.19:6567: connect: connection refused" Sep 10 12:58:48.557000 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:48.556933 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:58:49.902360 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:49.902272 2588 generic.go:356] "Generic (PLEG): container finished" podID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerID="043af474655df25da6e00d18fe493d7e2602a4e5908ddcc9295f2568a5d9c3a7" exitCode=137 Sep 10 12:58:49.902360 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:49.902297 2588 generic.go:356] "Generic (PLEG): container finished" podID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerID="369c47f93d74634cc9d7a84d335c055bac8e95d4c2e43d6e05ccbc82fac313d6" exitCode=137 Sep 10 12:58:49.902360 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:49.902333 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" event={"ID":"e7db58a8-4084-477e-ad7e-d90055e326d1","Type":"ContainerDied","Data":"043af474655df25da6e00d18fe493d7e2602a4e5908ddcc9295f2568a5d9c3a7"} Sep 10 12:58:49.902796 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:49.902365 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" event={"ID":"e7db58a8-4084-477e-ad7e-d90055e326d1","Type":"ContainerDied","Data":"369c47f93d74634cc9d7a84d335c055bac8e95d4c2e43d6e05ccbc82fac313d6"} Sep 10 12:58:50.212947 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.212920 2588 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:58:50.293088 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.293051 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-offline-tls\") pod \"e7db58a8-4084-477e-ad7e-d90055e326d1\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " Sep 10 12:58:50.293239 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.293097 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sg7dz\" (UniqueName: \"kubernetes.io/projected/e7db58a8-4084-477e-ad7e-d90055e326d1-kube-api-access-sg7dz\") pod \"e7db58a8-4084-477e-ad7e-d90055e326d1\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " Sep 10 12:58:50.293239 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.293125 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7db58a8-4084-477e-ad7e-d90055e326d1-feast-data\") pod \"e7db58a8-4084-477e-ad7e-d90055e326d1\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " Sep 10 12:58:50.293239 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.293147 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-online-tls\") pod \"e7db58a8-4084-477e-ad7e-d90055e326d1\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " Sep 10 12:58:50.293239 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.293181 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-ui-tls\") pod \"e7db58a8-4084-477e-ad7e-d90055e326d1\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " Sep 10 12:58:50.293239 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.293218 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-registry-tls\") pod \"e7db58a8-4084-477e-ad7e-d90055e326d1\" (UID: \"e7db58a8-4084-477e-ad7e-d90055e326d1\") " Sep 10 12:58:50.293669 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.293644 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7db58a8-4084-477e-ad7e-d90055e326d1-feast-data" (OuterVolumeSpecName: "feast-data") pod "e7db58a8-4084-477e-ad7e-d90055e326d1" (UID: "e7db58a8-4084-477e-ad7e-d90055e326d1"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 12:58:50.295438 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.295398 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e7db58a8-4084-477e-ad7e-d90055e326d1" (UID: "e7db58a8-4084-477e-ad7e-d90055e326d1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:58:50.295830 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.295792 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "e7db58a8-4084-477e-ad7e-d90055e326d1" (UID: "e7db58a8-4084-477e-ad7e-d90055e326d1"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:58:50.295933 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.295864 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-online-tls" (OuterVolumeSpecName: "online-tls") pod "e7db58a8-4084-477e-ad7e-d90055e326d1" (UID: "e7db58a8-4084-477e-ad7e-d90055e326d1"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:58:50.295933 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.295875 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7db58a8-4084-477e-ad7e-d90055e326d1-kube-api-access-sg7dz" (OuterVolumeSpecName: "kube-api-access-sg7dz") pod "e7db58a8-4084-477e-ad7e-d90055e326d1" (UID: "e7db58a8-4084-477e-ad7e-d90055e326d1"). InnerVolumeSpecName "kube-api-access-sg7dz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 12:58:50.295933 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.295881 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "e7db58a8-4084-477e-ad7e-d90055e326d1" (UID: "e7db58a8-4084-477e-ad7e-d90055e326d1"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 12:58:50.394450 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.394391 2588 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-offline-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:58:50.394450 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.394437 2588 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sg7dz\" (UniqueName: \"kubernetes.io/projected/e7db58a8-4084-477e-ad7e-d90055e326d1-kube-api-access-sg7dz\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:58:50.394450 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.394448 2588 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7db58a8-4084-477e-ad7e-d90055e326d1-feast-data\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:58:50.394700 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.394479 2588 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-online-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:58:50.394700 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.394488 2588 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-ui-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:58:50.394700 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.394496 2588 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7db58a8-4084-477e-ad7e-d90055e326d1-registry-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 12:58:50.908377 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.908341 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" event={"ID":"e7db58a8-4084-477e-ad7e-d90055e326d1","Type":"ContainerDied","Data":"671bc2a67600f1746dd74b446856038e0718d3854652c0336174662f61b1d05f"} Sep 10 12:58:50.908868 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.908387 2588 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph" Sep 10 12:58:50.908868 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.908395 2588 scope.go:117] "RemoveContainer" containerID="786d96ccf415e607a5c63c90ad6375f5ecd873272b4feaa247fe74c053d78ef0" Sep 10 12:58:50.916335 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.916309 2588 scope.go:117] "RemoveContainer" containerID="043af474655df25da6e00d18fe493d7e2602a4e5908ddcc9295f2568a5d9c3a7" Sep 10 12:58:50.924999 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.924978 2588 scope.go:117] "RemoveContainer" containerID="d3b21bef5504b0a0d6646012eb3a36561f8c46b6e7f00b3c528750fb4b1fb9c5" Sep 10 12:58:50.928968 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.928936 2588 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph"] Sep 10 12:58:50.933383 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.933357 2588 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-l6zph"] Sep 10 12:58:50.933450 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.933368 2588 scope.go:117] "RemoveContainer" containerID="369c47f93d74634cc9d7a84d335c055bac8e95d4c2e43d6e05ccbc82fac313d6" Sep 10 12:58:50.941572 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.941543 2588 scope.go:117] "RemoveContainer" containerID="336168667a6da921f54dcbbda18a4c3325f4222fc88a1f56fbb17c0088d6b586" Sep 10 12:58:50.949188 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:50.949168 2588 scope.go:117] "RemoveContainer" containerID="4812ea6d36a9817772a7b534fbe7d3ff15599f3443a14ff5e4d1c74d6e3834de" Sep 10 12:58:52.774670 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:58:52.774638 2588 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" path="/var/lib/kubelet/pods/e7db58a8-4084-477e-ad7e-d90055e326d1/volumes" Sep 10 12:59:46.595134 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.595045 2588 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm"] Sep 10 12:59:46.595619 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.595356 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="ui" Sep 10 12:59:46.595619 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.595370 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="ui" Sep 10 12:59:46.595619 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.595379 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="registry" Sep 10 12:59:46.595619 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.595384 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="registry" Sep 10 12:59:46.595619 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.595389 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="feast-init" Sep 10 12:59:46.595619 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.595395 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="feast-init" Sep 10 12:59:46.595619 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.595410 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="feast-apply" Sep 10 12:59:46.595619 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.595415 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="feast-apply" Sep 10 12:59:46.595619 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.595422 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="online" Sep 10 12:59:46.595619 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.595427 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="online" Sep 10 12:59:46.595619 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.595433 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="offline" Sep 10 12:59:46.595619 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.595437 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="offline" Sep 10 12:59:46.595619 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.595477 2588 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="online" Sep 10 12:59:46.595619 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.595483 2588 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="ui" Sep 10 12:59:46.595619 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.595489 2588 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="offline" Sep 10 12:59:46.595619 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.595496 2588 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7db58a8-4084-477e-ad7e-d90055e326d1" containerName="registry" Sep 10 12:59:46.598854 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.598832 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 12:59:46.602011 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.601969 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 10 12:59:46.602011 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.602002 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-gzjhp\"" Sep 10 12:59:46.602231 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.602014 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 10 12:59:46.602231 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.601976 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 10 12:59:46.602231 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.602094 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 10 12:59:46.602395 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.602337 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 10 12:59:46.602395 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.602384 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 10 12:59:46.615874 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.615845 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm"] Sep 10 12:59:46.700095 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.700058 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0d982cd2-0567-49e1-acbb-c941bdabf485-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-97hwm\" (UID: \"0d982cd2-0567-49e1-acbb-c941bdabf485\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 12:59:46.700095 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.700094 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0d982cd2-0567-49e1-acbb-c941bdabf485-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-97hwm\" (UID: \"0d982cd2-0567-49e1-acbb-c941bdabf485\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 12:59:46.700321 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.700172 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0d982cd2-0567-49e1-acbb-c941bdabf485-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-97hwm\" (UID: \"0d982cd2-0567-49e1-acbb-c941bdabf485\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 12:59:46.700321 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.700235 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0d982cd2-0567-49e1-acbb-c941bdabf485-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-97hwm\" (UID: \"0d982cd2-0567-49e1-acbb-c941bdabf485\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 12:59:46.700321 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.700259 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnns2\" (UniqueName: \"kubernetes.io/projected/0d982cd2-0567-49e1-acbb-c941bdabf485-kube-api-access-gnns2\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-97hwm\" (UID: \"0d982cd2-0567-49e1-acbb-c941bdabf485\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 12:59:46.700445 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.700331 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0d982cd2-0567-49e1-acbb-c941bdabf485-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-97hwm\" (UID: \"0d982cd2-0567-49e1-acbb-c941bdabf485\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 12:59:46.801625 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.801587 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0d982cd2-0567-49e1-acbb-c941bdabf485-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-97hwm\" (UID: \"0d982cd2-0567-49e1-acbb-c941bdabf485\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 12:59:46.801839 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.801634 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0d982cd2-0567-49e1-acbb-c941bdabf485-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-97hwm\" (UID: \"0d982cd2-0567-49e1-acbb-c941bdabf485\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 12:59:46.801839 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.801654 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0d982cd2-0567-49e1-acbb-c941bdabf485-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-97hwm\" (UID: \"0d982cd2-0567-49e1-acbb-c941bdabf485\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 12:59:46.801839 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.801695 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0d982cd2-0567-49e1-acbb-c941bdabf485-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-97hwm\" (UID: \"0d982cd2-0567-49e1-acbb-c941bdabf485\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 12:59:46.801839 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.801768 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0d982cd2-0567-49e1-acbb-c941bdabf485-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-97hwm\" (UID: \"0d982cd2-0567-49e1-acbb-c941bdabf485\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 12:59:46.801839 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.801792 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnns2\" (UniqueName: \"kubernetes.io/projected/0d982cd2-0567-49e1-acbb-c941bdabf485-kube-api-access-gnns2\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-97hwm\" (UID: \"0d982cd2-0567-49e1-acbb-c941bdabf485\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 12:59:46.802242 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.802212 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0d982cd2-0567-49e1-acbb-c941bdabf485-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-97hwm\" (UID: \"0d982cd2-0567-49e1-acbb-c941bdabf485\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 12:59:46.802435 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.802413 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0d982cd2-0567-49e1-acbb-c941bdabf485-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-97hwm\" (UID: \"0d982cd2-0567-49e1-acbb-c941bdabf485\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 12:59:46.804176 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.804152 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0d982cd2-0567-49e1-acbb-c941bdabf485-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-97hwm\" (UID: \"0d982cd2-0567-49e1-acbb-c941bdabf485\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 12:59:46.804309 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.804290 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0d982cd2-0567-49e1-acbb-c941bdabf485-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-97hwm\" (UID: \"0d982cd2-0567-49e1-acbb-c941bdabf485\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 12:59:46.804359 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.804291 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0d982cd2-0567-49e1-acbb-c941bdabf485-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-97hwm\" (UID: \"0d982cd2-0567-49e1-acbb-c941bdabf485\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 12:59:46.814897 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.814869 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnns2\" (UniqueName: \"kubernetes.io/projected/0d982cd2-0567-49e1-acbb-c941bdabf485-kube-api-access-gnns2\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-97hwm\" (UID: \"0d982cd2-0567-49e1-acbb-c941bdabf485\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 12:59:46.907972 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:46.907882 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 12:59:47.192436 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:47.192339 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm"] Sep 10 12:59:47.200259 ip-10-0-135-250 kubenswrapper[2588]: W0910 12:59:47.200231 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d982cd2_0567_49e1_acbb_c941bdabf485.slice/crio-fd27660b8a7a2fa46b72ac728830db8ba2d143f11eb6e9b3f11b81fddd46eb58 WatchSource:0}: Error finding container fd27660b8a7a2fa46b72ac728830db8ba2d143f11eb6e9b3f11b81fddd46eb58: Status 404 returned error can't find the container with id fd27660b8a7a2fa46b72ac728830db8ba2d143f11eb6e9b3f11b81fddd46eb58 Sep 10 12:59:48.098629 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:48.098595 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" event={"ID":"0d982cd2-0567-49e1-acbb-c941bdabf485","Type":"ContainerStarted","Data":"7e6b54889059634eb34c754420cc2dc8d8eb30e5ca2d6ca9c3ee13d8b51c700c"} Sep 10 12:59:48.099071 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:48.098640 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" event={"ID":"0d982cd2-0567-49e1-acbb-c941bdabf485","Type":"ContainerStarted","Data":"fd27660b8a7a2fa46b72ac728830db8ba2d143f11eb6e9b3f11b81fddd46eb58"} Sep 10 12:59:53.116504 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:53.116467 2588 generic.go:356] "Generic (PLEG): container finished" podID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerID="7e6b54889059634eb34c754420cc2dc8d8eb30e5ca2d6ca9c3ee13d8b51c700c" exitCode=0 Sep 10 12:59:53.116919 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:53.116523 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" event={"ID":"0d982cd2-0567-49e1-acbb-c941bdabf485","Type":"ContainerDied","Data":"7e6b54889059634eb34c754420cc2dc8d8eb30e5ca2d6ca9c3ee13d8b51c700c"} Sep 10 12:59:54.122515 ip-10-0-135-250 kubenswrapper[2588]: I0910 12:59:54.122480 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" event={"ID":"0d982cd2-0567-49e1-acbb-c941bdabf485","Type":"ContainerStarted","Data":"222df65ef7508e594444f0f6f5141734f9c81566871e194f8a536ac688652ffa"} Sep 10 13:00:18.212396 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:18.212360 2588 generic.go:356] "Generic (PLEG): container finished" podID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerID="222df65ef7508e594444f0f6f5141734f9c81566871e194f8a536ac688652ffa" exitCode=0 Sep 10 13:00:18.212837 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:18.212428 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" event={"ID":"0d982cd2-0567-49e1-acbb-c941bdabf485","Type":"ContainerDied","Data":"222df65ef7508e594444f0f6f5141734f9c81566871e194f8a536ac688652ffa"} Sep 10 13:00:19.220182 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:19.220140 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" event={"ID":"0d982cd2-0567-49e1-acbb-c941bdabf485","Type":"ContainerStarted","Data":"aa71f82f9b98ba65464ec98064c23cf7e3aed8c172714ef0a38468ec0d8e6959"} Sep 10 13:00:19.220554 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:19.220203 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" event={"ID":"0d982cd2-0567-49e1-acbb-c941bdabf485","Type":"ContainerStarted","Data":"fa1a45b204766c86e0309adf304e01c702a8d3d225ac5ba6377d4cb85e094994"} Sep 10 13:00:19.220554 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:19.220223 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" event={"ID":"0d982cd2-0567-49e1-acbb-c941bdabf485","Type":"ContainerStarted","Data":"558dd8384ec2d60ecbedda81f3b45f31dfe6cb5f8b3757e0d41201bf9d9a94a9"} Sep 10 13:00:19.248478 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:19.248407 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podStartSLOduration=33.248379298 podStartE2EDuration="33.248379298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:59:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:00:19.24693361 +0000 UTC m=+1824.980528328" watchObservedRunningTime="2026-09-10 13:00:19.248379298 +0000 UTC m=+1824.981974018" Sep 10 13:00:19.908567 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:19.908527 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 13:00:19.908851 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:19.908820 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 13:00:19.908851 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:19.908851 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 13:00:19.910092 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:19.910056 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.21:8443: connect: connection refused" Sep 10 13:00:19.910206 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:19.910086 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.21:8816: connect: connection refused" Sep 10 13:00:19.910206 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:19.910119 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": dial tcp 10.132.0.21:6567: connect: connection refused" Sep 10 13:00:22.908248 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:22.908201 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.21:8816: connect: connection refused" Sep 10 13:00:22.908655 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:22.908201 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.21:8443: connect: connection refused" Sep 10 13:00:22.908655 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:22.908220 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": dial tcp 10.132.0.21:6567: connect: connection refused" Sep 10 13:00:25.908698 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:25.908647 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.21:8816: connect: connection refused" Sep 10 13:00:25.908698 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:25.908682 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": dial tcp 10.132.0.21:6567: connect: connection refused" Sep 10 13:00:25.909138 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:25.908647 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.21:8443: connect: connection refused" Sep 10 13:00:26.908336 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:26.908293 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 13:00:26.908519 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:26.908378 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 13:00:26.908519 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:26.908422 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 13:00:28.909784 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:28.909731 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 13:00:28.910186 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:28.909897 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 13:00:28.910456 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:28.910433 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 13:00:28.910625 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:28.910611 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 13:00:29.912900 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:29.912855 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 10 13:00:32.912922 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:32.912871 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 10 13:00:35.912944 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:35.912891 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 10 13:00:38.912877 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:38.912833 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 10 13:00:41.912954 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:41.912887 2588 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 10 13:00:44.868093 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:44.868058 2588 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 13:00:45.745118 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:45.745083 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 13:00:47.912935 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:00:47.912881 2588 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 10 13:01:46.688969 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:01:46.688925 2588 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm"] Sep 10 13:01:46.689509 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:01:46.689138 2588 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="online" containerID="cri-o://558dd8384ec2d60ecbedda81f3b45f31dfe6cb5f8b3757e0d41201bf9d9a94a9" gracePeriod=30 Sep 10 13:01:46.689509 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:01:46.689200 2588 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="offline" containerID="cri-o://fa1a45b204766c86e0309adf304e01c702a8d3d225ac5ba6377d4cb85e094994" gracePeriod=30 Sep 10 13:01:46.689509 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:01:46.689207 2588 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="ui" containerID="cri-o://aa71f82f9b98ba65464ec98064c23cf7e3aed8c172714ef0a38468ec0d8e6959" gracePeriod=30 Sep 10 13:01:46.873487 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:01:46.873449 2588 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b"] Sep 10 13:01:46.873811 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:01:46.873776 2588 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="registry" containerID="cri-o://86a2edc6e23f52a0e001048692d855678871c60792afa43eb0a4f69b952cefde" gracePeriod=30 Sep 10 13:01:46.873911 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:01:46.873819 2588 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="online" containerID="cri-o://d4d535d5374b43bc2d1bcc586a3f7ba1625024283cc2177de7dab01eea6ad85a" gracePeriod=30 Sep 10 13:01:46.873911 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:01:46.873826 2588 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="offline" containerID="cri-o://4117b49761c0a6be585532cc6efe638e1575f360481ef829da3ca2a54c4c45dc" gracePeriod=30 Sep 10 13:01:46.874028 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:01:46.873819 2588 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="ui" containerID="cri-o://7fccdc2e242db81e4d8345a5f04aaba9221d0919510a119336e4f5be9e01a606" gracePeriod=30 Sep 10 13:01:47.550376 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:01:47.550340 2588 generic.go:356] "Generic (PLEG): container finished" podID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerID="aa71f82f9b98ba65464ec98064c23cf7e3aed8c172714ef0a38468ec0d8e6959" exitCode=0 Sep 10 13:01:47.550562 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:01:47.550392 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" event={"ID":"0d982cd2-0567-49e1-acbb-c941bdabf485","Type":"ContainerDied","Data":"aa71f82f9b98ba65464ec98064c23cf7e3aed8c172714ef0a38468ec0d8e6959"} Sep 10 13:01:48.557270 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:01:48.557231 2588 generic.go:356] "Generic (PLEG): container finished" podID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerID="7fccdc2e242db81e4d8345a5f04aaba9221d0919510a119336e4f5be9e01a606" exitCode=0 Sep 10 13:01:48.557270 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:01:48.557258 2588 generic.go:356] "Generic (PLEG): container finished" podID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerID="d4d535d5374b43bc2d1bcc586a3f7ba1625024283cc2177de7dab01eea6ad85a" exitCode=0 Sep 10 13:01:48.557693 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:01:48.557303 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" event={"ID":"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05","Type":"ContainerDied","Data":"7fccdc2e242db81e4d8345a5f04aaba9221d0919510a119336e4f5be9e01a606"} Sep 10 13:01:48.557693 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:01:48.557352 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" event={"ID":"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05","Type":"ContainerDied","Data":"d4d535d5374b43bc2d1bcc586a3f7ba1625024283cc2177de7dab01eea6ad85a"} Sep 10 13:01:48.559775 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:01:48.559749 2588 generic.go:356] "Generic (PLEG): container finished" podID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerID="558dd8384ec2d60ecbedda81f3b45f31dfe6cb5f8b3757e0d41201bf9d9a94a9" exitCode=0 Sep 10 13:01:48.559775 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:01:48.559765 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" event={"ID":"0d982cd2-0567-49e1-acbb-c941bdabf485","Type":"ContainerDied","Data":"558dd8384ec2d60ecbedda81f3b45f31dfe6cb5f8b3757e0d41201bf9d9a94a9"} Sep 10 13:01:48.910520 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:01:48.910437 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.21:8443: connect: connection refused" Sep 10 13:01:51.870961 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:01:51.870917 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="online" probeResult="failure" output="Get \"https://10.132.0.20:6567/health\": dial tcp 10.132.0.20:6567: connect: connection refused" Sep 10 13:01:54.868992 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:01:54.868944 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": dial tcp 10.132.0.21:6567: connect: connection refused" Sep 10 13:01:54.880517 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:01:54.880474 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.20:8443: connect: connection refused" Sep 10 13:01:58.910649 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:01:58.910602 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.21:8443: connect: connection refused" Sep 10 13:02:01.870317 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:01.870276 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="online" probeResult="failure" output="Get \"https://10.132.0.20:6567/health\": dial tcp 10.132.0.20:6567: connect: connection refused" Sep 10 13:02:04.869042 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:04.869002 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": dial tcp 10.132.0.21:6567: connect: connection refused" Sep 10 13:02:04.880748 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:04.880708 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.20:8443: connect: connection refused" Sep 10 13:02:08.910612 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:08.910554 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.21:8443: connect: connection refused" Sep 10 13:02:08.911090 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:08.910685 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 13:02:11.870289 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:11.870244 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="online" probeResult="failure" output="Get \"https://10.132.0.20:6567/health\": dial tcp 10.132.0.20:6567: connect: connection refused" Sep 10 13:02:11.870682 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:11.870368 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 13:02:14.869221 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:14.869175 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="online" probeResult="failure" output="Get \"https://10.132.0.21:6567/health\": dial tcp 10.132.0.21:6567: connect: connection refused" Sep 10 13:02:14.871693 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:14.869297 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 13:02:14.881019 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:14.880989 2588 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.20:8443: connect: connection refused" Sep 10 13:02:14.881124 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:14.881101 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 13:02:17.331973 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.331948 2588 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 13:02:17.415836 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.415798 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0d982cd2-0567-49e1-acbb-c941bdabf485-online-tls\") pod \"0d982cd2-0567-49e1-acbb-c941bdabf485\" (UID: \"0d982cd2-0567-49e1-acbb-c941bdabf485\") " Sep 10 13:02:17.416021 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.415868 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0d982cd2-0567-49e1-acbb-c941bdabf485-feast-data\") pod \"0d982cd2-0567-49e1-acbb-c941bdabf485\" (UID: \"0d982cd2-0567-49e1-acbb-c941bdabf485\") " Sep 10 13:02:17.416021 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.415887 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnns2\" (UniqueName: \"kubernetes.io/projected/0d982cd2-0567-49e1-acbb-c941bdabf485-kube-api-access-gnns2\") pod \"0d982cd2-0567-49e1-acbb-c941bdabf485\" (UID: \"0d982cd2-0567-49e1-acbb-c941bdabf485\") " Sep 10 13:02:17.416021 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.415904 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0d982cd2-0567-49e1-acbb-c941bdabf485-registry-tls\") pod \"0d982cd2-0567-49e1-acbb-c941bdabf485\" (UID: \"0d982cd2-0567-49e1-acbb-c941bdabf485\") " Sep 10 13:02:17.416195 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.416065 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0d982cd2-0567-49e1-acbb-c941bdabf485-ui-tls\") pod \"0d982cd2-0567-49e1-acbb-c941bdabf485\" (UID: \"0d982cd2-0567-49e1-acbb-c941bdabf485\") " Sep 10 13:02:17.416195 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.416134 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0d982cd2-0567-49e1-acbb-c941bdabf485-offline-tls\") pod \"0d982cd2-0567-49e1-acbb-c941bdabf485\" (UID: \"0d982cd2-0567-49e1-acbb-c941bdabf485\") " Sep 10 13:02:17.416354 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.416326 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d982cd2-0567-49e1-acbb-c941bdabf485-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0d982cd2-0567-49e1-acbb-c941bdabf485" (UID: "0d982cd2-0567-49e1-acbb-c941bdabf485"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 13:02:17.416455 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.416430 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d982cd2-0567-49e1-acbb-c941bdabf485-feast-data" (OuterVolumeSpecName: "feast-data") pod "0d982cd2-0567-49e1-acbb-c941bdabf485" (UID: "0d982cd2-0567-49e1-acbb-c941bdabf485"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:02:17.419375 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.419346 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d982cd2-0567-49e1-acbb-c941bdabf485-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "0d982cd2-0567-49e1-acbb-c941bdabf485" (UID: "0d982cd2-0567-49e1-acbb-c941bdabf485"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 13:02:17.419375 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.419353 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d982cd2-0567-49e1-acbb-c941bdabf485-kube-api-access-gnns2" (OuterVolumeSpecName: "kube-api-access-gnns2") pod "0d982cd2-0567-49e1-acbb-c941bdabf485" (UID: "0d982cd2-0567-49e1-acbb-c941bdabf485"). InnerVolumeSpecName "kube-api-access-gnns2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:02:17.419375 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.419370 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d982cd2-0567-49e1-acbb-c941bdabf485-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "0d982cd2-0567-49e1-acbb-c941bdabf485" (UID: "0d982cd2-0567-49e1-acbb-c941bdabf485"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 13:02:17.419567 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.419386 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d982cd2-0567-49e1-acbb-c941bdabf485-online-tls" (OuterVolumeSpecName: "online-tls") pod "0d982cd2-0567-49e1-acbb-c941bdabf485" (UID: "0d982cd2-0567-49e1-acbb-c941bdabf485"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 13:02:17.510061 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.510035 2588 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 13:02:17.517365 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.517343 2588 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0d982cd2-0567-49e1-acbb-c941bdabf485-ui-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 13:02:17.517365 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.517363 2588 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0d982cd2-0567-49e1-acbb-c941bdabf485-offline-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 13:02:17.517517 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.517373 2588 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0d982cd2-0567-49e1-acbb-c941bdabf485-online-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 13:02:17.517517 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.517380 2588 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0d982cd2-0567-49e1-acbb-c941bdabf485-feast-data\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 13:02:17.517517 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.517388 2588 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnns2\" (UniqueName: \"kubernetes.io/projected/0d982cd2-0567-49e1-acbb-c941bdabf485-kube-api-access-gnns2\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 13:02:17.517517 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.517396 2588 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0d982cd2-0567-49e1-acbb-c941bdabf485-registry-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 13:02:17.618433 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.618377 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtmm9\" (UniqueName: \"kubernetes.io/projected/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-kube-api-access-rtmm9\") pod \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\" (UID: \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\") " Sep 10 13:02:17.618650 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.618466 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-feast-data\") pod \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\" (UID: \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\") " Sep 10 13:02:17.618650 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.618496 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-registry-tls\") pod \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\" (UID: \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\") " Sep 10 13:02:17.618650 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.618552 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-offline-tls\") pod \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\" (UID: \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\") " Sep 10 13:02:17.618650 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.618578 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-online-tls\") pod \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\" (UID: \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\") " Sep 10 13:02:17.618650 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.618615 2588 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-ui-tls\") pod \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\" (UID: \"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05\") " Sep 10 13:02:17.619071 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.619034 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-feast-data" (OuterVolumeSpecName: "feast-data") pod "6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" (UID: "6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 13:02:17.620677 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.620642 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-kube-api-access-rtmm9" (OuterVolumeSpecName: "kube-api-access-rtmm9") pod "6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" (UID: "6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05"). InnerVolumeSpecName "kube-api-access-rtmm9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 13:02:17.620853 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.620828 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" (UID: "6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 13:02:17.620912 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.620849 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" (UID: "6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 13:02:17.621086 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.621066 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" (UID: "6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 13:02:17.621145 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.621086 2588 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-online-tls" (OuterVolumeSpecName: "online-tls") pod "6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" (UID: "6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 13:02:17.667712 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.667679 2588 generic.go:356] "Generic (PLEG): container finished" podID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerID="fa1a45b204766c86e0309adf304e01c702a8d3d225ac5ba6377d4cb85e094994" exitCode=137 Sep 10 13:02:17.667890 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.667758 2588 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" Sep 10 13:02:17.667890 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.667767 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" event={"ID":"0d982cd2-0567-49e1-acbb-c941bdabf485","Type":"ContainerDied","Data":"fa1a45b204766c86e0309adf304e01c702a8d3d225ac5ba6377d4cb85e094994"} Sep 10 13:02:17.667890 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.667818 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm" event={"ID":"0d982cd2-0567-49e1-acbb-c941bdabf485","Type":"ContainerDied","Data":"fd27660b8a7a2fa46b72ac728830db8ba2d143f11eb6e9b3f11b81fddd46eb58"} Sep 10 13:02:17.667890 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.667840 2588 scope.go:117] "RemoveContainer" containerID="aa71f82f9b98ba65464ec98064c23cf7e3aed8c172714ef0a38468ec0d8e6959" Sep 10 13:02:17.670955 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.670929 2588 generic.go:356] "Generic (PLEG): container finished" podID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerID="4117b49761c0a6be585532cc6efe638e1575f360481ef829da3ca2a54c4c45dc" exitCode=137 Sep 10 13:02:17.670955 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.670950 2588 generic.go:356] "Generic (PLEG): container finished" podID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerID="86a2edc6e23f52a0e001048692d855678871c60792afa43eb0a4f69b952cefde" exitCode=137 Sep 10 13:02:17.671120 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.670994 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" event={"ID":"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05","Type":"ContainerDied","Data":"4117b49761c0a6be585532cc6efe638e1575f360481ef829da3ca2a54c4c45dc"} Sep 10 13:02:17.671120 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.671014 2588 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" Sep 10 13:02:17.671120 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.671032 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" event={"ID":"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05","Type":"ContainerDied","Data":"86a2edc6e23f52a0e001048692d855678871c60792afa43eb0a4f69b952cefde"} Sep 10 13:02:17.671120 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.671052 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b" event={"ID":"6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05","Type":"ContainerDied","Data":"e03ca3b9c629c9d208d87a749fb6850167c3a2185f2c5811df7be455ad434cb4"} Sep 10 13:02:17.678034 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.678014 2588 scope.go:117] "RemoveContainer" containerID="fa1a45b204766c86e0309adf304e01c702a8d3d225ac5ba6377d4cb85e094994" Sep 10 13:02:17.685678 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.685659 2588 scope.go:117] "RemoveContainer" containerID="558dd8384ec2d60ecbedda81f3b45f31dfe6cb5f8b3757e0d41201bf9d9a94a9" Sep 10 13:02:17.693136 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.693110 2588 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm"] Sep 10 13:02:17.693817 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.693659 2588 scope.go:117] "RemoveContainer" containerID="222df65ef7508e594444f0f6f5141734f9c81566871e194f8a536ac688652ffa" Sep 10 13:02:17.700934 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.700908 2588 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-97hwm"] Sep 10 13:02:17.709776 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.709755 2588 scope.go:117] "RemoveContainer" containerID="7e6b54889059634eb34c754420cc2dc8d8eb30e5ca2d6ca9c3ee13d8b51c700c" Sep 10 13:02:17.714132 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.714107 2588 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b"] Sep 10 13:02:17.717978 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.717954 2588 scope.go:117] "RemoveContainer" containerID="aa71f82f9b98ba65464ec98064c23cf7e3aed8c172714ef0a38468ec0d8e6959" Sep 10 13:02:17.718319 ip-10-0-135-250 kubenswrapper[2588]: E0910 13:02:17.718292 2588 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa71f82f9b98ba65464ec98064c23cf7e3aed8c172714ef0a38468ec0d8e6959\": container with ID starting with aa71f82f9b98ba65464ec98064c23cf7e3aed8c172714ef0a38468ec0d8e6959 not found: ID does not exist" containerID="aa71f82f9b98ba65464ec98064c23cf7e3aed8c172714ef0a38468ec0d8e6959" Sep 10 13:02:17.718371 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.718322 2588 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa71f82f9b98ba65464ec98064c23cf7e3aed8c172714ef0a38468ec0d8e6959"} err="failed to get container status \"aa71f82f9b98ba65464ec98064c23cf7e3aed8c172714ef0a38468ec0d8e6959\": rpc error: code = NotFound desc = could not find container \"aa71f82f9b98ba65464ec98064c23cf7e3aed8c172714ef0a38468ec0d8e6959\": container with ID starting with aa71f82f9b98ba65464ec98064c23cf7e3aed8c172714ef0a38468ec0d8e6959 not found: ID does not exist" Sep 10 13:02:17.718371 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.718341 2588 scope.go:117] "RemoveContainer" containerID="fa1a45b204766c86e0309adf304e01c702a8d3d225ac5ba6377d4cb85e094994" Sep 10 13:02:17.718632 ip-10-0-135-250 kubenswrapper[2588]: E0910 13:02:17.718609 2588 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa1a45b204766c86e0309adf304e01c702a8d3d225ac5ba6377d4cb85e094994\": container with ID starting with fa1a45b204766c86e0309adf304e01c702a8d3d225ac5ba6377d4cb85e094994 not found: ID does not exist" containerID="fa1a45b204766c86e0309adf304e01c702a8d3d225ac5ba6377d4cb85e094994" Sep 10 13:02:17.718681 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.718634 2588 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa1a45b204766c86e0309adf304e01c702a8d3d225ac5ba6377d4cb85e094994"} err="failed to get container status \"fa1a45b204766c86e0309adf304e01c702a8d3d225ac5ba6377d4cb85e094994\": rpc error: code = NotFound desc = could not find container \"fa1a45b204766c86e0309adf304e01c702a8d3d225ac5ba6377d4cb85e094994\": container with ID starting with fa1a45b204766c86e0309adf304e01c702a8d3d225ac5ba6377d4cb85e094994 not found: ID does not exist" Sep 10 13:02:17.718681 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.718649 2588 scope.go:117] "RemoveContainer" containerID="558dd8384ec2d60ecbedda81f3b45f31dfe6cb5f8b3757e0d41201bf9d9a94a9" Sep 10 13:02:17.718920 ip-10-0-135-250 kubenswrapper[2588]: E0910 13:02:17.718902 2588 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"558dd8384ec2d60ecbedda81f3b45f31dfe6cb5f8b3757e0d41201bf9d9a94a9\": container with ID starting with 558dd8384ec2d60ecbedda81f3b45f31dfe6cb5f8b3757e0d41201bf9d9a94a9 not found: ID does not exist" containerID="558dd8384ec2d60ecbedda81f3b45f31dfe6cb5f8b3757e0d41201bf9d9a94a9" Sep 10 13:02:17.718965 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.718926 2588 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"558dd8384ec2d60ecbedda81f3b45f31dfe6cb5f8b3757e0d41201bf9d9a94a9"} err="failed to get container status \"558dd8384ec2d60ecbedda81f3b45f31dfe6cb5f8b3757e0d41201bf9d9a94a9\": rpc error: code = NotFound desc = could not find container \"558dd8384ec2d60ecbedda81f3b45f31dfe6cb5f8b3757e0d41201bf9d9a94a9\": container with ID starting with 558dd8384ec2d60ecbedda81f3b45f31dfe6cb5f8b3757e0d41201bf9d9a94a9 not found: ID does not exist" Sep 10 13:02:17.718965 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.718941 2588 scope.go:117] "RemoveContainer" containerID="222df65ef7508e594444f0f6f5141734f9c81566871e194f8a536ac688652ffa" Sep 10 13:02:17.719081 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.719035 2588 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-online-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 13:02:17.719081 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.719052 2588 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-ui-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 13:02:17.719081 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.719062 2588 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtmm9\" (UniqueName: \"kubernetes.io/projected/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-kube-api-access-rtmm9\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 13:02:17.719081 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.719073 2588 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-feast-data\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 13:02:17.719227 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.719086 2588 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-registry-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 13:02:17.719227 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.719098 2588 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05-offline-tls\") on node \"ip-10-0-135-250.ec2.internal\" DevicePath \"\"" Sep 10 13:02:17.719227 ip-10-0-135-250 kubenswrapper[2588]: E0910 13:02:17.719187 2588 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"222df65ef7508e594444f0f6f5141734f9c81566871e194f8a536ac688652ffa\": container with ID starting with 222df65ef7508e594444f0f6f5141734f9c81566871e194f8a536ac688652ffa not found: ID does not exist" containerID="222df65ef7508e594444f0f6f5141734f9c81566871e194f8a536ac688652ffa" Sep 10 13:02:17.719227 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.719208 2588 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"222df65ef7508e594444f0f6f5141734f9c81566871e194f8a536ac688652ffa"} err="failed to get container status \"222df65ef7508e594444f0f6f5141734f9c81566871e194f8a536ac688652ffa\": rpc error: code = NotFound desc = could not find container \"222df65ef7508e594444f0f6f5141734f9c81566871e194f8a536ac688652ffa\": container with ID starting with 222df65ef7508e594444f0f6f5141734f9c81566871e194f8a536ac688652ffa not found: ID does not exist" Sep 10 13:02:17.719227 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.719222 2588 scope.go:117] "RemoveContainer" containerID="7e6b54889059634eb34c754420cc2dc8d8eb30e5ca2d6ca9c3ee13d8b51c700c" Sep 10 13:02:17.719440 ip-10-0-135-250 kubenswrapper[2588]: E0910 13:02:17.719424 2588 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e6b54889059634eb34c754420cc2dc8d8eb30e5ca2d6ca9c3ee13d8b51c700c\": container with ID starting with 7e6b54889059634eb34c754420cc2dc8d8eb30e5ca2d6ca9c3ee13d8b51c700c not found: ID does not exist" containerID="7e6b54889059634eb34c754420cc2dc8d8eb30e5ca2d6ca9c3ee13d8b51c700c" Sep 10 13:02:17.719478 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.719441 2588 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e6b54889059634eb34c754420cc2dc8d8eb30e5ca2d6ca9c3ee13d8b51c700c"} err="failed to get container status \"7e6b54889059634eb34c754420cc2dc8d8eb30e5ca2d6ca9c3ee13d8b51c700c\": rpc error: code = NotFound desc = could not find container \"7e6b54889059634eb34c754420cc2dc8d8eb30e5ca2d6ca9c3ee13d8b51c700c\": container with ID starting with 7e6b54889059634eb34c754420cc2dc8d8eb30e5ca2d6ca9c3ee13d8b51c700c not found: ID does not exist" Sep 10 13:02:17.719478 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.719454 2588 scope.go:117] "RemoveContainer" containerID="7fccdc2e242db81e4d8345a5f04aaba9221d0919510a119336e4f5be9e01a606" Sep 10 13:02:17.722402 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.722380 2588 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-fmn7b"] Sep 10 13:02:17.727526 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.727503 2588 scope.go:117] "RemoveContainer" containerID="4117b49761c0a6be585532cc6efe638e1575f360481ef829da3ca2a54c4c45dc" Sep 10 13:02:17.735395 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.735375 2588 scope.go:117] "RemoveContainer" containerID="d4d535d5374b43bc2d1bcc586a3f7ba1625024283cc2177de7dab01eea6ad85a" Sep 10 13:02:17.742781 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.742756 2588 scope.go:117] "RemoveContainer" containerID="86a2edc6e23f52a0e001048692d855678871c60792afa43eb0a4f69b952cefde" Sep 10 13:02:17.750563 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.750543 2588 scope.go:117] "RemoveContainer" containerID="a5824ce190e8807d04a1950f1aa0b6f566fa94d9e794059aee4b91799f42ed64" Sep 10 13:02:17.757675 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.757653 2588 scope.go:117] "RemoveContainer" containerID="597afeacdeccf4abb7977a50597358c23354f8751ace25a48c7f84a96b369395" Sep 10 13:02:17.764840 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.764821 2588 scope.go:117] "RemoveContainer" containerID="7fccdc2e242db81e4d8345a5f04aaba9221d0919510a119336e4f5be9e01a606" Sep 10 13:02:17.765131 ip-10-0-135-250 kubenswrapper[2588]: E0910 13:02:17.765110 2588 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fccdc2e242db81e4d8345a5f04aaba9221d0919510a119336e4f5be9e01a606\": container with ID starting with 7fccdc2e242db81e4d8345a5f04aaba9221d0919510a119336e4f5be9e01a606 not found: ID does not exist" containerID="7fccdc2e242db81e4d8345a5f04aaba9221d0919510a119336e4f5be9e01a606" Sep 10 13:02:17.765172 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.765143 2588 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fccdc2e242db81e4d8345a5f04aaba9221d0919510a119336e4f5be9e01a606"} err="failed to get container status \"7fccdc2e242db81e4d8345a5f04aaba9221d0919510a119336e4f5be9e01a606\": rpc error: code = NotFound desc = could not find container \"7fccdc2e242db81e4d8345a5f04aaba9221d0919510a119336e4f5be9e01a606\": container with ID starting with 7fccdc2e242db81e4d8345a5f04aaba9221d0919510a119336e4f5be9e01a606 not found: ID does not exist" Sep 10 13:02:17.765172 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.765163 2588 scope.go:117] "RemoveContainer" containerID="4117b49761c0a6be585532cc6efe638e1575f360481ef829da3ca2a54c4c45dc" Sep 10 13:02:17.765417 ip-10-0-135-250 kubenswrapper[2588]: E0910 13:02:17.765399 2588 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4117b49761c0a6be585532cc6efe638e1575f360481ef829da3ca2a54c4c45dc\": container with ID starting with 4117b49761c0a6be585532cc6efe638e1575f360481ef829da3ca2a54c4c45dc not found: ID does not exist" containerID="4117b49761c0a6be585532cc6efe638e1575f360481ef829da3ca2a54c4c45dc" Sep 10 13:02:17.765462 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.765423 2588 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4117b49761c0a6be585532cc6efe638e1575f360481ef829da3ca2a54c4c45dc"} err="failed to get container status \"4117b49761c0a6be585532cc6efe638e1575f360481ef829da3ca2a54c4c45dc\": rpc error: code = NotFound desc = could not find container \"4117b49761c0a6be585532cc6efe638e1575f360481ef829da3ca2a54c4c45dc\": container with ID starting with 4117b49761c0a6be585532cc6efe638e1575f360481ef829da3ca2a54c4c45dc not found: ID does not exist" Sep 10 13:02:17.765462 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.765438 2588 scope.go:117] "RemoveContainer" containerID="d4d535d5374b43bc2d1bcc586a3f7ba1625024283cc2177de7dab01eea6ad85a" Sep 10 13:02:17.765670 ip-10-0-135-250 kubenswrapper[2588]: E0910 13:02:17.765654 2588 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4d535d5374b43bc2d1bcc586a3f7ba1625024283cc2177de7dab01eea6ad85a\": container with ID starting with d4d535d5374b43bc2d1bcc586a3f7ba1625024283cc2177de7dab01eea6ad85a not found: ID does not exist" containerID="d4d535d5374b43bc2d1bcc586a3f7ba1625024283cc2177de7dab01eea6ad85a" Sep 10 13:02:17.765750 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.765677 2588 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4d535d5374b43bc2d1bcc586a3f7ba1625024283cc2177de7dab01eea6ad85a"} err="failed to get container status \"d4d535d5374b43bc2d1bcc586a3f7ba1625024283cc2177de7dab01eea6ad85a\": rpc error: code = NotFound desc = could not find container \"d4d535d5374b43bc2d1bcc586a3f7ba1625024283cc2177de7dab01eea6ad85a\": container with ID starting with d4d535d5374b43bc2d1bcc586a3f7ba1625024283cc2177de7dab01eea6ad85a not found: ID does not exist" Sep 10 13:02:17.765750 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.765698 2588 scope.go:117] "RemoveContainer" containerID="86a2edc6e23f52a0e001048692d855678871c60792afa43eb0a4f69b952cefde" Sep 10 13:02:17.766001 ip-10-0-135-250 kubenswrapper[2588]: E0910 13:02:17.765984 2588 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86a2edc6e23f52a0e001048692d855678871c60792afa43eb0a4f69b952cefde\": container with ID starting with 86a2edc6e23f52a0e001048692d855678871c60792afa43eb0a4f69b952cefde not found: ID does not exist" containerID="86a2edc6e23f52a0e001048692d855678871c60792afa43eb0a4f69b952cefde" Sep 10 13:02:17.766052 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.766013 2588 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86a2edc6e23f52a0e001048692d855678871c60792afa43eb0a4f69b952cefde"} err="failed to get container status \"86a2edc6e23f52a0e001048692d855678871c60792afa43eb0a4f69b952cefde\": rpc error: code = NotFound desc = could not find container \"86a2edc6e23f52a0e001048692d855678871c60792afa43eb0a4f69b952cefde\": container with ID starting with 86a2edc6e23f52a0e001048692d855678871c60792afa43eb0a4f69b952cefde not found: ID does not exist" Sep 10 13:02:17.766052 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.766029 2588 scope.go:117] "RemoveContainer" containerID="a5824ce190e8807d04a1950f1aa0b6f566fa94d9e794059aee4b91799f42ed64" Sep 10 13:02:17.766301 ip-10-0-135-250 kubenswrapper[2588]: E0910 13:02:17.766284 2588 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5824ce190e8807d04a1950f1aa0b6f566fa94d9e794059aee4b91799f42ed64\": container with ID starting with a5824ce190e8807d04a1950f1aa0b6f566fa94d9e794059aee4b91799f42ed64 not found: ID does not exist" containerID="a5824ce190e8807d04a1950f1aa0b6f566fa94d9e794059aee4b91799f42ed64" Sep 10 13:02:17.766350 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.766304 2588 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5824ce190e8807d04a1950f1aa0b6f566fa94d9e794059aee4b91799f42ed64"} err="failed to get container status \"a5824ce190e8807d04a1950f1aa0b6f566fa94d9e794059aee4b91799f42ed64\": rpc error: code = NotFound desc = could not find container \"a5824ce190e8807d04a1950f1aa0b6f566fa94d9e794059aee4b91799f42ed64\": container with ID starting with a5824ce190e8807d04a1950f1aa0b6f566fa94d9e794059aee4b91799f42ed64 not found: ID does not exist" Sep 10 13:02:17.766350 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.766318 2588 scope.go:117] "RemoveContainer" containerID="597afeacdeccf4abb7977a50597358c23354f8751ace25a48c7f84a96b369395" Sep 10 13:02:17.766554 ip-10-0-135-250 kubenswrapper[2588]: E0910 13:02:17.766537 2588 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"597afeacdeccf4abb7977a50597358c23354f8751ace25a48c7f84a96b369395\": container with ID starting with 597afeacdeccf4abb7977a50597358c23354f8751ace25a48c7f84a96b369395 not found: ID does not exist" containerID="597afeacdeccf4abb7977a50597358c23354f8751ace25a48c7f84a96b369395" Sep 10 13:02:17.766618 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.766560 2588 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"597afeacdeccf4abb7977a50597358c23354f8751ace25a48c7f84a96b369395"} err="failed to get container status \"597afeacdeccf4abb7977a50597358c23354f8751ace25a48c7f84a96b369395\": rpc error: code = NotFound desc = could not find container \"597afeacdeccf4abb7977a50597358c23354f8751ace25a48c7f84a96b369395\": container with ID starting with 597afeacdeccf4abb7977a50597358c23354f8751ace25a48c7f84a96b369395 not found: ID does not exist" Sep 10 13:02:17.766618 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.766581 2588 scope.go:117] "RemoveContainer" containerID="7fccdc2e242db81e4d8345a5f04aaba9221d0919510a119336e4f5be9e01a606" Sep 10 13:02:17.766961 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.766941 2588 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fccdc2e242db81e4d8345a5f04aaba9221d0919510a119336e4f5be9e01a606"} err="failed to get container status \"7fccdc2e242db81e4d8345a5f04aaba9221d0919510a119336e4f5be9e01a606\": rpc error: code = NotFound desc = could not find container \"7fccdc2e242db81e4d8345a5f04aaba9221d0919510a119336e4f5be9e01a606\": container with ID starting with 7fccdc2e242db81e4d8345a5f04aaba9221d0919510a119336e4f5be9e01a606 not found: ID does not exist" Sep 10 13:02:17.766961 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.766960 2588 scope.go:117] "RemoveContainer" containerID="4117b49761c0a6be585532cc6efe638e1575f360481ef829da3ca2a54c4c45dc" Sep 10 13:02:17.767201 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.767181 2588 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4117b49761c0a6be585532cc6efe638e1575f360481ef829da3ca2a54c4c45dc"} err="failed to get container status \"4117b49761c0a6be585532cc6efe638e1575f360481ef829da3ca2a54c4c45dc\": rpc error: code = NotFound desc = could not find container \"4117b49761c0a6be585532cc6efe638e1575f360481ef829da3ca2a54c4c45dc\": container with ID starting with 4117b49761c0a6be585532cc6efe638e1575f360481ef829da3ca2a54c4c45dc not found: ID does not exist" Sep 10 13:02:17.767269 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.767202 2588 scope.go:117] "RemoveContainer" containerID="d4d535d5374b43bc2d1bcc586a3f7ba1625024283cc2177de7dab01eea6ad85a" Sep 10 13:02:17.767452 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.767434 2588 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4d535d5374b43bc2d1bcc586a3f7ba1625024283cc2177de7dab01eea6ad85a"} err="failed to get container status \"d4d535d5374b43bc2d1bcc586a3f7ba1625024283cc2177de7dab01eea6ad85a\": rpc error: code = NotFound desc = could not find container \"d4d535d5374b43bc2d1bcc586a3f7ba1625024283cc2177de7dab01eea6ad85a\": container with ID starting with d4d535d5374b43bc2d1bcc586a3f7ba1625024283cc2177de7dab01eea6ad85a not found: ID does not exist" Sep 10 13:02:17.767499 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.767453 2588 scope.go:117] "RemoveContainer" containerID="86a2edc6e23f52a0e001048692d855678871c60792afa43eb0a4f69b952cefde" Sep 10 13:02:17.767663 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.767644 2588 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86a2edc6e23f52a0e001048692d855678871c60792afa43eb0a4f69b952cefde"} err="failed to get container status \"86a2edc6e23f52a0e001048692d855678871c60792afa43eb0a4f69b952cefde\": rpc error: code = NotFound desc = could not find container \"86a2edc6e23f52a0e001048692d855678871c60792afa43eb0a4f69b952cefde\": container with ID starting with 86a2edc6e23f52a0e001048692d855678871c60792afa43eb0a4f69b952cefde not found: ID does not exist" Sep 10 13:02:17.767728 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.767663 2588 scope.go:117] "RemoveContainer" containerID="a5824ce190e8807d04a1950f1aa0b6f566fa94d9e794059aee4b91799f42ed64" Sep 10 13:02:17.767880 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.767862 2588 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5824ce190e8807d04a1950f1aa0b6f566fa94d9e794059aee4b91799f42ed64"} err="failed to get container status \"a5824ce190e8807d04a1950f1aa0b6f566fa94d9e794059aee4b91799f42ed64\": rpc error: code = NotFound desc = could not find container \"a5824ce190e8807d04a1950f1aa0b6f566fa94d9e794059aee4b91799f42ed64\": container with ID starting with a5824ce190e8807d04a1950f1aa0b6f566fa94d9e794059aee4b91799f42ed64 not found: ID does not exist" Sep 10 13:02:17.767930 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.767881 2588 scope.go:117] "RemoveContainer" containerID="597afeacdeccf4abb7977a50597358c23354f8751ace25a48c7f84a96b369395" Sep 10 13:02:17.768081 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:17.768064 2588 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"597afeacdeccf4abb7977a50597358c23354f8751ace25a48c7f84a96b369395"} err="failed to get container status \"597afeacdeccf4abb7977a50597358c23354f8751ace25a48c7f84a96b369395\": rpc error: code = NotFound desc = could not find container \"597afeacdeccf4abb7977a50597358c23354f8751ace25a48c7f84a96b369395\": container with ID starting with 597afeacdeccf4abb7977a50597358c23354f8751ace25a48c7f84a96b369395 not found: ID does not exist" Sep 10 13:02:18.779343 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:18.779309 2588 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" path="/var/lib/kubelet/pods/0d982cd2-0567-49e1-acbb-c941bdabf485/volumes" Sep 10 13:02:18.779858 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:18.779844 2588 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" path="/var/lib/kubelet/pods/6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05/volumes" Sep 10 13:02:54.461094 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461016 2588 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nd6nf/must-gather-n8472"] Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461255 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="ui" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461266 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="ui" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461275 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="feast-apply" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461280 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="feast-apply" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461286 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="online" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461291 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="online" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461298 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="registry" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461304 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="registry" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461310 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="online" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461316 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="online" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461322 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="offline" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461327 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="offline" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461333 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="feast-init" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461338 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="feast-init" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461347 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="feast-apply" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461352 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="feast-apply" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461360 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="offline" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461365 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="offline" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461375 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="ui" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461380 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="ui" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461385 2588 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="feast-init" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461390 2588 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="feast-init" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461425 2588 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="offline" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461432 2588 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="ui" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461438 2588 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="registry" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461444 2588 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="ui" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461449 2588 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="online" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461454 2588 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d982cd2-0567-49e1-acbb-c941bdabf485" containerName="offline" Sep 10 13:02:54.461533 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.461460 2588 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ef6e76b-dcd0-49dd-8afe-a2e6911d2d05" containerName="online" Sep 10 13:02:54.464336 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.464320 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nd6nf/must-gather-n8472" Sep 10 13:02:54.471185 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.471164 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nd6nf\"/\"default-dockercfg-9f665\"" Sep 10 13:02:54.471322 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.471304 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nd6nf\"/\"kube-root-ca.crt\"" Sep 10 13:02:54.472216 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.472200 2588 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nd6nf\"/\"openshift-service-ca.crt\"" Sep 10 13:02:54.486506 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.486480 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nd6nf/must-gather-n8472"] Sep 10 13:02:54.495614 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.495592 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txwfn\" (UniqueName: \"kubernetes.io/projected/cdcc2859-31de-46bd-ae9e-4ce124c23c65-kube-api-access-txwfn\") pod \"must-gather-n8472\" (UID: \"cdcc2859-31de-46bd-ae9e-4ce124c23c65\") " pod="openshift-must-gather-nd6nf/must-gather-n8472" Sep 10 13:02:54.495703 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.495631 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cdcc2859-31de-46bd-ae9e-4ce124c23c65-must-gather-output\") pod \"must-gather-n8472\" (UID: \"cdcc2859-31de-46bd-ae9e-4ce124c23c65\") " pod="openshift-must-gather-nd6nf/must-gather-n8472" Sep 10 13:02:54.596312 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.596274 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txwfn\" (UniqueName: \"kubernetes.io/projected/cdcc2859-31de-46bd-ae9e-4ce124c23c65-kube-api-access-txwfn\") pod \"must-gather-n8472\" (UID: \"cdcc2859-31de-46bd-ae9e-4ce124c23c65\") " pod="openshift-must-gather-nd6nf/must-gather-n8472" Sep 10 13:02:54.596312 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.596319 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cdcc2859-31de-46bd-ae9e-4ce124c23c65-must-gather-output\") pod \"must-gather-n8472\" (UID: \"cdcc2859-31de-46bd-ae9e-4ce124c23c65\") " pod="openshift-must-gather-nd6nf/must-gather-n8472" Sep 10 13:02:54.596613 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.596599 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cdcc2859-31de-46bd-ae9e-4ce124c23c65-must-gather-output\") pod \"must-gather-n8472\" (UID: \"cdcc2859-31de-46bd-ae9e-4ce124c23c65\") " pod="openshift-must-gather-nd6nf/must-gather-n8472" Sep 10 13:02:54.605377 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.605353 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txwfn\" (UniqueName: \"kubernetes.io/projected/cdcc2859-31de-46bd-ae9e-4ce124c23c65-kube-api-access-txwfn\") pod \"must-gather-n8472\" (UID: \"cdcc2859-31de-46bd-ae9e-4ce124c23c65\") " pod="openshift-must-gather-nd6nf/must-gather-n8472" Sep 10 13:02:54.775421 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.775394 2588 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nd6nf\"/\"default-dockercfg-9f665\"" Sep 10 13:02:54.782822 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:54.782801 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nd6nf/must-gather-n8472" Sep 10 13:02:55.059606 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:55.059526 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nd6nf/must-gather-n8472"] Sep 10 13:02:55.062874 ip-10-0-135-250 kubenswrapper[2588]: W0910 13:02:55.062847 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcdcc2859_31de_46bd_ae9e_4ce124c23c65.slice/crio-6d6bcb616efc7edb48502daf104dbc8096dc0df9057bcd001130d1d1141c534f WatchSource:0}: Error finding container 6d6bcb616efc7edb48502daf104dbc8096dc0df9057bcd001130d1d1141c534f: Status 404 returned error can't find the container with id 6d6bcb616efc7edb48502daf104dbc8096dc0df9057bcd001130d1d1141c534f Sep 10 13:02:55.064719 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:55.064703 2588 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 13:02:55.798867 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:55.798692 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nd6nf/must-gather-n8472" event={"ID":"cdcc2859-31de-46bd-ae9e-4ce124c23c65","Type":"ContainerStarted","Data":"6d6bcb616efc7edb48502daf104dbc8096dc0df9057bcd001130d1d1141c534f"} Sep 10 13:02:56.804310 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:56.804265 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nd6nf/must-gather-n8472" event={"ID":"cdcc2859-31de-46bd-ae9e-4ce124c23c65","Type":"ContainerStarted","Data":"763c1f2a087e0bc2dbc92308f4cc5b17f72979cdc1067dd11d51f117287c425b"} Sep 10 13:02:56.804310 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:56.804314 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nd6nf/must-gather-n8472" event={"ID":"cdcc2859-31de-46bd-ae9e-4ce124c23c65","Type":"ContainerStarted","Data":"c03c22219945daa4935ecea8615a079f975763a64ff552f4795e283038d4f6db"} Sep 10 13:02:57.588328 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:57.588298 2588 ???:1] "http: TLS handshake error from 10.0.135.250:37946: EOF" Sep 10 13:02:57.588521 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:57.588405 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wclkd_cb45e2aa-d5a7-47e9-828e-186e39ffd6b8/global-pull-secret-syncer/0.log" Sep 10 13:02:57.714583 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:57.714532 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-km6jk_6b43b97f-6877-402b-87be-390b1a42c4f8/konnectivity-agent/0.log" Sep 10 13:02:57.782901 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:02:57.782862 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-250.ec2.internal_928ccfc5fa7f66e685a010324e61a3c7/haproxy/0.log" Sep 10 13:03:02.682385 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:02.682346 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l89fj_4c803e13-07a0-4250-8f81-80ac7cdc47bd/node-exporter/0.log" Sep 10 13:03:02.723310 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:02.723272 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l89fj_4c803e13-07a0-4250-8f81-80ac7cdc47bd/kube-rbac-proxy/0.log" Sep 10 13:03:02.774056 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:02.773960 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l89fj_4c803e13-07a0-4250-8f81-80ac7cdc47bd/init-textfile/0.log" Sep 10 13:03:04.844395 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:04.844335 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nd6nf/must-gather-n8472" podStartSLOduration=10.050682127 podStartE2EDuration="10.844312575s" totalImagesPullingTime="793.630448ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:02:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:02:56.829795611 +0000 UTC m=+1982.563390327" watchObservedRunningTime="2026-09-10 13:03:04.844312575 +0000 UTC m=+1990.577907293" Sep 10 13:03:04.845857 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:04.845830 2588 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6"] Sep 10 13:03:04.850664 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:04.850647 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6" Sep 10 13:03:04.871145 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:04.871114 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6"] Sep 10 13:03:04.872859 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:04.872831 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c031c809-4245-47e6-acc5-d3585200e399-proc\") pod \"perf-node-gather-daemonset-7dmw6\" (UID: \"c031c809-4245-47e6-acc5-d3585200e399\") " pod="openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6" Sep 10 13:03:04.872993 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:04.872871 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c031c809-4245-47e6-acc5-d3585200e399-podres\") pod \"perf-node-gather-daemonset-7dmw6\" (UID: \"c031c809-4245-47e6-acc5-d3585200e399\") " pod="openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6" Sep 10 13:03:04.872993 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:04.872932 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c031c809-4245-47e6-acc5-d3585200e399-lib-modules\") pod \"perf-node-gather-daemonset-7dmw6\" (UID: \"c031c809-4245-47e6-acc5-d3585200e399\") " pod="openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6" Sep 10 13:03:04.872993 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:04.872962 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c031c809-4245-47e6-acc5-d3585200e399-sys\") pod \"perf-node-gather-daemonset-7dmw6\" (UID: \"c031c809-4245-47e6-acc5-d3585200e399\") " pod="openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6" Sep 10 13:03:04.872993 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:04.872987 2588 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2l6x\" (UniqueName: \"kubernetes.io/projected/c031c809-4245-47e6-acc5-d3585200e399-kube-api-access-z2l6x\") pod \"perf-node-gather-daemonset-7dmw6\" (UID: \"c031c809-4245-47e6-acc5-d3585200e399\") " pod="openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6" Sep 10 13:03:04.974215 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:04.974174 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c031c809-4245-47e6-acc5-d3585200e399-proc\") pod \"perf-node-gather-daemonset-7dmw6\" (UID: \"c031c809-4245-47e6-acc5-d3585200e399\") " pod="openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6" Sep 10 13:03:04.974215 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:04.974213 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c031c809-4245-47e6-acc5-d3585200e399-podres\") pod \"perf-node-gather-daemonset-7dmw6\" (UID: \"c031c809-4245-47e6-acc5-d3585200e399\") " pod="openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6" Sep 10 13:03:04.974453 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:04.974314 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c031c809-4245-47e6-acc5-d3585200e399-proc\") pod \"perf-node-gather-daemonset-7dmw6\" (UID: \"c031c809-4245-47e6-acc5-d3585200e399\") " pod="openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6" Sep 10 13:03:04.974453 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:04.974325 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c031c809-4245-47e6-acc5-d3585200e399-podres\") pod \"perf-node-gather-daemonset-7dmw6\" (UID: \"c031c809-4245-47e6-acc5-d3585200e399\") " pod="openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6" Sep 10 13:03:04.974453 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:04.974339 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c031c809-4245-47e6-acc5-d3585200e399-lib-modules\") pod \"perf-node-gather-daemonset-7dmw6\" (UID: \"c031c809-4245-47e6-acc5-d3585200e399\") " pod="openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6" Sep 10 13:03:04.974453 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:04.974375 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c031c809-4245-47e6-acc5-d3585200e399-sys\") pod \"perf-node-gather-daemonset-7dmw6\" (UID: \"c031c809-4245-47e6-acc5-d3585200e399\") " pod="openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6" Sep 10 13:03:04.974453 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:04.974409 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c031c809-4245-47e6-acc5-d3585200e399-lib-modules\") pod \"perf-node-gather-daemonset-7dmw6\" (UID: \"c031c809-4245-47e6-acc5-d3585200e399\") " pod="openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6" Sep 10 13:03:04.974453 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:04.974420 2588 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2l6x\" (UniqueName: \"kubernetes.io/projected/c031c809-4245-47e6-acc5-d3585200e399-kube-api-access-z2l6x\") pod \"perf-node-gather-daemonset-7dmw6\" (UID: \"c031c809-4245-47e6-acc5-d3585200e399\") " pod="openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6" Sep 10 13:03:04.974453 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:04.974430 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c031c809-4245-47e6-acc5-d3585200e399-sys\") pod \"perf-node-gather-daemonset-7dmw6\" (UID: \"c031c809-4245-47e6-acc5-d3585200e399\") " pod="openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6" Sep 10 13:03:04.988189 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:04.988126 2588 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2l6x\" (UniqueName: \"kubernetes.io/projected/c031c809-4245-47e6-acc5-d3585200e399-kube-api-access-z2l6x\") pod \"perf-node-gather-daemonset-7dmw6\" (UID: \"c031c809-4245-47e6-acc5-d3585200e399\") " pod="openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6" Sep 10 13:03:05.160918 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:05.160817 2588 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6" Sep 10 13:03:05.451829 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:05.451800 2588 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6"] Sep 10 13:03:05.452519 ip-10-0-135-250 kubenswrapper[2588]: W0910 13:03:05.452486 2588 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc031c809_4245_47e6_acc5_d3585200e399.slice/crio-6a3dd0baf47d90e923146b0082fecaf2cad6bd2cfee9685d9b779ea5b2673eed WatchSource:0}: Error finding container 6a3dd0baf47d90e923146b0082fecaf2cad6bd2cfee9685d9b779ea5b2673eed: Status 404 returned error can't find the container with id 6a3dd0baf47d90e923146b0082fecaf2cad6bd2cfee9685d9b779ea5b2673eed Sep 10 13:03:05.843373 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:05.843332 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6" event={"ID":"c031c809-4245-47e6-acc5-d3585200e399","Type":"ContainerStarted","Data":"67c093e8fc093614a5cb0daf14a6e5d5ba8d7dc78567ceb46fbbc269ffbccdd3"} Sep 10 13:03:05.843569 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:05.843390 2588 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6" event={"ID":"c031c809-4245-47e6-acc5-d3585200e399","Type":"ContainerStarted","Data":"6a3dd0baf47d90e923146b0082fecaf2cad6bd2cfee9685d9b779ea5b2673eed"} Sep 10 13:03:05.843569 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:05.843410 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6" Sep 10 13:03:05.868713 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:05.868652 2588 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6" podStartSLOduration=1.868632103 podStartE2EDuration="1.868632103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:03:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:03:05.867022223 +0000 UTC m=+1991.600616940" watchObservedRunningTime="2026-09-10 13:03:05.868632103 +0000 UTC m=+1991.602226821" Sep 10 13:03:07.107432 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:07.107404 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f75jz_f3aadb41-77bc-4679-9ea3-0b47e499c6e4/dns/0.log" Sep 10 13:03:07.138062 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:07.138025 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f75jz_f3aadb41-77bc-4679-9ea3-0b47e499c6e4/kube-rbac-proxy/0.log" Sep 10 13:03:07.302561 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:07.302522 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rwdsg_2d9a758b-9a83-405b-9b01-7143273959ff/dns-node-resolver/0.log" Sep 10 13:03:07.809701 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:07.809669 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kmxmc_1019e2de-09cf-419b-a4ce-7e09921a4e04/node-ca/0.log" Sep 10 13:03:09.051715 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:09.051685 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b5xwv_09a8b6b8-6084-4098-804d-260e69715516/serve-healthcheck-canary/0.log" Sep 10 13:03:09.650321 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:09.650293 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wqq6h_ecc55eac-e95e-4d11-9436-f465ac9c84b3/kube-rbac-proxy/0.log" Sep 10 13:03:09.682880 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:09.682854 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wqq6h_ecc55eac-e95e-4d11-9436-f465ac9c84b3/exporter/0.log" Sep 10 13:03:09.710701 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:09.710673 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wqq6h_ecc55eac-e95e-4d11-9436-f465ac9c84b3/extractor/0.log" Sep 10 13:03:11.857123 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:11.857094 2588 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nd6nf/perf-node-gather-daemonset-7dmw6" Sep 10 13:03:15.989408 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:15.989325 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-478c6_334e2bed-0266-4d0f-964f-96dcb9843a25/kube-multus-additional-cni-plugins/0.log" Sep 10 13:03:16.026173 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:16.026117 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-478c6_334e2bed-0266-4d0f-964f-96dcb9843a25/egress-router-binary-copy/0.log" Sep 10 13:03:16.051367 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:16.051342 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-478c6_334e2bed-0266-4d0f-964f-96dcb9843a25/cni-plugins/0.log" Sep 10 13:03:16.077833 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:16.077800 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-478c6_334e2bed-0266-4d0f-964f-96dcb9843a25/bond-cni-plugin/0.log" Sep 10 13:03:16.104881 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:16.104846 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-478c6_334e2bed-0266-4d0f-964f-96dcb9843a25/routeoverride-cni/0.log" Sep 10 13:03:16.135551 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:16.135528 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-478c6_334e2bed-0266-4d0f-964f-96dcb9843a25/whereabouts-cni-bincopy/0.log" Sep 10 13:03:16.177141 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:16.177107 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-478c6_334e2bed-0266-4d0f-964f-96dcb9843a25/whereabouts-cni/0.log" Sep 10 13:03:16.655422 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:16.655346 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-tc789_76bf8bce-b7c3-4617-87e4-0be0765a14d0/kube-multus/0.log" Sep 10 13:03:16.731182 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:16.731154 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cmbbq_90ce254f-e418-4e45-ab41-427cd3a1626e/network-metrics-daemon/0.log" Sep 10 13:03:16.753856 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:16.753830 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cmbbq_90ce254f-e418-4e45-ab41-427cd3a1626e/kube-rbac-proxy/0.log" Sep 10 13:03:18.029952 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:18.029917 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b47ts_25672a2d-d6e3-47aa-880d-0751856a6b47/ovn-controller/0.log" Sep 10 13:03:18.058257 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:18.058231 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b47ts_25672a2d-d6e3-47aa-880d-0751856a6b47/ovn-acl-logging/0.log" Sep 10 13:03:18.087509 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:18.087467 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b47ts_25672a2d-d6e3-47aa-880d-0751856a6b47/kube-rbac-proxy-node/0.log" Sep 10 13:03:18.146298 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:18.146272 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b47ts_25672a2d-d6e3-47aa-880d-0751856a6b47/kube-rbac-proxy-ovn-metrics/0.log" Sep 10 13:03:18.178465 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:18.178435 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b47ts_25672a2d-d6e3-47aa-880d-0751856a6b47/northd/0.log" Sep 10 13:03:18.209327 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:18.209295 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b47ts_25672a2d-d6e3-47aa-880d-0751856a6b47/nbdb/0.log" Sep 10 13:03:18.254405 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:18.254378 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b47ts_25672a2d-d6e3-47aa-880d-0751856a6b47/sbdb/0.log" Sep 10 13:03:18.300577 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:18.300506 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b47ts_25672a2d-d6e3-47aa-880d-0751856a6b47/ovnkube-controller/0.log" Sep 10 13:03:19.600311 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:19.600286 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-685456655f-bvfvx_a0e08fb9-ba6f-4d67-8335-aeef23ffe1d6/check-endpoints/0.log" Sep 10 13:03:19.668631 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:19.668601 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-p5wcp_bdfce9e5-4daa-4b23-8f66-62549603ef7f/network-check-target-container/0.log" Sep 10 13:03:20.880787 ip-10-0-135-250 kubenswrapper[2588]: I0910 13:03:20.880761 2588 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-bjnv5_df440146-c599-4722-b96d-986b68d93e40/iptables-alerter/0.log"