Sep 07 10:06:04.083194 ip-10-0-140-225 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 07 10:06:04.083209 ip-10-0-140-225 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 07 10:06:04.083219 ip-10-0-140-225 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 07 10:06:04.083537 ip-10-0-140-225 systemd[1]: Failed to start Kubernetes Kubelet. Sep 07 10:06:14.119729 ip-10-0-140-225 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 07 10:06:14.119749 ip-10-0-140-225 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot e69eb4f2d2224de9a398c416cc8ad7cb -- Sep 07 10:08:02.908536 ip-10-0-140-225 systemd[1]: Starting Kubernetes Kubelet... Sep 07 10:08:03.332471 ip-10-0-140-225 kubenswrapper[2552]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 07 10:08:03.332471 ip-10-0-140-225 kubenswrapper[2552]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 07 10:08:03.332471 ip-10-0-140-225 kubenswrapper[2552]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 07 10:08:03.332471 ip-10-0-140-225 kubenswrapper[2552]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 07 10:08:03.332471 ip-10-0-140-225 kubenswrapper[2552]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 07 10:08:03.333108 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.332483 2552 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 07 10:08:03.339049 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339027 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 10:08:03.339049 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339046 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 10:08:03.339116 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339051 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 10:08:03.339116 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339056 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 10:08:03.339116 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339060 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 10:08:03.339116 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339063 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 10:08:03.339116 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339067 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 10:08:03.339116 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339069 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 10:08:03.339116 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339073 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 10:08:03.339116 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339082 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 10:08:03.339116 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339086 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 10:08:03.339116 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339089 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 10:08:03.339116 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339091 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 10:08:03.339116 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339094 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 10:08:03.339116 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339097 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 10:08:03.339116 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339099 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 10:08:03.339116 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339102 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 10:08:03.339116 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339104 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 10:08:03.339116 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339107 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 10:08:03.339116 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339110 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 10:08:03.339116 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339112 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 10:08:03.339616 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339115 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 10:08:03.339616 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339118 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 10:08:03.339616 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339121 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 10:08:03.339616 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339124 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 10:08:03.339616 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339127 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 10:08:03.339616 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339129 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 10:08:03.339616 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339132 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 10:08:03.339616 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339135 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 10:08:03.339616 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339137 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 10:08:03.339616 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339140 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 10:08:03.339616 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339142 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 10:08:03.339616 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339145 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 10:08:03.339616 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339148 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 10:08:03.339616 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339151 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 10:08:03.339616 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339154 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 10:08:03.339616 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339156 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 10:08:03.339616 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339159 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 10:08:03.339616 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339162 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 10:08:03.339616 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339165 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 10:08:03.339616 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339167 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 10:08:03.340098 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339169 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 10:08:03.340098 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339172 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 10:08:03.340098 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339174 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 10:08:03.340098 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339176 2552 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 10:08:03.340098 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339179 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 10:08:03.340098 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339181 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 10:08:03.340098 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339184 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 10:08:03.340098 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339186 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 10:08:03.340098 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339190 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 10:08:03.340098 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339192 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 10:08:03.340098 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339194 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 10:08:03.340098 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339197 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 10:08:03.340098 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339199 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 10:08:03.340098 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339202 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 10:08:03.340098 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339206 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 10:08:03.340098 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339208 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 10:08:03.340098 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339211 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 10:08:03.340098 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339213 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 10:08:03.340098 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339216 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 10:08:03.340098 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339218 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 10:08:03.340601 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339221 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 10:08:03.340601 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339223 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 10:08:03.340601 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339226 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 10:08:03.340601 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339228 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 10:08:03.340601 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339231 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 10:08:03.340601 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339234 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 10:08:03.340601 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339236 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 10:08:03.340601 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339249 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 10:08:03.340601 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339252 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 10:08:03.340601 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339255 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 10:08:03.340601 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339259 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 10:08:03.340601 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339269 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 10:08:03.340601 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339273 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 10:08:03.340601 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339275 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 10:08:03.340601 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339278 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 10:08:03.340601 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339280 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 10:08:03.340601 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339283 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 10:08:03.340601 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339286 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 10:08:03.340601 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339289 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 10:08:03.340601 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339292 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 10:08:03.341089 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339294 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 10:08:03.341089 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339297 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 10:08:03.341089 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339300 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 10:08:03.341089 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339302 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 10:08:03.341089 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339305 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 10:08:03.341089 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339308 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 10:08:03.341089 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339310 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 10:08:03.341089 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339313 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 10:08:03.341089 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339315 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 10:08:03.341089 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339318 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 10:08:03.341089 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339321 2552 feature_gate.go:328] unrecognized feature gate: Example Sep 07 10:08:03.341089 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339323 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 10:08:03.341089 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339326 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 10:08:03.341089 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339329 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 10:08:03.341089 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339332 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 10:08:03.341089 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339334 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 10:08:03.341089 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339337 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 10:08:03.341089 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.339339 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 10:08:03.341089 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340028 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 10:08:03.341565 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340035 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 10:08:03.341565 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340038 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 10:08:03.341565 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340040 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 10:08:03.341565 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340043 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 10:08:03.341565 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340046 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 10:08:03.341565 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340049 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 10:08:03.341565 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340052 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 10:08:03.341565 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340054 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 10:08:03.341565 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340057 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 10:08:03.341565 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340060 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 10:08:03.341565 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340062 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 10:08:03.341565 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340065 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 10:08:03.341565 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340068 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 10:08:03.341565 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340070 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 10:08:03.341565 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340073 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 10:08:03.341565 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340075 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 10:08:03.341565 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340078 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 10:08:03.341565 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340080 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 10:08:03.341565 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340083 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 10:08:03.341565 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340085 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 10:08:03.342094 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340088 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 10:08:03.342094 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340091 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 10:08:03.342094 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340094 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 10:08:03.342094 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340096 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 10:08:03.342094 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340099 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 10:08:03.342094 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340102 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 10:08:03.342094 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340104 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 10:08:03.342094 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340107 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 10:08:03.342094 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340109 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 10:08:03.342094 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340112 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 10:08:03.342094 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340116 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 10:08:03.342094 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340119 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 10:08:03.342094 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340123 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 10:08:03.342094 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340127 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 10:08:03.342094 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340130 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 10:08:03.342094 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340132 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 10:08:03.342094 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340135 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 10:08:03.342094 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340139 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 10:08:03.342094 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340142 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 10:08:03.342579 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340145 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 10:08:03.342579 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340147 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 10:08:03.342579 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340151 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 10:08:03.342579 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340154 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 10:08:03.342579 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340157 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 10:08:03.342579 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340161 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 10:08:03.342579 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340164 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 10:08:03.342579 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340167 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 10:08:03.342579 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340169 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 10:08:03.342579 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340172 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 10:08:03.342579 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340175 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 10:08:03.342579 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340178 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 10:08:03.342579 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340181 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 10:08:03.342579 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340184 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 10:08:03.342579 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340187 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 10:08:03.342579 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340189 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 10:08:03.342579 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340193 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 10:08:03.342579 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340197 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 10:08:03.342579 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340200 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 10:08:03.343051 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340203 2552 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 10:08:03.343051 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340205 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 10:08:03.343051 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340208 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 10:08:03.343051 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340211 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 10:08:03.343051 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340214 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 10:08:03.343051 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340216 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 10:08:03.343051 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340219 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 10:08:03.343051 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340221 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 10:08:03.343051 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340223 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 10:08:03.343051 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340226 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 10:08:03.343051 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340229 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 10:08:03.343051 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340231 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 10:08:03.343051 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340234 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 10:08:03.343051 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340237 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 10:08:03.343051 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340255 2552 feature_gate.go:328] unrecognized feature gate: Example Sep 07 10:08:03.343051 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340258 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 10:08:03.343051 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340261 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 10:08:03.343051 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340264 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 10:08:03.343051 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340266 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 10:08:03.343051 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340269 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 10:08:03.343562 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340272 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 10:08:03.343562 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340274 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 10:08:03.343562 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340277 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 10:08:03.343562 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340280 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 10:08:03.343562 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340282 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 10:08:03.343562 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340285 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 10:08:03.343562 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340287 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 10:08:03.343562 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340291 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 10:08:03.343562 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340294 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 10:08:03.343562 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340296 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 10:08:03.343562 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340299 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 10:08:03.343562 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340301 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 10:08:03.343562 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340304 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 10:08:03.343562 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340307 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 10:08:03.343562 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340309 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 10:08:03.343562 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340312 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 10:08:03.343562 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340315 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 10:08:03.343562 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340317 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 10:08:03.343562 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340320 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 10:08:03.343562 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340322 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 10:08:03.344047 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340713 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 10:08:03.344047 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340721 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 10:08:03.344047 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340725 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 10:08:03.344047 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340728 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 10:08:03.344047 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340731 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 10:08:03.344047 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340734 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 10:08:03.344047 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340737 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 10:08:03.344047 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340740 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 10:08:03.344047 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340743 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 10:08:03.344047 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340745 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 10:08:03.344047 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340748 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 10:08:03.344047 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340751 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 10:08:03.344047 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340754 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 10:08:03.344047 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340756 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 10:08:03.344047 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340759 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 10:08:03.344047 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340762 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 10:08:03.344047 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340765 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 10:08:03.344047 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340768 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 10:08:03.344047 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340770 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 10:08:03.344047 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340773 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 10:08:03.344549 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340775 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 10:08:03.344549 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340778 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 10:08:03.344549 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340781 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 10:08:03.344549 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340783 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 10:08:03.344549 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340786 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 10:08:03.344549 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340788 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 10:08:03.344549 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340790 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 10:08:03.344549 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340793 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 10:08:03.344549 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340795 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 10:08:03.344549 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340798 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 10:08:03.344549 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340801 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 10:08:03.344549 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340803 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 10:08:03.344549 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340806 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 10:08:03.344549 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340810 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 10:08:03.344549 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340812 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 10:08:03.344549 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340815 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 10:08:03.344549 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340817 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 10:08:03.344549 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340820 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 10:08:03.344549 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340823 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 10:08:03.345009 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340826 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 10:08:03.345009 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340828 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 10:08:03.345009 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340831 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 10:08:03.345009 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340833 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 10:08:03.345009 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340836 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 10:08:03.345009 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340838 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 10:08:03.345009 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340841 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 10:08:03.345009 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340843 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 10:08:03.345009 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340846 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 10:08:03.345009 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340849 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 10:08:03.345009 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340853 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 10:08:03.345009 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340857 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 10:08:03.345009 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340860 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 10:08:03.345009 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340862 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 10:08:03.345009 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340865 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 10:08:03.345009 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340868 2552 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 10:08:03.345009 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340873 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 10:08:03.345009 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340876 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 10:08:03.345009 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340879 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 10:08:03.345494 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340882 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 10:08:03.345494 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340885 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 10:08:03.345494 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340887 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 10:08:03.345494 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340890 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 10:08:03.345494 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340892 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 10:08:03.345494 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340895 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 10:08:03.345494 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340897 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 10:08:03.345494 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340901 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 10:08:03.345494 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340904 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 10:08:03.345494 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340906 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 10:08:03.345494 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340909 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 10:08:03.345494 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340912 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 10:08:03.345494 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340917 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 10:08:03.345494 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340920 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 10:08:03.345494 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340922 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 10:08:03.345494 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340925 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 10:08:03.345494 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340927 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 10:08:03.345494 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340930 2552 feature_gate.go:328] unrecognized feature gate: Example Sep 07 10:08:03.345494 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340932 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 10:08:03.345494 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340935 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 10:08:03.345975 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340939 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 10:08:03.345975 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340942 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 10:08:03.345975 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340945 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 10:08:03.345975 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340947 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 10:08:03.345975 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340950 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 10:08:03.345975 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340952 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 10:08:03.345975 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340955 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 10:08:03.345975 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340957 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 10:08:03.345975 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340960 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 10:08:03.345975 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340962 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 10:08:03.345975 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340965 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 10:08:03.345975 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340967 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 10:08:03.345975 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340970 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 10:08:03.345975 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340972 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 10:08:03.345975 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340974 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 10:08:03.345975 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340977 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 10:08:03.345975 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340979 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 10:08:03.345975 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340982 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 10:08:03.345975 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340984 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 10:08:03.346443 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340988 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 10:08:03.346443 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.340990 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 10:08:03.346443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341071 2552 flags.go:64] FLAG: --address="0.0.0.0" Sep 07 10:08:03.346443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341103 2552 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 07 10:08:03.346443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341109 2552 flags.go:64] FLAG: --anonymous-auth="true" Sep 07 10:08:03.346443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341113 2552 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 07 10:08:03.346443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341118 2552 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 07 10:08:03.346443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341122 2552 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 07 10:08:03.346443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341126 2552 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 07 10:08:03.346443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341130 2552 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 07 10:08:03.346443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341133 2552 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 07 10:08:03.346443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341136 2552 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 07 10:08:03.346443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341139 2552 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 07 10:08:03.346443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341142 2552 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 07 10:08:03.346443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341148 2552 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 07 10:08:03.346443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341151 2552 flags.go:64] FLAG: --cgroup-root="" Sep 07 10:08:03.346443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341154 2552 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 07 10:08:03.346443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341157 2552 flags.go:64] FLAG: --client-ca-file="" Sep 07 10:08:03.346443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341160 2552 flags.go:64] FLAG: --cloud-provider="external" Sep 07 10:08:03.346443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341163 2552 flags.go:64] FLAG: --cluster-dns="[]" Sep 07 10:08:03.346443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341167 2552 flags.go:64] FLAG: --cluster-domain="" Sep 07 10:08:03.346443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341170 2552 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 07 10:08:03.346443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341173 2552 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 07 10:08:03.346443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341176 2552 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341179 2552 flags.go:64] FLAG: --container-log-max-files="5" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341183 2552 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341186 2552 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341189 2552 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341193 2552 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341196 2552 flags.go:64] FLAG: --contention-profiling="false" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341199 2552 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341202 2552 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341207 2552 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341210 2552 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341214 2552 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341217 2552 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341220 2552 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341222 2552 flags.go:64] FLAG: --enable-load-reader="false" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341225 2552 flags.go:64] FLAG: --enable-server="true" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341229 2552 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341233 2552 flags.go:64] FLAG: --event-burst="100" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341236 2552 flags.go:64] FLAG: --event-qps="50" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341254 2552 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341258 2552 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341261 2552 flags.go:64] FLAG: --eviction-hard="" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341264 2552 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341269 2552 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341272 2552 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 07 10:08:03.347058 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341275 2552 flags.go:64] FLAG: --eviction-soft="" Sep 07 10:08:03.347689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341277 2552 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 07 10:08:03.347689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341280 2552 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 07 10:08:03.347689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341283 2552 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 07 10:08:03.347689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341287 2552 flags.go:64] FLAG: --experimental-mounter-path="" Sep 07 10:08:03.347689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341289 2552 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 07 10:08:03.347689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341292 2552 flags.go:64] FLAG: --fail-swap-on="true" Sep 07 10:08:03.347689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341295 2552 flags.go:64] FLAG: --feature-gates="" Sep 07 10:08:03.347689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341299 2552 flags.go:64] FLAG: --file-check-frequency="20s" Sep 07 10:08:03.347689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341302 2552 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 07 10:08:03.347689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341305 2552 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 07 10:08:03.347689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341308 2552 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 07 10:08:03.347689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341311 2552 flags.go:64] FLAG: --healthz-port="10248" Sep 07 10:08:03.347689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341314 2552 flags.go:64] FLAG: --help="false" Sep 07 10:08:03.347689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341317 2552 flags.go:64] FLAG: --hostname-override="ip-10-0-140-225.ec2.internal" Sep 07 10:08:03.347689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341320 2552 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 07 10:08:03.347689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341325 2552 flags.go:64] FLAG: --http-check-frequency="20s" Sep 07 10:08:03.347689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341327 2552 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 07 10:08:03.347689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341331 2552 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 07 10:08:03.347689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341334 2552 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 07 10:08:03.347689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341337 2552 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 07 10:08:03.347689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341340 2552 flags.go:64] FLAG: --image-service-endpoint="" Sep 07 10:08:03.347689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341342 2552 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 07 10:08:03.347689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341345 2552 flags.go:64] FLAG: --kube-api-burst="100" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341352 2552 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341355 2552 flags.go:64] FLAG: --kube-api-qps="50" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341358 2552 flags.go:64] FLAG: --kube-reserved="" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341361 2552 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341364 2552 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341367 2552 flags.go:64] FLAG: --kubelet-cgroups="" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341371 2552 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341373 2552 flags.go:64] FLAG: --lock-file="" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341376 2552 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341379 2552 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341382 2552 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341387 2552 flags.go:64] FLAG: --log-json-split-stream="false" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341390 2552 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341393 2552 flags.go:64] FLAG: --log-text-split-stream="false" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341395 2552 flags.go:64] FLAG: --logging-format="text" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341398 2552 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341402 2552 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341404 2552 flags.go:64] FLAG: --manifest-url="" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341407 2552 flags.go:64] FLAG: --manifest-url-header="" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341412 2552 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341415 2552 flags.go:64] FLAG: --max-open-files="1000000" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341420 2552 flags.go:64] FLAG: --max-pods="110" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341423 2552 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341426 2552 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 07 10:08:03.348291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341430 2552 flags.go:64] FLAG: --memory-manager-policy="None" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341433 2552 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341436 2552 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341439 2552 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341442 2552 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341451 2552 flags.go:64] FLAG: --node-status-max-images="50" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341453 2552 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341457 2552 flags.go:64] FLAG: --oom-score-adj="-999" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341459 2552 flags.go:64] FLAG: --pod-cidr="" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341464 2552 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341469 2552 flags.go:64] FLAG: --pod-manifest-path="" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341472 2552 flags.go:64] FLAG: --pod-max-pids="-1" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341475 2552 flags.go:64] FLAG: --pods-per-core="0" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341478 2552 flags.go:64] FLAG: --port="10250" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341481 2552 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341485 2552 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-086749011d3b3ee56" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341489 2552 flags.go:64] FLAG: --qos-reserved="" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341492 2552 flags.go:64] FLAG: --read-only-port="10255" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341495 2552 flags.go:64] FLAG: --register-node="true" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341497 2552 flags.go:64] FLAG: --register-with-taints="" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341501 2552 flags.go:64] FLAG: --registry-burst="10" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341504 2552 flags.go:64] FLAG: --registry-qps="5" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341507 2552 flags.go:64] FLAG: --reserved-cpus="" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341510 2552 flags.go:64] FLAG: --reserved-memory="" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341513 2552 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 07 10:08:03.348910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341516 2552 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341519 2552 flags.go:64] FLAG: --rotate-certificates="false" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341522 2552 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341525 2552 flags.go:64] FLAG: --runonce="false" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341528 2552 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341531 2552 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341534 2552 flags.go:64] FLAG: --seccomp-default="false" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341538 2552 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341541 2552 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341543 2552 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341546 2552 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341549 2552 flags.go:64] FLAG: --storage-driver-password="root" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341552 2552 flags.go:64] FLAG: --storage-driver-secure="false" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341555 2552 flags.go:64] FLAG: --storage-driver-table="stats" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341558 2552 flags.go:64] FLAG: --storage-driver-user="root" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341561 2552 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341564 2552 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341568 2552 flags.go:64] FLAG: --system-cgroups="" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341571 2552 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341576 2552 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341579 2552 flags.go:64] FLAG: --tls-cert-file="" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341582 2552 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341588 2552 flags.go:64] FLAG: --tls-min-version="" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341591 2552 flags.go:64] FLAG: --tls-private-key-file="" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341594 2552 flags.go:64] FLAG: --topology-manager-policy="none" Sep 07 10:08:03.349538 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341597 2552 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 07 10:08:03.350194 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341600 2552 flags.go:64] FLAG: --topology-manager-scope="container" Sep 07 10:08:03.350194 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341603 2552 flags.go:64] FLAG: --v="2" Sep 07 10:08:03.350194 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341607 2552 flags.go:64] FLAG: --version="false" Sep 07 10:08:03.350194 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341611 2552 flags.go:64] FLAG: --vmodule="" Sep 07 10:08:03.350194 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341614 2552 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 07 10:08:03.350194 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.341617 2552 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 07 10:08:03.350194 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341730 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 10:08:03.350194 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341733 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 10:08:03.350194 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341736 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 10:08:03.350194 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341739 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 10:08:03.350194 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341742 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 10:08:03.350194 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341745 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 10:08:03.350194 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341747 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 10:08:03.350194 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341750 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 10:08:03.350194 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341753 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 10:08:03.350194 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341755 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 10:08:03.350194 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341758 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 10:08:03.350194 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341761 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 10:08:03.350194 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341763 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 10:08:03.350194 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341765 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 10:08:03.350194 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341768 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 10:08:03.350710 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341771 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 10:08:03.350710 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341773 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 10:08:03.350710 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341776 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 10:08:03.350710 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341779 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 10:08:03.350710 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341782 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 10:08:03.350710 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341785 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 10:08:03.350710 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341788 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 10:08:03.350710 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341792 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 10:08:03.350710 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341795 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 10:08:03.350710 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341797 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 10:08:03.350710 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341800 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 10:08:03.350710 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341803 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 10:08:03.350710 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341805 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 10:08:03.350710 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341808 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 10:08:03.350710 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341810 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 10:08:03.350710 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341813 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 10:08:03.350710 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341815 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 10:08:03.350710 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341818 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 10:08:03.350710 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341821 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 10:08:03.351314 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341823 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 10:08:03.351314 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341826 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 10:08:03.351314 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341828 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 10:08:03.351314 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341831 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 10:08:03.351314 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341834 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 10:08:03.351314 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341837 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 10:08:03.351314 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341840 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 10:08:03.351314 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341843 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 10:08:03.351314 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341845 2552 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 10:08:03.351314 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341847 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 10:08:03.351314 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341850 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 10:08:03.351314 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341866 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 10:08:03.351314 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341870 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 10:08:03.351314 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341874 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 10:08:03.351314 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341877 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 10:08:03.351314 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341881 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 10:08:03.351314 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341884 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 10:08:03.351314 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341888 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 10:08:03.351314 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341891 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 10:08:03.351314 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341894 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 10:08:03.351810 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341898 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 10:08:03.351810 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341901 2552 feature_gate.go:328] unrecognized feature gate: Example Sep 07 10:08:03.351810 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341903 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 10:08:03.351810 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341906 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 10:08:03.351810 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341909 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 10:08:03.351810 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341912 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 10:08:03.351810 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341914 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 10:08:03.351810 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341917 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 10:08:03.351810 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341919 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 10:08:03.351810 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341922 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 10:08:03.351810 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341924 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 10:08:03.351810 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341927 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 10:08:03.351810 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341929 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 10:08:03.351810 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341932 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 10:08:03.351810 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341934 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 10:08:03.351810 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341937 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 10:08:03.351810 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341939 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 10:08:03.351810 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341942 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 10:08:03.351810 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341945 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 10:08:03.351810 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341947 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 10:08:03.352320 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341950 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 10:08:03.352320 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341952 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 10:08:03.352320 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341955 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 10:08:03.352320 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341957 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 10:08:03.352320 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341960 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 10:08:03.352320 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341962 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 10:08:03.352320 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341965 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 10:08:03.352320 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341967 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 10:08:03.352320 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341970 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 10:08:03.352320 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341974 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 10:08:03.352320 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341977 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 10:08:03.352320 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341980 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 10:08:03.352320 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341985 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 10:08:03.352320 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341988 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 10:08:03.352320 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341991 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 10:08:03.352320 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341994 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 10:08:03.352320 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.341997 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 10:08:03.352320 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.342000 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 10:08:03.352320 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.342003 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 10:08:03.352793 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.342005 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 10:08:03.352793 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.342008 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 10:08:03.352793 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.342010 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 10:08:03.352793 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.342013 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 10:08:03.352793 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.342015 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 10:08:03.352793 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.342018 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 10:08:03.352793 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.342618 2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 07 10:08:03.352793 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.350017 2552 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 07 10:08:03.352793 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.350031 2552 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 07 10:08:03.352793 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.350935 2552 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 07 10:08:03.352793 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.350943 2552 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 07 10:08:03.352793 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351002 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 10:08:03.352793 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351009 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 10:08:03.352793 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351014 2552 feature_gate.go:328] unrecognized feature gate: Example Sep 07 10:08:03.352793 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351017 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 10:08:03.353174 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351020 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 10:08:03.353174 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351023 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 10:08:03.353174 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351027 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 10:08:03.353174 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351031 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 10:08:03.353174 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351034 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 10:08:03.353174 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351037 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 10:08:03.353174 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351040 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 10:08:03.353174 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351043 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 10:08:03.353174 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351046 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 10:08:03.353174 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351049 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 10:08:03.353174 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351053 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 10:08:03.353174 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351056 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 10:08:03.353174 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351059 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 10:08:03.353174 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351062 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 10:08:03.353174 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351065 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 10:08:03.353174 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351067 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 10:08:03.353174 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351070 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 10:08:03.353174 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351073 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 10:08:03.353174 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351076 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 10:08:03.353651 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351078 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 10:08:03.353651 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351081 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 10:08:03.353651 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351083 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 10:08:03.353651 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351086 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 10:08:03.353651 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351089 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 10:08:03.353651 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351091 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 10:08:03.353651 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351095 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 10:08:03.353651 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351099 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 10:08:03.353651 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351101 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 10:08:03.353651 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351104 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 10:08:03.353651 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351106 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 10:08:03.353651 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351109 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 10:08:03.353651 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351112 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 10:08:03.353651 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351115 2552 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 10:08:03.353651 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351117 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 10:08:03.353651 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351120 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 10:08:03.353651 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351122 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 10:08:03.353651 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351125 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 10:08:03.353651 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351128 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 10:08:03.353651 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351130 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 10:08:03.354180 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351133 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 10:08:03.354180 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351136 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 10:08:03.354180 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351139 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 10:08:03.354180 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351141 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 10:08:03.354180 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351144 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 10:08:03.354180 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351147 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 10:08:03.354180 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351149 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 10:08:03.354180 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351152 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 10:08:03.354180 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351155 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 10:08:03.354180 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351157 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 10:08:03.354180 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351160 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 10:08:03.354180 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351162 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 10:08:03.354180 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351165 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 10:08:03.354180 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351167 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 10:08:03.354180 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351170 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 10:08:03.354180 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351172 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 10:08:03.354180 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351175 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 10:08:03.354180 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351178 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 10:08:03.354180 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351181 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 10:08:03.354180 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351183 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 10:08:03.354677 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351186 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 10:08:03.354677 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351189 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 10:08:03.354677 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351193 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 10:08:03.354677 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351197 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 10:08:03.354677 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351199 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 10:08:03.354677 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351202 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 10:08:03.354677 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351205 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 10:08:03.354677 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351207 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 10:08:03.354677 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351210 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 10:08:03.354677 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351213 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 10:08:03.354677 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351215 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 10:08:03.354677 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351218 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 10:08:03.354677 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351220 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 10:08:03.354677 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351223 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 10:08:03.354677 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351226 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 10:08:03.354677 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351229 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 10:08:03.354677 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351232 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 10:08:03.354677 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351234 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 10:08:03.354677 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351237 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 10:08:03.354677 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351280 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 10:08:03.355160 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351287 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 10:08:03.355160 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351290 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 10:08:03.355160 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351293 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 10:08:03.355160 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351295 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 10:08:03.355160 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351298 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 10:08:03.355160 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351301 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 10:08:03.355160 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351304 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 10:08:03.355160 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351306 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 10:08:03.355160 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351309 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 10:08:03.355160 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351311 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 10:08:03.355160 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351314 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 10:08:03.355160 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351317 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 10:08:03.355160 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351319 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 10:08:03.355160 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351323 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 10:08:03.355160 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351326 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 10:08:03.355160 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351328 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 10:08:03.355687 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.351334 2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 07 10:08:03.355687 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351461 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 10:08:03.355687 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351466 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 10:08:03.355687 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351469 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 10:08:03.355687 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351472 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 10:08:03.355687 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351475 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 10:08:03.355687 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351478 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 10:08:03.355687 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351481 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 10:08:03.355687 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351484 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 10:08:03.355687 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351487 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 10:08:03.355687 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351489 2552 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 10:08:03.355687 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351492 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 10:08:03.355687 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351495 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 10:08:03.355687 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351500 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 10:08:03.356027 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351504 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 10:08:03.356027 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351507 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 10:08:03.356027 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351510 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 10:08:03.356027 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351513 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 10:08:03.356027 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351516 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 10:08:03.356027 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351519 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 10:08:03.356027 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351522 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 10:08:03.356027 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351524 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 10:08:03.356027 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351527 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 10:08:03.356027 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351530 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 10:08:03.356027 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351533 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 10:08:03.356027 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351536 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 10:08:03.356027 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351539 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 10:08:03.356027 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351542 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 10:08:03.356027 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351544 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 10:08:03.356027 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351547 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 10:08:03.356027 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351550 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 10:08:03.356027 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351553 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 10:08:03.356027 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351556 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 10:08:03.356027 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351558 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 10:08:03.356511 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351561 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 10:08:03.356511 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351563 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 10:08:03.356511 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351566 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 10:08:03.356511 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351568 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 10:08:03.356511 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351571 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 10:08:03.356511 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351574 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 10:08:03.356511 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351576 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 10:08:03.356511 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351579 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 10:08:03.356511 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351581 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 10:08:03.356511 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351584 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 10:08:03.356511 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351586 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 10:08:03.356511 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351589 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 10:08:03.356511 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351592 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 10:08:03.356511 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351595 2552 feature_gate.go:328] unrecognized feature gate: Example Sep 07 10:08:03.356511 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351598 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 10:08:03.356511 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351601 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 10:08:03.356511 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351603 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 10:08:03.356511 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351606 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 10:08:03.356511 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351608 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 10:08:03.356511 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351611 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 10:08:03.356990 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351614 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 10:08:03.356990 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351616 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 10:08:03.356990 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351619 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 10:08:03.356990 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351622 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 10:08:03.356990 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351624 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 10:08:03.356990 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351627 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 10:08:03.356990 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351629 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 10:08:03.356990 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351632 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 10:08:03.356990 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351634 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 10:08:03.356990 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351637 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 10:08:03.356990 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351639 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 10:08:03.356990 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351642 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 10:08:03.356990 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351645 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 10:08:03.356990 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351648 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 10:08:03.356990 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351650 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 10:08:03.356990 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351652 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 10:08:03.356990 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351655 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 10:08:03.356990 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351658 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 10:08:03.356990 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351661 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 10:08:03.356990 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351663 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 10:08:03.357496 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351666 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 10:08:03.357496 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351668 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 10:08:03.357496 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351671 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 10:08:03.357496 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351674 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 10:08:03.357496 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351676 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 10:08:03.357496 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351679 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 10:08:03.357496 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351682 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 10:08:03.357496 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351684 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 10:08:03.357496 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351687 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 10:08:03.357496 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351690 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 10:08:03.357496 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351692 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 10:08:03.357496 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351695 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 10:08:03.357496 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351698 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 10:08:03.357496 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351702 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 10:08:03.357496 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351705 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 10:08:03.357496 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351708 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 10:08:03.357496 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351711 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 10:08:03.357496 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351715 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 10:08:03.357496 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351718 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 10:08:03.357496 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351721 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 10:08:03.357997 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351724 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 10:08:03.357997 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351726 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 10:08:03.357997 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351729 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 10:08:03.357997 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351731 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 10:08:03.357997 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351733 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 10:08:03.357997 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:03.351736 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 10:08:03.357997 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.351741 2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 07 10:08:03.357997 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.351845 2552 server.go:962] "Client rotation is on, will bootstrap in background" Sep 07 10:08:03.357997 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.353833 2552 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 07 10:08:03.357997 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.354866 2552 server.go:1019] "Starting client certificate rotation" Sep 07 10:08:03.357997 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.354966 2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 07 10:08:03.357997 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.354996 2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 07 10:08:03.357997 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.357396 2552 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 07 10:08:03.359313 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.359288 2552 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 07 10:08:03.369568 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.369546 2552 log.go:25] "Validated CRI v1 runtime API" Sep 07 10:08:03.372517 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.372502 2552 log.go:25] "Validated CRI v1 image API" Sep 07 10:08:03.373395 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.373378 2552 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 07 10:08:03.375788 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.375769 2552 fs.go:135] Filesystem UUIDs: map[6844ddd2-7cd1-4e2b-bb1d-d0e3411acae1:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 86864b60-65f6-444c-912a-ea5b26f78c3a:/dev/nvme0n1p4] Sep 07 10:08:03.375849 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.375787 2552 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 07 10:08:03.380987 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.380872 2552 manager.go:216] Machine: {Timestamp:2026-09-07 10:08:03.379527353 +0000 UTC m=+0.347939137 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3117279 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27ef9fe0df2521e802a8ff7711a421 SystemUUID:ec27ef9f-e0df-2521-e802-a8ff7711a421 BootID:e69eb4f2-d222-4de9-a398-c416cc8ad7cb Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f8:37:54:3e:1f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f8:37:54:3e:1f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2a:13:03:15:c1:44 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 07 10:08:03.380987 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.380983 2552 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 07 10:08:03.381097 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.381060 2552 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 07 10:08:03.384062 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.384031 2552 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 07 10:08:03.384198 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.384065 2552 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-225.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 07 10:08:03.384811 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.384800 2552 topology_manager.go:138] "Creating topology manager with none policy" Sep 07 10:08:03.384847 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.384813 2552 container_manager_linux.go:309] "Creating device plugin manager" Sep 07 10:08:03.384847 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.384821 2552 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 07 10:08:03.385453 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.385442 2552 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 07 10:08:03.386846 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.386836 2552 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 07 10:08:03.390641 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.390625 2552 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 07 10:08:03.390941 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.390929 2552 state_mem.go:36] "Initialized new in-memory state store" Sep 07 10:08:03.391060 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.391051 2552 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 07 10:08:03.391093 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.391080 2552 kubelet.go:486] "Attempting to sync node with API server" Sep 07 10:08:03.391093 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.391090 2552 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 07 10:08:03.391150 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.391104 2552 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 07 10:08:03.391150 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.391112 2552 kubelet.go:398] "Adding apiserver pod source" Sep 07 10:08:03.391150 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.391121 2552 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 07 10:08:03.391531 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.391519 2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 07 10:08:03.391602 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.391540 2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 07 10:08:03.393859 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.393833 2552 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 07 10:08:03.395978 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.395965 2552 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 07 10:08:03.396051 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.395994 2552 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 07 10:08:03.396146 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.396137 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 07 10:08:03.396177 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.396151 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 07 10:08:03.396177 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.396157 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 07 10:08:03.396177 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.396162 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 07 10:08:03.396177 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.396170 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 07 10:08:03.396177 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.396175 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 07 10:08:03.396818 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.396809 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 07 10:08:03.396818 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.396818 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 07 10:08:03.396870 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.396826 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 07 10:08:03.396870 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.396832 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 07 10:08:03.396870 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.396840 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 07 10:08:03.396870 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.396847 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 07 10:08:03.397593 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.397578 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 07 10:08:03.397593 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.397587 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 07 10:08:03.399453 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.399434 2552 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-225.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 07 10:08:03.399453 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:03.399437 2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-225.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 07 10:08:03.399453 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:03.399448 2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 07 10:08:03.401330 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.401315 2552 server.go:1268] "Started kubelet" Sep 07 10:08:03.401403 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.401362 2552 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 07 10:08:03.401403 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.401385 2552 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 07 10:08:03.401494 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.401470 2552 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 07 10:08:03.406085 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.405600 2552 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 07 10:08:03.405980 ip-10-0-140-225 systemd[1]: Started Kubernetes Kubelet. Sep 07 10:08:03.408747 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.407680 2552 server.go:310] "Adding debug handlers to kubelet server" Sep 07 10:08:03.408747 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:03.408488 2552 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 07 10:08:03.412257 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:03.411061 2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-225.ec2.internal.18d30201b7fd96c8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-225.ec2.internal,UID:ip-10-0-140-225.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-225.ec2.internal,},FirstTimestamp:2026-09-07 10:08:03.401291464 +0000 UTC m=+0.369703247,LastTimestamp:2026-09-07 10:08:03.401291464 +0000 UTC m=+0.369703247,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-225.ec2.internal,}" Sep 07 10:08:03.413173 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.413149 2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 07 10:08:03.413772 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.413755 2552 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 07 10:08:03.414317 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.414297 2552 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 07 10:08:03.414378 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.414320 2552 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 07 10:08:03.414378 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.414298 2552 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 07 10:08:03.414378 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.414376 2552 reconstruct.go:97] "Volume reconstruction finished" Sep 07 10:08:03.414496 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.414383 2552 reconciler.go:29] "Reconciler: start to sync state" Sep 07 10:08:03.414546 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:03.414506 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-225.ec2.internal\" not found" Sep 07 10:08:03.415027 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.415011 2552 factory.go:55] Registering systemd factory Sep 07 10:08:03.415027 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.415028 2552 factory.go:223] Registration of the systemd container factory successfully Sep 07 10:08:03.415223 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.415211 2552 factory.go:153] Registering CRI-O factory Sep 07 10:08:03.415223 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.415222 2552 factory.go:223] Registration of the crio container factory successfully Sep 07 10:08:03.415328 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.415292 2552 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 07 10:08:03.415328 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.415310 2552 factory.go:103] Registering Raw factory Sep 07 10:08:03.415328 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.415325 2552 manager.go:1196] Started watching for new ooms in manager Sep 07 10:08:03.415613 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.415602 2552 manager.go:349] Starting recovery of all containers Sep 07 10:08:03.423647 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:03.423612 2552 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 07 10:08:03.423647 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:03.423605 2552 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-225.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 07 10:08:03.425605 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.425578 2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tz8bk" Sep 07 10:08:03.425961 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.425945 2552 manager.go:354] Recovery completed Sep 07 10:08:03.430177 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.430165 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 10:08:03.432003 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.431988 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeHasSufficientMemory" Sep 07 10:08:03.432056 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.432023 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 10:08:03.432056 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.432038 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeHasSufficientPID" Sep 07 10:08:03.432634 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.432619 2552 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 07 10:08:03.432634 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.432632 2552 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 07 10:08:03.432634 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.432633 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 10:08:03.432849 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.432646 2552 state_mem.go:36] "Initialized new in-memory state store" Sep 07 10:08:03.433319 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.433305 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeHasSufficientMemory" Sep 07 10:08:03.433386 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.433338 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 10:08:03.433386 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.433353 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeHasSufficientPID" Sep 07 10:08:03.434359 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.434345 2552 policy_none.go:49] "None policy: Start" Sep 07 10:08:03.434427 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.434365 2552 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 07 10:08:03.434427 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.434378 2552 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 07 10:08:03.434621 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:03.434557 2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-225.ec2.internal.18d30201b9d23c49 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-225.ec2.internal,UID:ip-10-0-140-225.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-225.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-225.ec2.internal,},FirstTimestamp:2026-09-07 10:08:03.432004681 +0000 UTC m=+0.400416469,LastTimestamp:2026-09-07 10:08:03.432004681 +0000 UTC m=+0.400416469,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-225.ec2.internal,}" Sep 07 10:08:03.435905 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.435893 2552 policy_none.go:47] "Start" Sep 07 10:08:03.439124 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.439106 2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tz8bk" Sep 07 10:08:03.443507 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:03.443434 2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-225.ec2.internal.18d30201b9d2a2e4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-225.ec2.internal,UID:ip-10-0-140-225.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-225.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-225.ec2.internal,},FirstTimestamp:2026-09-07 10:08:03.432030948 +0000 UTC m=+0.400442735,LastTimestamp:2026-09-07 10:08:03.432030948 +0000 UTC m=+0.400442735,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-225.ec2.internal,}" Sep 07 10:08:03.492789 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.472920 2552 manager.go:337] "Starting Device Plugin manager" Sep 07 10:08:03.492789 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:03.472954 2552 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 07 10:08:03.492789 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.472965 2552 server.go:87] "Starting device plugin registration server" Sep 07 10:08:03.492789 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.473307 2552 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 07 10:08:03.492789 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.473319 2552 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 07 10:08:03.492789 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.473416 2552 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 07 10:08:03.492789 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.473495 2552 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 07 10:08:03.492789 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.473506 2552 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 07 10:08:03.492789 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:03.474086 2552 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 07 10:08:03.492789 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:03.474130 2552 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-225.ec2.internal\" not found" Sep 07 10:08:03.514474 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.514439 2552 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 07 10:08:03.515702 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.515686 2552 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 07 10:08:03.515791 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.515740 2552 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 07 10:08:03.515791 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.515764 2552 kubelet.go:2448] "Starting kubelet main sync loop" Sep 07 10:08:03.515893 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:03.515862 2552 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 07 10:08:03.518076 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.518053 2552 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 10:08:03.574172 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.574139 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 10:08:03.575166 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.575141 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeHasSufficientMemory" Sep 07 10:08:03.575310 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.575175 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 10:08:03.575310 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.575188 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeHasSufficientPID" Sep 07 10:08:03.575310 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.575218 2552 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-225.ec2.internal" Sep 07 10:08:03.581655 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.581639 2552 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-225.ec2.internal" Sep 07 10:08:03.581719 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:03.581662 2552 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-225.ec2.internal\": node \"ip-10-0-140-225.ec2.internal\" not found" Sep 07 10:08:03.600533 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:03.600439 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-225.ec2.internal\" not found" Sep 07 10:08:03.616822 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.616798 2552 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-225.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-225.ec2.internal"] Sep 07 10:08:03.616905 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.616859 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 10:08:03.617746 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.617732 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeHasSufficientMemory" Sep 07 10:08:03.617801 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.617758 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 10:08:03.617801 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.617768 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeHasSufficientPID" Sep 07 10:08:03.620658 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.620646 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 10:08:03.620824 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.620811 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-225.ec2.internal" Sep 07 10:08:03.620864 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.620845 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 10:08:03.621396 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.621379 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeHasSufficientMemory" Sep 07 10:08:03.621496 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.621404 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 10:08:03.621496 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.621413 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeHasSufficientPID" Sep 07 10:08:03.621496 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.621379 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeHasSufficientMemory" Sep 07 10:08:03.621496 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.621482 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 10:08:03.621496 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.621495 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeHasSufficientPID" Sep 07 10:08:03.623485 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.623472 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-225.ec2.internal" Sep 07 10:08:03.623537 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.623496 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 10:08:03.624409 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.624395 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeHasSufficientMemory" Sep 07 10:08:03.624497 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.624420 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 10:08:03.624497 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.624433 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeHasSufficientPID" Sep 07 10:08:03.644901 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:03.644880 2552 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-225.ec2.internal\" not found" node="ip-10-0-140-225.ec2.internal" Sep 07 10:08:03.649398 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:03.649382 2552 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-225.ec2.internal\" not found" node="ip-10-0-140-225.ec2.internal" Sep 07 10:08:03.701136 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:03.701101 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-225.ec2.internal\" not found" Sep 07 10:08:03.715625 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.715602 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/615daa2f0d915b9ccc186d19a8658ab6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-225.ec2.internal\" (UID: \"615daa2f0d915b9ccc186d19a8658ab6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-225.ec2.internal" Sep 07 10:08:03.715717 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.715629 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/615daa2f0d915b9ccc186d19a8658ab6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-225.ec2.internal\" (UID: \"615daa2f0d915b9ccc186d19a8658ab6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-225.ec2.internal" Sep 07 10:08:03.715717 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.715653 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/35886f0cd374d86381ddcbd8cc29c623-config\") pod \"kube-apiserver-proxy-ip-10-0-140-225.ec2.internal\" (UID: \"35886f0cd374d86381ddcbd8cc29c623\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-225.ec2.internal" Sep 07 10:08:03.801872 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:03.801836 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-225.ec2.internal\" not found" Sep 07 10:08:03.816123 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.816096 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/35886f0cd374d86381ddcbd8cc29c623-config\") pod \"kube-apiserver-proxy-ip-10-0-140-225.ec2.internal\" (UID: \"35886f0cd374d86381ddcbd8cc29c623\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-225.ec2.internal" Sep 07 10:08:03.816229 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.816125 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/615daa2f0d915b9ccc186d19a8658ab6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-225.ec2.internal\" (UID: \"615daa2f0d915b9ccc186d19a8658ab6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-225.ec2.internal" Sep 07 10:08:03.816229 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.816141 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/615daa2f0d915b9ccc186d19a8658ab6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-225.ec2.internal\" (UID: \"615daa2f0d915b9ccc186d19a8658ab6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-225.ec2.internal" Sep 07 10:08:03.816229 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.816197 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/615daa2f0d915b9ccc186d19a8658ab6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-225.ec2.internal\" (UID: \"615daa2f0d915b9ccc186d19a8658ab6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-225.ec2.internal" Sep 07 10:08:03.816229 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.816207 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/35886f0cd374d86381ddcbd8cc29c623-config\") pod \"kube-apiserver-proxy-ip-10-0-140-225.ec2.internal\" (UID: \"35886f0cd374d86381ddcbd8cc29c623\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-225.ec2.internal" Sep 07 10:08:03.816373 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.816207 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/615daa2f0d915b9ccc186d19a8658ab6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-225.ec2.internal\" (UID: \"615daa2f0d915b9ccc186d19a8658ab6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-225.ec2.internal" Sep 07 10:08:03.902792 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:03.902722 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-225.ec2.internal\" not found" Sep 07 10:08:03.948176 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.948152 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-225.ec2.internal" Sep 07 10:08:03.951855 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:03.951830 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-225.ec2.internal" Sep 07 10:08:04.003552 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:04.003520 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-225.ec2.internal\" not found" Sep 07 10:08:04.106943 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:04.106911 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-225.ec2.internal\" not found" Sep 07 10:08:04.208918 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:04.208839 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-225.ec2.internal\" not found" Sep 07 10:08:04.309057 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:04.309020 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-225.ec2.internal\" not found" Sep 07 10:08:04.355172 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:04.355145 2552 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 07 10:08:04.355663 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:04.355329 2552 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 07 10:08:04.409837 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:04.409808 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-225.ec2.internal\" not found" Sep 07 10:08:04.442154 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:04.442110 2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-06 10:03:03 +0000 UTC" deadline="2028-04-11 18:03:40.083396678 +0000 UTC" Sep 07 10:08:04.442154 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:04.442147 2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13975h55m35.641254936s" Sep 07 10:08:04.478624 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:04.478550 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 10:08:04.479851 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:04.479824 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeHasSufficientMemory" Sep 07 10:08:04.479972 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:04.479863 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 10:08:04.479972 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:04.479879 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeHasSufficientPID" Sep 07 10:08:04.510971 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:04.510929 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-225.ec2.internal\" not found" Sep 07 10:08:04.549318 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:04.549277 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35886f0cd374d86381ddcbd8cc29c623.slice/crio-203487c0dd743d8780f201fdfedac13ba32d96a322f1e6ca1cd599aa03047477 WatchSource:0}: Error finding container 203487c0dd743d8780f201fdfedac13ba32d96a322f1e6ca1cd599aa03047477: Status 404 returned error can't find the container with id 203487c0dd743d8780f201fdfedac13ba32d96a322f1e6ca1cd599aa03047477 Sep 07 10:08:04.555975 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:04.555938 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod615daa2f0d915b9ccc186d19a8658ab6.slice/crio-27c9c06416ef47a6641c11bbe69b879884a8b77fa0611c35d86a3236c73d4a5b WatchSource:0}: Error finding container 27c9c06416ef47a6641c11bbe69b879884a8b77fa0611c35d86a3236c73d4a5b: Status 404 returned error can't find the container with id 27c9c06416ef47a6641c11bbe69b879884a8b77fa0611c35d86a3236c73d4a5b Sep 07 10:08:04.558145 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:04.558128 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 07 10:08:04.613192 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:04.613146 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-225.ec2.internal\" not found" Sep 07 10:08:04.704526 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:04.704503 2552 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 10:08:04.713792 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:04.713774 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-225.ec2.internal\" not found" Sep 07 10:08:04.814863 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:04.814789 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-225.ec2.internal\" not found" Sep 07 10:08:04.867389 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:04.867366 2552 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 10:08:04.907544 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:04.907522 2552 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 10:08:04.914452 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:04.914432 2552 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-225.ec2.internal" Sep 07 10:08:04.925152 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:04.925133 2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 07 10:08:04.926413 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:04.926400 2552 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-225.ec2.internal" Sep 07 10:08:04.934239 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:04.934225 2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 07 10:08:05.323357 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.323311 2552 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 10:08:05.395870 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.391933 2552 apiserver.go:52] "Watching apiserver" Sep 07 10:08:05.409571 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.409534 2552 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 07 10:08:05.412088 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.412058 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-v9stm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-225.ec2.internal","openshift-multus/multus-additional-cni-plugins-gmbs2","openshift-ovn-kubernetes/ovnkube-node-lcshb","kube-system/konnectivity-agent-4qt9r","openshift-dns/node-resolver-kqh2w","openshift-multus/multus-vs9qt","openshift-multus/network-metrics-daemon-f5sf9","openshift-network-diagnostics/network-check-target-w9rwx","openshift-network-operator/iptables-alerter-z27cp","kube-system/global-pull-secret-syncer-llgqf","kube-system/kube-apiserver-proxy-ip-10-0-140-225.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq","openshift-cluster-node-tuning-operator/tuned-9rj9f"] Sep 07 10:08:05.414737 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.414715 2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 07 10:08:05.414863 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.414803 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.420299 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.417833 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" Sep 07 10:08:05.420299 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.418761 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 07 10:08:05.420299 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.418827 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 07 10:08:05.420299 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.419025 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 07 10:08:05.420299 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.419228 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 07 10:08:05.420299 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.419519 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kwkch\"" Sep 07 10:08:05.420299 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.419841 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 07 10:08:05.420738 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.420357 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8gkd5\"" Sep 07 10:08:05.420738 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.420667 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 07 10:08:05.421038 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.420865 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 07 10:08:05.422819 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.422543 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gmbs2" Sep 07 10:08:05.424910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.424606 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 07 10:08:05.425544 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.424868 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bzdrq\"" Sep 07 10:08:05.425544 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.425065 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-host-var-lib-kubelet\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.425544 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.425110 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-multus-conf-dir\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.425544 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.425133 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-system-cni-dir\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.425544 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.425157 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2293164a-1a0c-49ea-95b1-d5d0b223d585-cni-binary-copy\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.425544 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.425181 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-hostroot\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.425544 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.424873 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 07 10:08:05.425544 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.425203 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-etc-kubernetes\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.425544 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.425229 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmplf\" (UniqueName: \"kubernetes.io/projected/2293164a-1a0c-49ea-95b1-d5d0b223d585-kube-api-access-hmplf\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.425544 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.425256 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-cnibin\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.425544 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.425302 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-multus-socket-dir-parent\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.425544 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.425330 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-multus-cni-dir\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.425544 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.425355 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-host-run-k8s-cni-cncf-io\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.425544 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.424974 2552 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 07 10:08:05.425544 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.425377 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2293164a-1a0c-49ea-95b1-d5d0b223d585-multus-daemon-config\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.425544 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.425493 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-host-run-multus-certs\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.425544 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.425516 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-host-run-netns\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.425544 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.425539 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-host-var-lib-cni-bin\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.426421 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.425560 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-host-var-lib-cni-multus\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.428027 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.427195 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.429744 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.429410 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 07 10:08:05.429744 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.429493 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8k9jq\"" Sep 07 10:08:05.429744 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.429522 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 07 10:08:05.429744 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.429706 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 07 10:08:05.429744 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.429706 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 07 10:08:05.430085 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.429954 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 07 10:08:05.430442 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.430408 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4qt9r" Sep 07 10:08:05.430582 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.430567 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 07 10:08:05.432443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.432407 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 07 10:08:05.432653 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.432630 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 07 10:08:05.433257 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.432882 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wlxjf\"" Sep 07 10:08:05.433257 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.432937 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kqh2w" Sep 07 10:08:05.435198 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.434884 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-slbff\"" Sep 07 10:08:05.435198 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.434921 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 07 10:08:05.435377 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.435278 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 07 10:08:05.435800 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.435560 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:05.435800 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:05.435621 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w9rwx" podUID="1359e8cd-f362-45c5-8071-c59d23eab440" Sep 07 10:08:05.437830 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.437807 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v9stm" Sep 07 10:08:05.440908 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.440884 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.443879 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.443858 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z27cp" Sep 07 10:08:05.444567 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.444532 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 07 10:08:05.444894 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.444874 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-v244m\"" Sep 07 10:08:05.444976 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.444966 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 07 10:08:05.445117 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.445090 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 07 10:08:05.445283 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.445090 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 07 10:08:05.446048 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.446021 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 07 10:08:05.446134 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.446117 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 07 10:08:05.446553 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.446538 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 07 10:08:05.446599 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.445810 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xjmtb\"" Sep 07 10:08:05.446640 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.446545 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6b5mn\"" Sep 07 10:08:05.446640 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.446634 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:05.446730 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.446545 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 07 10:08:05.446779 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:05.446761 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llgqf" podUID="5c35d143-5a49-4636-b86b-c1dd3f6df6c3" Sep 07 10:08:05.448826 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.448713 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:05.448826 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:05.448775 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5sf9" podUID="6cae549b-87f4-4031-b984-83a7d64f9dd3" Sep 07 10:08:05.448826 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.448797 2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cthxr" Sep 07 10:08:05.457264 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.457194 2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cthxr" Sep 07 10:08:05.515408 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.515382 2552 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 07 10:08:05.520825 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.520776 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-225.ec2.internal" event={"ID":"615daa2f0d915b9ccc186d19a8658ab6","Type":"ContainerStarted","Data":"27c9c06416ef47a6641c11bbe69b879884a8b77fa0611c35d86a3236c73d4a5b"} Sep 07 10:08:05.522029 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.522007 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-225.ec2.internal" event={"ID":"35886f0cd374d86381ddcbd8cc29c623","Type":"ContainerStarted","Data":"203487c0dd743d8780f201fdfedac13ba32d96a322f1e6ca1cd599aa03047477"} Sep 07 10:08:05.525734 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.525707 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-etc-openvswitch\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.525832 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.525736 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fe03fc91-4aa7-46e8-a9a2-e78173df653b-ovn-node-metrics-cert\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.525832 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.525754 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36ee18c1-8608-4bf7-84d8-f94c414d02bc-host-slash\") pod \"iptables-alerter-z27cp\" (UID: \"36ee18c1-8608-4bf7-84d8-f94c414d02bc\") " pod="openshift-network-operator/iptables-alerter-z27cp" Sep 07 10:08:05.525832 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.525782 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-multus-cni-dir\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.525988 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.525870 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-multus-cni-dir\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.525988 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.525946 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fe03fc91-4aa7-46e8-a9a2-e78173df653b-ovnkube-script-lib\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.525988 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.525973 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9878\" (UniqueName: \"kubernetes.io/projected/fe03fc91-4aa7-46e8-a9a2-e78173df653b-kube-api-access-f9878\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.526130 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526003 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-host-var-lib-cni-bin\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.526130 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526028 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-multus-conf-dir\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.526130 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526050 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-systemd-units\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.526130 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526077 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2293164a-1a0c-49ea-95b1-d5d0b223d585-cni-binary-copy\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.526130 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526103 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-hostroot\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.526130 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526110 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-host-var-lib-cni-bin\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.526431 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526121 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-multus-conf-dir\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.526431 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526127 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a04783c7-15ce-4d0c-82a3-6b946c5ecda0-system-cni-dir\") pod \"multus-additional-cni-plugins-gmbs2\" (UID: \"a04783c7-15ce-4d0c-82a3-6b946c5ecda0\") " pod="openshift-multus/multus-additional-cni-plugins-gmbs2" Sep 07 10:08:05.526431 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526165 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-hostroot\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.526431 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526174 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-log-socket\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.526431 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526217 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b2836d86-221b-4f6e-9d5e-5f0edf93e61b-registration-dir\") pod \"aws-ebs-csi-driver-node-m4tlq\" (UID: \"b2836d86-221b-4f6e-9d5e-5f0edf93e61b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" Sep 07 10:08:05.526431 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526259 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a04783c7-15ce-4d0c-82a3-6b946c5ecda0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gmbs2\" (UID: \"a04783c7-15ce-4d0c-82a3-6b946c5ecda0\") " pod="openshift-multus/multus-additional-cni-plugins-gmbs2" Sep 07 10:08:05.526431 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526287 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-var-lib-kubelet\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.526431 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526309 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-kubelet-config\") pod \"global-pull-secret-syncer-llgqf\" (UID: \"5c35d143-5a49-4636-b86b-c1dd3f6df6c3\") " pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:05.526431 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526344 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-host-run-multus-certs\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.526431 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526370 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b2836d86-221b-4f6e-9d5e-5f0edf93e61b-device-dir\") pod \"aws-ebs-csi-driver-node-m4tlq\" (UID: \"b2836d86-221b-4f6e-9d5e-5f0edf93e61b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" Sep 07 10:08:05.526431 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526409 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-host-run-netns\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.526431 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526413 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-host-run-multus-certs\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.527047 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526452 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-host-var-lib-cni-multus\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.527047 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526482 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-host-run-netns\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.527047 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526519 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxrwz\" (UniqueName: \"kubernetes.io/projected/a04783c7-15ce-4d0c-82a3-6b946c5ecda0-kube-api-access-sxrwz\") pod \"multus-additional-cni-plugins-gmbs2\" (UID: \"a04783c7-15ce-4d0c-82a3-6b946c5ecda0\") " pod="openshift-multus/multus-additional-cni-plugins-gmbs2" Sep 07 10:08:05.527047 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526556 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-host-var-lib-cni-multus\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.527047 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526594 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aff3118f-d191-4990-b60b-958c0c686aec-host\") pod \"node-ca-v9stm\" (UID: \"aff3118f-d191-4990-b60b-958c0c686aec\") " pod="openshift-image-registry/node-ca-v9stm" Sep 07 10:08:05.527047 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526622 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-etc-modprobe-d\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.527047 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526651 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c06f0d60-b864-4616-82e6-3bf85f17249b-tmp\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.527047 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526690 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-run-systemd\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.527047 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526714 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fe03fc91-4aa7-46e8-a9a2-e78173df653b-env-overrides\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.527047 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526736 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a04783c7-15ce-4d0c-82a3-6b946c5ecda0-cnibin\") pod \"multus-additional-cni-plugins-gmbs2\" (UID: \"a04783c7-15ce-4d0c-82a3-6b946c5ecda0\") " pod="openshift-multus/multus-additional-cni-plugins-gmbs2" Sep 07 10:08:05.527047 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526755 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0e3d4718-d092-48e6-86f6-f09e1793eac1-konnectivity-ca\") pod \"konnectivity-agent-4qt9r\" (UID: \"0e3d4718-d092-48e6-86f6-f09e1793eac1\") " pod="kube-system/konnectivity-agent-4qt9r" Sep 07 10:08:05.527047 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526775 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dhv8\" (UniqueName: \"kubernetes.io/projected/aff3118f-d191-4990-b60b-958c0c686aec-kube-api-access-5dhv8\") pod \"node-ca-v9stm\" (UID: \"aff3118f-d191-4990-b60b-958c0c686aec\") " pod="openshift-image-registry/node-ca-v9stm" Sep 07 10:08:05.527047 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526796 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-host\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.527047 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526815 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-host-kubelet\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.527047 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526836 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f9e1b00d-e190-452a-8322-333f9397c2fa-hosts-file\") pod \"node-resolver-kqh2w\" (UID: \"f9e1b00d-e190-452a-8322-333f9397c2fa\") " pod="openshift-dns/node-resolver-kqh2w" Sep 07 10:08:05.527047 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526859 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w9cm\" (UniqueName: \"kubernetes.io/projected/f9e1b00d-e190-452a-8322-333f9397c2fa-kube-api-access-7w9cm\") pod \"node-resolver-kqh2w\" (UID: \"f9e1b00d-e190-452a-8322-333f9397c2fa\") " pod="openshift-dns/node-resolver-kqh2w" Sep 07 10:08:05.527047 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526883 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-original-pull-secret\") pod \"global-pull-secret-syncer-llgqf\" (UID: \"5c35d143-5a49-4636-b86b-c1dd3f6df6c3\") " pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:05.527954 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526922 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b2836d86-221b-4f6e-9d5e-5f0edf93e61b-etc-selinux\") pod \"aws-ebs-csi-driver-node-m4tlq\" (UID: \"b2836d86-221b-4f6e-9d5e-5f0edf93e61b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" Sep 07 10:08:05.527954 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526946 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a04783c7-15ce-4d0c-82a3-6b946c5ecda0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gmbs2\" (UID: \"a04783c7-15ce-4d0c-82a3-6b946c5ecda0\") " pod="openshift-multus/multus-additional-cni-plugins-gmbs2" Sep 07 10:08:05.527954 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526969 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a04783c7-15ce-4d0c-82a3-6b946c5ecda0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gmbs2\" (UID: \"a04783c7-15ce-4d0c-82a3-6b946c5ecda0\") " pod="openshift-multus/multus-additional-cni-plugins-gmbs2" Sep 07 10:08:05.527954 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.526993 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-host-run-k8s-cni-cncf-io\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.527954 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527019 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2293164a-1a0c-49ea-95b1-d5d0b223d585-multus-daemon-config\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.527954 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527041 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b2836d86-221b-4f6e-9d5e-5f0edf93e61b-sys-fs\") pod \"aws-ebs-csi-driver-node-m4tlq\" (UID: \"b2836d86-221b-4f6e-9d5e-5f0edf93e61b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" Sep 07 10:08:05.527954 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527056 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-host-run-k8s-cni-cncf-io\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.527954 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527070 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgvsz\" (UniqueName: \"kubernetes.io/projected/1359e8cd-f362-45c5-8071-c59d23eab440-kube-api-access-cgvsz\") pod \"network-check-target-w9rwx\" (UID: \"1359e8cd-f362-45c5-8071-c59d23eab440\") " pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:05.527954 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527122 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-var-lib-openvswitch\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.527954 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527148 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-run-ovn\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.527954 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527169 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-node-log\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.527954 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527200 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-host-cni-netd\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.527954 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527260 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-host-var-lib-kubelet\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.527954 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527297 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-host-var-lib-kubelet\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.527954 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527338 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/aff3118f-d191-4990-b60b-958c0c686aec-serviceca\") pod \"node-ca-v9stm\" (UID: \"aff3118f-d191-4990-b60b-958c0c686aec\") " pod="openshift-image-registry/node-ca-v9stm" Sep 07 10:08:05.527954 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527363 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-run\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.527954 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527390 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c06f0d60-b864-4616-82e6-3bf85f17249b-etc-tuned\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.528660 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527406 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2293164a-1a0c-49ea-95b1-d5d0b223d585-cni-binary-copy\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.528660 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527453 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-run-openvswitch\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.528660 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527486 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-host-cni-bin\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.528660 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527509 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw9qf\" (UniqueName: \"kubernetes.io/projected/36ee18c1-8608-4bf7-84d8-f94c414d02bc-kube-api-access-cw9qf\") pod \"iptables-alerter-z27cp\" (UID: \"36ee18c1-8608-4bf7-84d8-f94c414d02bc\") " pod="openshift-network-operator/iptables-alerter-z27cp" Sep 07 10:08:05.528660 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527535 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-etc-kubernetes\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.528660 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527561 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmplf\" (UniqueName: \"kubernetes.io/projected/2293164a-1a0c-49ea-95b1-d5d0b223d585-kube-api-access-hmplf\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.528660 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527582 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2293164a-1a0c-49ea-95b1-d5d0b223d585-multus-daemon-config\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.528660 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527585 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-etc-kubernetes\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.528660 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527603 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-etc-kubernetes\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.528660 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527620 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-etc-sysctl-d\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.528660 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527647 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-host-slash\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.528660 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527687 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/36ee18c1-8608-4bf7-84d8-f94c414d02bc-iptables-alerter-script\") pod \"iptables-alerter-z27cp\" (UID: \"36ee18c1-8608-4bf7-84d8-f94c414d02bc\") " pod="openshift-network-operator/iptables-alerter-z27cp" Sep 07 10:08:05.528660 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527719 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-multus-socket-dir-parent\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.528660 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527793 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs\") pod \"network-metrics-daemon-f5sf9\" (UID: \"6cae549b-87f4-4031-b984-83a7d64f9dd3\") " pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:05.528660 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527828 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b2836d86-221b-4f6e-9d5e-5f0edf93e61b-socket-dir\") pod \"aws-ebs-csi-driver-node-m4tlq\" (UID: \"b2836d86-221b-4f6e-9d5e-5f0edf93e61b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" Sep 07 10:08:05.528660 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527850 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a04783c7-15ce-4d0c-82a3-6b946c5ecda0-cni-binary-copy\") pod \"multus-additional-cni-plugins-gmbs2\" (UID: \"a04783c7-15ce-4d0c-82a3-6b946c5ecda0\") " pod="openshift-multus/multus-additional-cni-plugins-gmbs2" Sep 07 10:08:05.528660 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527875 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-etc-sysconfig\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.529437 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527891 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-multus-socket-dir-parent\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.529437 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527945 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-etc-sysctl-conf\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.529437 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.527978 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.529437 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.528000 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fe03fc91-4aa7-46e8-a9a2-e78173df653b-ovnkube-config\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.529437 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.528019 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-dbus\") pod \"global-pull-secret-syncer-llgqf\" (UID: \"5c35d143-5a49-4636-b86b-c1dd3f6df6c3\") " pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:05.529437 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.528043 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssqwl\" (UniqueName: \"kubernetes.io/projected/c06f0d60-b864-4616-82e6-3bf85f17249b-kube-api-access-ssqwl\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.529437 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.528067 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-host-run-ovn-kubernetes\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.529437 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.528108 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9e1b00d-e190-452a-8322-333f9397c2fa-tmp-dir\") pod \"node-resolver-kqh2w\" (UID: \"f9e1b00d-e190-452a-8322-333f9397c2fa\") " pod="openshift-dns/node-resolver-kqh2w" Sep 07 10:08:05.529437 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.528132 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-system-cni-dir\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.529437 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.528151 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhkpz\" (UniqueName: \"kubernetes.io/projected/b2836d86-221b-4f6e-9d5e-5f0edf93e61b-kube-api-access-qhkpz\") pod \"aws-ebs-csi-driver-node-m4tlq\" (UID: \"b2836d86-221b-4f6e-9d5e-5f0edf93e61b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" Sep 07 10:08:05.529437 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.528165 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0e3d4718-d092-48e6-86f6-f09e1793eac1-agent-certs\") pod \"konnectivity-agent-4qt9r\" (UID: \"0e3d4718-d092-48e6-86f6-f09e1793eac1\") " pod="kube-system/konnectivity-agent-4qt9r" Sep 07 10:08:05.529437 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.528214 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-lib-modules\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.529437 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.528255 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz9nt\" (UniqueName: \"kubernetes.io/projected/6cae549b-87f4-4031-b984-83a7d64f9dd3-kube-api-access-nz9nt\") pod \"network-metrics-daemon-f5sf9\" (UID: \"6cae549b-87f4-4031-b984-83a7d64f9dd3\") " pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:05.529437 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.528282 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-cnibin\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.529437 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.528294 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-system-cni-dir\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.529437 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.528305 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b2836d86-221b-4f6e-9d5e-5f0edf93e61b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m4tlq\" (UID: \"b2836d86-221b-4f6e-9d5e-5f0edf93e61b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" Sep 07 10:08:05.529437 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.528327 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-etc-systemd\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.530071 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.528350 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-sys\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.530071 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.528376 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-host-run-netns\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.530071 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.528420 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2293164a-1a0c-49ea-95b1-d5d0b223d585-cnibin\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.533531 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.533403 2552 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 07 10:08:05.537119 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.537101 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmplf\" (UniqueName: \"kubernetes.io/projected/2293164a-1a0c-49ea-95b1-d5d0b223d585-kube-api-access-hmplf\") pod \"multus-vs9qt\" (UID: \"2293164a-1a0c-49ea-95b1-d5d0b223d585\") " pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.629319 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629227 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-etc-kubernetes\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.629319 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629276 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-etc-sysctl-d\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.629319 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629291 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-host-slash\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.629319 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629306 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/36ee18c1-8608-4bf7-84d8-f94c414d02bc-iptables-alerter-script\") pod \"iptables-alerter-z27cp\" (UID: \"36ee18c1-8608-4bf7-84d8-f94c414d02bc\") " pod="openshift-network-operator/iptables-alerter-z27cp" Sep 07 10:08:05.629319 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629321 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs\") pod \"network-metrics-daemon-f5sf9\" (UID: \"6cae549b-87f4-4031-b984-83a7d64f9dd3\") " pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:05.629696 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629342 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b2836d86-221b-4f6e-9d5e-5f0edf93e61b-socket-dir\") pod \"aws-ebs-csi-driver-node-m4tlq\" (UID: \"b2836d86-221b-4f6e-9d5e-5f0edf93e61b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" Sep 07 10:08:05.629696 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629367 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a04783c7-15ce-4d0c-82a3-6b946c5ecda0-cni-binary-copy\") pod \"multus-additional-cni-plugins-gmbs2\" (UID: \"a04783c7-15ce-4d0c-82a3-6b946c5ecda0\") " pod="openshift-multus/multus-additional-cni-plugins-gmbs2" Sep 07 10:08:05.629696 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629390 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-etc-sysconfig\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.629696 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629412 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-etc-sysctl-conf\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.629696 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629413 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-etc-kubernetes\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.629696 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629436 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.629696 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629461 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fe03fc91-4aa7-46e8-a9a2-e78173df653b-ovnkube-config\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.629696 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629476 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-etc-sysconfig\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.629696 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:05.629478 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:05.629696 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629388 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-host-slash\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.629696 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629480 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-etc-sysctl-d\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.629696 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629545 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-dbus\") pod \"global-pull-secret-syncer-llgqf\" (UID: \"5c35d143-5a49-4636-b86b-c1dd3f6df6c3\") " pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:05.629696 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:05.629562 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs podName:6cae549b-87f4-4031-b984-83a7d64f9dd3 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:06.129539086 +0000 UTC m=+3.097950877 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs") pod "network-metrics-daemon-f5sf9" (UID: "6cae549b-87f4-4031-b984-83a7d64f9dd3") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:05.629696 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629483 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-dbus\") pod \"global-pull-secret-syncer-llgqf\" (UID: \"5c35d143-5a49-4636-b86b-c1dd3f6df6c3\") " pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:05.629696 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629600 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssqwl\" (UniqueName: \"kubernetes.io/projected/c06f0d60-b864-4616-82e6-3bf85f17249b-kube-api-access-ssqwl\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.629696 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629619 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b2836d86-221b-4f6e-9d5e-5f0edf93e61b-socket-dir\") pod \"aws-ebs-csi-driver-node-m4tlq\" (UID: \"b2836d86-221b-4f6e-9d5e-5f0edf93e61b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" Sep 07 10:08:05.629696 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629626 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-host-run-ovn-kubernetes\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.630583 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629631 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-etc-sysctl-conf\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.630583 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629672 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.630583 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629872 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9e1b00d-e190-452a-8322-333f9397c2fa-tmp-dir\") pod \"node-resolver-kqh2w\" (UID: \"f9e1b00d-e190-452a-8322-333f9397c2fa\") " pod="openshift-dns/node-resolver-kqh2w" Sep 07 10:08:05.630583 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629900 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhkpz\" (UniqueName: \"kubernetes.io/projected/b2836d86-221b-4f6e-9d5e-5f0edf93e61b-kube-api-access-qhkpz\") pod \"aws-ebs-csi-driver-node-m4tlq\" (UID: \"b2836d86-221b-4f6e-9d5e-5f0edf93e61b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" Sep 07 10:08:05.630583 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629924 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0e3d4718-d092-48e6-86f6-f09e1793eac1-agent-certs\") pod \"konnectivity-agent-4qt9r\" (UID: \"0e3d4718-d092-48e6-86f6-f09e1793eac1\") " pod="kube-system/konnectivity-agent-4qt9r" Sep 07 10:08:05.630583 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629954 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-lib-modules\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.630583 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.629977 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz9nt\" (UniqueName: \"kubernetes.io/projected/6cae549b-87f4-4031-b984-83a7d64f9dd3-kube-api-access-nz9nt\") pod \"network-metrics-daemon-f5sf9\" (UID: \"6cae549b-87f4-4031-b984-83a7d64f9dd3\") " pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:05.630583 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630002 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b2836d86-221b-4f6e-9d5e-5f0edf93e61b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m4tlq\" (UID: \"b2836d86-221b-4f6e-9d5e-5f0edf93e61b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" Sep 07 10:08:05.630583 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630019 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-etc-systemd\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.630583 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630035 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-sys\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.630583 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630048 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-host-run-netns\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.630583 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630066 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-etc-openvswitch\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.630583 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630081 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fe03fc91-4aa7-46e8-a9a2-e78173df653b-ovn-node-metrics-cert\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.630583 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630097 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36ee18c1-8608-4bf7-84d8-f94c414d02bc-host-slash\") pod \"iptables-alerter-z27cp\" (UID: \"36ee18c1-8608-4bf7-84d8-f94c414d02bc\") " pod="openshift-network-operator/iptables-alerter-z27cp" Sep 07 10:08:05.630583 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630111 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fe03fc91-4aa7-46e8-a9a2-e78173df653b-ovnkube-script-lib\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.630583 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630126 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9878\" (UniqueName: \"kubernetes.io/projected/fe03fc91-4aa7-46e8-a9a2-e78173df653b-kube-api-access-f9878\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.630583 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630135 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/36ee18c1-8608-4bf7-84d8-f94c414d02bc-iptables-alerter-script\") pod \"iptables-alerter-z27cp\" (UID: \"36ee18c1-8608-4bf7-84d8-f94c414d02bc\") " pod="openshift-network-operator/iptables-alerter-z27cp" Sep 07 10:08:05.631402 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630145 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a04783c7-15ce-4d0c-82a3-6b946c5ecda0-cni-binary-copy\") pod \"multus-additional-cni-plugins-gmbs2\" (UID: \"a04783c7-15ce-4d0c-82a3-6b946c5ecda0\") " pod="openshift-multus/multus-additional-cni-plugins-gmbs2" Sep 07 10:08:05.631402 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630141 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-systemd-units\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.631402 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630196 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-systemd-units\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.631402 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630190 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b2836d86-221b-4f6e-9d5e-5f0edf93e61b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m4tlq\" (UID: \"b2836d86-221b-4f6e-9d5e-5f0edf93e61b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" Sep 07 10:08:05.631402 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630216 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fe03fc91-4aa7-46e8-a9a2-e78173df653b-ovnkube-config\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.631402 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630272 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-sys\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.631402 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630283 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-etc-systemd\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.631402 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630298 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-host-run-netns\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.631402 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630334 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-etc-openvswitch\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.631402 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630380 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-lib-modules\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.631402 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630490 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9e1b00d-e190-452a-8322-333f9397c2fa-tmp-dir\") pod \"node-resolver-kqh2w\" (UID: \"f9e1b00d-e190-452a-8322-333f9397c2fa\") " pod="openshift-dns/node-resolver-kqh2w" Sep 07 10:08:05.631402 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630568 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-host-run-ovn-kubernetes\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.631402 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630614 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a04783c7-15ce-4d0c-82a3-6b946c5ecda0-system-cni-dir\") pod \"multus-additional-cni-plugins-gmbs2\" (UID: \"a04783c7-15ce-4d0c-82a3-6b946c5ecda0\") " pod="openshift-multus/multus-additional-cni-plugins-gmbs2" Sep 07 10:08:05.631402 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630640 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-log-socket\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.631402 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630677 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b2836d86-221b-4f6e-9d5e-5f0edf93e61b-registration-dir\") pod \"aws-ebs-csi-driver-node-m4tlq\" (UID: \"b2836d86-221b-4f6e-9d5e-5f0edf93e61b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" Sep 07 10:08:05.631402 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630704 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a04783c7-15ce-4d0c-82a3-6b946c5ecda0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gmbs2\" (UID: \"a04783c7-15ce-4d0c-82a3-6b946c5ecda0\") " pod="openshift-multus/multus-additional-cni-plugins-gmbs2" Sep 07 10:08:05.631402 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630730 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-var-lib-kubelet\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.632233 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630752 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-kubelet-config\") pod \"global-pull-secret-syncer-llgqf\" (UID: \"5c35d143-5a49-4636-b86b-c1dd3f6df6c3\") " pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:05.632233 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630799 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b2836d86-221b-4f6e-9d5e-5f0edf93e61b-device-dir\") pod \"aws-ebs-csi-driver-node-m4tlq\" (UID: \"b2836d86-221b-4f6e-9d5e-5f0edf93e61b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" Sep 07 10:08:05.632233 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630825 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxrwz\" (UniqueName: \"kubernetes.io/projected/a04783c7-15ce-4d0c-82a3-6b946c5ecda0-kube-api-access-sxrwz\") pod \"multus-additional-cni-plugins-gmbs2\" (UID: \"a04783c7-15ce-4d0c-82a3-6b946c5ecda0\") " pod="openshift-multus/multus-additional-cni-plugins-gmbs2" Sep 07 10:08:05.632233 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630847 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aff3118f-d191-4990-b60b-958c0c686aec-host\") pod \"node-ca-v9stm\" (UID: \"aff3118f-d191-4990-b60b-958c0c686aec\") " pod="openshift-image-registry/node-ca-v9stm" Sep 07 10:08:05.632233 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630869 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-etc-modprobe-d\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.632233 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630890 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c06f0d60-b864-4616-82e6-3bf85f17249b-tmp\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.632233 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630940 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-log-socket\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.632233 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630947 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36ee18c1-8608-4bf7-84d8-f94c414d02bc-host-slash\") pod \"iptables-alerter-z27cp\" (UID: \"36ee18c1-8608-4bf7-84d8-f94c414d02bc\") " pod="openshift-network-operator/iptables-alerter-z27cp" Sep 07 10:08:05.632233 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630973 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a04783c7-15ce-4d0c-82a3-6b946c5ecda0-system-cni-dir\") pod \"multus-additional-cni-plugins-gmbs2\" (UID: \"a04783c7-15ce-4d0c-82a3-6b946c5ecda0\") " pod="openshift-multus/multus-additional-cni-plugins-gmbs2" Sep 07 10:08:05.632233 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.630996 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-run-systemd\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.632233 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.631024 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b2836d86-221b-4f6e-9d5e-5f0edf93e61b-device-dir\") pod \"aws-ebs-csi-driver-node-m4tlq\" (UID: \"b2836d86-221b-4f6e-9d5e-5f0edf93e61b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" Sep 07 10:08:05.632233 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.631034 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fe03fc91-4aa7-46e8-a9a2-e78173df653b-env-overrides\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.632233 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.631062 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a04783c7-15ce-4d0c-82a3-6b946c5ecda0-cnibin\") pod \"multus-additional-cni-plugins-gmbs2\" (UID: \"a04783c7-15ce-4d0c-82a3-6b946c5ecda0\") " pod="openshift-multus/multus-additional-cni-plugins-gmbs2" Sep 07 10:08:05.632233 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.631089 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0e3d4718-d092-48e6-86f6-f09e1793eac1-konnectivity-ca\") pod \"konnectivity-agent-4qt9r\" (UID: \"0e3d4718-d092-48e6-86f6-f09e1793eac1\") " pod="kube-system/konnectivity-agent-4qt9r" Sep 07 10:08:05.632233 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.631150 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-kubelet-config\") pod \"global-pull-secret-syncer-llgqf\" (UID: \"5c35d143-5a49-4636-b86b-c1dd3f6df6c3\") " pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:05.632233 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.631207 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a04783c7-15ce-4d0c-82a3-6b946c5ecda0-cnibin\") pod \"multus-additional-cni-plugins-gmbs2\" (UID: \"a04783c7-15ce-4d0c-82a3-6b946c5ecda0\") " pod="openshift-multus/multus-additional-cni-plugins-gmbs2" Sep 07 10:08:05.632233 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.631236 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aff3118f-d191-4990-b60b-958c0c686aec-host\") pod \"node-ca-v9stm\" (UID: \"aff3118f-d191-4990-b60b-958c0c686aec\") " pod="openshift-image-registry/node-ca-v9stm" Sep 07 10:08:05.633070 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.631400 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-var-lib-kubelet\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.633070 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.631694 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a04783c7-15ce-4d0c-82a3-6b946c5ecda0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gmbs2\" (UID: \"a04783c7-15ce-4d0c-82a3-6b946c5ecda0\") " pod="openshift-multus/multus-additional-cni-plugins-gmbs2" Sep 07 10:08:05.633070 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.631742 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-etc-modprobe-d\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.633070 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.631754 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b2836d86-221b-4f6e-9d5e-5f0edf93e61b-registration-dir\") pod \"aws-ebs-csi-driver-node-m4tlq\" (UID: \"b2836d86-221b-4f6e-9d5e-5f0edf93e61b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" Sep 07 10:08:05.633070 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.631762 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0e3d4718-d092-48e6-86f6-f09e1793eac1-konnectivity-ca\") pod \"konnectivity-agent-4qt9r\" (UID: \"0e3d4718-d092-48e6-86f6-f09e1793eac1\") " pod="kube-system/konnectivity-agent-4qt9r" Sep 07 10:08:05.633070 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.631780 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fe03fc91-4aa7-46e8-a9a2-e78173df653b-env-overrides\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.633070 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.631794 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-run-systemd\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.633070 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.631800 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dhv8\" (UniqueName: \"kubernetes.io/projected/aff3118f-d191-4990-b60b-958c0c686aec-kube-api-access-5dhv8\") pod \"node-ca-v9stm\" (UID: \"aff3118f-d191-4990-b60b-958c0c686aec\") " pod="openshift-image-registry/node-ca-v9stm" Sep 07 10:08:05.633070 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.631826 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-host\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.633070 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.631851 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fe03fc91-4aa7-46e8-a9a2-e78173df653b-ovnkube-script-lib\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.633070 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.631857 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-host-kubelet\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.633070 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.631865 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-host\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.633070 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.631885 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f9e1b00d-e190-452a-8322-333f9397c2fa-hosts-file\") pod \"node-resolver-kqh2w\" (UID: \"f9e1b00d-e190-452a-8322-333f9397c2fa\") " pod="openshift-dns/node-resolver-kqh2w" Sep 07 10:08:05.633070 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632004 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-host-kubelet\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.633070 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632035 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w9cm\" (UniqueName: \"kubernetes.io/projected/f9e1b00d-e190-452a-8322-333f9397c2fa-kube-api-access-7w9cm\") pod \"node-resolver-kqh2w\" (UID: \"f9e1b00d-e190-452a-8322-333f9397c2fa\") " pod="openshift-dns/node-resolver-kqh2w" Sep 07 10:08:05.633070 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632051 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f9e1b00d-e190-452a-8322-333f9397c2fa-hosts-file\") pod \"node-resolver-kqh2w\" (UID: \"f9e1b00d-e190-452a-8322-333f9397c2fa\") " pod="openshift-dns/node-resolver-kqh2w" Sep 07 10:08:05.633070 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632067 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-original-pull-secret\") pod \"global-pull-secret-syncer-llgqf\" (UID: \"5c35d143-5a49-4636-b86b-c1dd3f6df6c3\") " pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:05.633949 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632174 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b2836d86-221b-4f6e-9d5e-5f0edf93e61b-etc-selinux\") pod \"aws-ebs-csi-driver-node-m4tlq\" (UID: \"b2836d86-221b-4f6e-9d5e-5f0edf93e61b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" Sep 07 10:08:05.633949 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:05.632256 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:05.633949 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632294 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a04783c7-15ce-4d0c-82a3-6b946c5ecda0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gmbs2\" (UID: \"a04783c7-15ce-4d0c-82a3-6b946c5ecda0\") " pod="openshift-multus/multus-additional-cni-plugins-gmbs2" Sep 07 10:08:05.633949 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632336 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b2836d86-221b-4f6e-9d5e-5f0edf93e61b-etc-selinux\") pod \"aws-ebs-csi-driver-node-m4tlq\" (UID: \"b2836d86-221b-4f6e-9d5e-5f0edf93e61b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" Sep 07 10:08:05.633949 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632363 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a04783c7-15ce-4d0c-82a3-6b946c5ecda0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gmbs2\" (UID: \"a04783c7-15ce-4d0c-82a3-6b946c5ecda0\") " pod="openshift-multus/multus-additional-cni-plugins-gmbs2" Sep 07 10:08:05.633949 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:05.632378 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-original-pull-secret podName:5c35d143-5a49-4636-b86b-c1dd3f6df6c3 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:06.132360117 +0000 UTC m=+3.100771905 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-original-pull-secret") pod "global-pull-secret-syncer-llgqf" (UID: "5c35d143-5a49-4636-b86b-c1dd3f6df6c3") : object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:05.633949 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632429 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b2836d86-221b-4f6e-9d5e-5f0edf93e61b-sys-fs\") pod \"aws-ebs-csi-driver-node-m4tlq\" (UID: \"b2836d86-221b-4f6e-9d5e-5f0edf93e61b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" Sep 07 10:08:05.633949 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632464 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgvsz\" (UniqueName: \"kubernetes.io/projected/1359e8cd-f362-45c5-8071-c59d23eab440-kube-api-access-cgvsz\") pod \"network-check-target-w9rwx\" (UID: \"1359e8cd-f362-45c5-8071-c59d23eab440\") " pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:05.633949 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632490 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-var-lib-openvswitch\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.633949 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632482 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a04783c7-15ce-4d0c-82a3-6b946c5ecda0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gmbs2\" (UID: \"a04783c7-15ce-4d0c-82a3-6b946c5ecda0\") " pod="openshift-multus/multus-additional-cni-plugins-gmbs2" Sep 07 10:08:05.633949 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632529 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-run-ovn\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.633949 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632535 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-var-lib-openvswitch\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.633949 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632553 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-node-log\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.633949 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632585 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b2836d86-221b-4f6e-9d5e-5f0edf93e61b-sys-fs\") pod \"aws-ebs-csi-driver-node-m4tlq\" (UID: \"b2836d86-221b-4f6e-9d5e-5f0edf93e61b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" Sep 07 10:08:05.633949 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632626 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-host-cni-netd\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.633949 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632664 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/aff3118f-d191-4990-b60b-958c0c686aec-serviceca\") pod \"node-ca-v9stm\" (UID: \"aff3118f-d191-4990-b60b-958c0c686aec\") " pod="openshift-image-registry/node-ca-v9stm" Sep 07 10:08:05.633949 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632680 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-run-ovn\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.634806 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632710 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-run\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.634806 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632736 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c06f0d60-b864-4616-82e6-3bf85f17249b-etc-tuned\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.634806 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632762 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-run-openvswitch\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.634806 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632816 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a04783c7-15ce-4d0c-82a3-6b946c5ecda0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gmbs2\" (UID: \"a04783c7-15ce-4d0c-82a3-6b946c5ecda0\") " pod="openshift-multus/multus-additional-cni-plugins-gmbs2" Sep 07 10:08:05.634806 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632918 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-host-cni-bin\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.634806 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632948 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw9qf\" (UniqueName: \"kubernetes.io/projected/36ee18c1-8608-4bf7-84d8-f94c414d02bc-kube-api-access-cw9qf\") pod \"iptables-alerter-z27cp\" (UID: \"36ee18c1-8608-4bf7-84d8-f94c414d02bc\") " pod="openshift-network-operator/iptables-alerter-z27cp" Sep 07 10:08:05.634806 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.632987 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-node-log\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.634806 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.633082 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/aff3118f-d191-4990-b60b-958c0c686aec-serviceca\") pod \"node-ca-v9stm\" (UID: \"aff3118f-d191-4990-b60b-958c0c686aec\") " pod="openshift-image-registry/node-ca-v9stm" Sep 07 10:08:05.634806 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.633112 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-run-openvswitch\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.634806 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.633140 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c06f0d60-b864-4616-82e6-3bf85f17249b-run\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.634806 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.633176 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-host-cni-netd\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.634806 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.633219 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fe03fc91-4aa7-46e8-a9a2-e78173df653b-host-cni-bin\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.634806 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.633400 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fe03fc91-4aa7-46e8-a9a2-e78173df653b-ovn-node-metrics-cert\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.634806 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.633924 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c06f0d60-b864-4616-82e6-3bf85f17249b-tmp\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.634806 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.634481 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0e3d4718-d092-48e6-86f6-f09e1793eac1-agent-certs\") pod \"konnectivity-agent-4qt9r\" (UID: \"0e3d4718-d092-48e6-86f6-f09e1793eac1\") " pod="kube-system/konnectivity-agent-4qt9r" Sep 07 10:08:05.635558 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.635537 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c06f0d60-b864-4616-82e6-3bf85f17249b-etc-tuned\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.638178 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:05.638158 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 10:08:05.638178 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:05.638181 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 10:08:05.638372 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:05.638194 2552 projected.go:196] Error preparing data for projected volume kube-api-access-cgvsz for pod openshift-network-diagnostics/network-check-target-w9rwx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:05.638372 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:05.638261 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1359e8cd-f362-45c5-8071-c59d23eab440-kube-api-access-cgvsz podName:1359e8cd-f362-45c5-8071-c59d23eab440 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:06.13822911 +0000 UTC m=+3.106640883 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cgvsz" (UniqueName: "kubernetes.io/projected/1359e8cd-f362-45c5-8071-c59d23eab440-kube-api-access-cgvsz") pod "network-check-target-w9rwx" (UID: "1359e8cd-f362-45c5-8071-c59d23eab440") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:05.638479 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.638463 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssqwl\" (UniqueName: \"kubernetes.io/projected/c06f0d60-b864-4616-82e6-3bf85f17249b-kube-api-access-ssqwl\") pod \"tuned-9rj9f\" (UID: \"c06f0d60-b864-4616-82e6-3bf85f17249b\") " pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.639750 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.639725 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9878\" (UniqueName: \"kubernetes.io/projected/fe03fc91-4aa7-46e8-a9a2-e78173df653b-kube-api-access-f9878\") pod \"ovnkube-node-lcshb\" (UID: \"fe03fc91-4aa7-46e8-a9a2-e78173df653b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.640530 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.640505 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz9nt\" (UniqueName: \"kubernetes.io/projected/6cae549b-87f4-4031-b984-83a7d64f9dd3-kube-api-access-nz9nt\") pod \"network-metrics-daemon-f5sf9\" (UID: \"6cae549b-87f4-4031-b984-83a7d64f9dd3\") " pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:05.640900 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.640870 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhkpz\" (UniqueName: \"kubernetes.io/projected/b2836d86-221b-4f6e-9d5e-5f0edf93e61b-kube-api-access-qhkpz\") pod \"aws-ebs-csi-driver-node-m4tlq\" (UID: \"b2836d86-221b-4f6e-9d5e-5f0edf93e61b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" Sep 07 10:08:05.641127 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.641104 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w9cm\" (UniqueName: \"kubernetes.io/projected/f9e1b00d-e190-452a-8322-333f9397c2fa-kube-api-access-7w9cm\") pod \"node-resolver-kqh2w\" (UID: \"f9e1b00d-e190-452a-8322-333f9397c2fa\") " pod="openshift-dns/node-resolver-kqh2w" Sep 07 10:08:05.641172 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.641149 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxrwz\" (UniqueName: \"kubernetes.io/projected/a04783c7-15ce-4d0c-82a3-6b946c5ecda0-kube-api-access-sxrwz\") pod \"multus-additional-cni-plugins-gmbs2\" (UID: \"a04783c7-15ce-4d0c-82a3-6b946c5ecda0\") " pod="openshift-multus/multus-additional-cni-plugins-gmbs2" Sep 07 10:08:05.641901 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.641881 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dhv8\" (UniqueName: \"kubernetes.io/projected/aff3118f-d191-4990-b60b-958c0c686aec-kube-api-access-5dhv8\") pod \"node-ca-v9stm\" (UID: \"aff3118f-d191-4990-b60b-958c0c686aec\") " pod="openshift-image-registry/node-ca-v9stm" Sep 07 10:08:05.642063 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.642042 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw9qf\" (UniqueName: \"kubernetes.io/projected/36ee18c1-8608-4bf7-84d8-f94c414d02bc-kube-api-access-cw9qf\") pod \"iptables-alerter-z27cp\" (UID: \"36ee18c1-8608-4bf7-84d8-f94c414d02bc\") " pod="openshift-network-operator/iptables-alerter-z27cp" Sep 07 10:08:05.734067 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.734032 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vs9qt" Sep 07 10:08:05.744166 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.744144 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" Sep 07 10:08:05.755030 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.755006 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gmbs2" Sep 07 10:08:05.761920 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.761900 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:05.768891 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.768868 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4qt9r" Sep 07 10:08:05.775597 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.775580 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kqh2w" Sep 07 10:08:05.783189 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.783172 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v9stm" Sep 07 10:08:05.791691 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.791672 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" Sep 07 10:08:05.798307 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:05.798287 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z27cp" Sep 07 10:08:06.136863 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:06.136830 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs\") pod \"network-metrics-daemon-f5sf9\" (UID: \"6cae549b-87f4-4031-b984-83a7d64f9dd3\") " pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:06.137029 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:06.136910 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-original-pull-secret\") pod \"global-pull-secret-syncer-llgqf\" (UID: \"5c35d143-5a49-4636-b86b-c1dd3f6df6c3\") " pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:06.137029 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:06.137022 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:06.137163 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:06.137084 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-original-pull-secret podName:5c35d143-5a49-4636-b86b-c1dd3f6df6c3 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:07.137066708 +0000 UTC m=+4.105478496 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-original-pull-secret") pod "global-pull-secret-syncer-llgqf" (UID: "5c35d143-5a49-4636-b86b-c1dd3f6df6c3") : object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:06.137163 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:06.137023 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:06.137163 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:06.137162 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs podName:6cae549b-87f4-4031-b984-83a7d64f9dd3 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:07.137149254 +0000 UTC m=+4.105561024 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs") pod "network-metrics-daemon-f5sf9" (UID: "6cae549b-87f4-4031-b984-83a7d64f9dd3") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:06.237475 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:06.237438 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgvsz\" (UniqueName: \"kubernetes.io/projected/1359e8cd-f362-45c5-8071-c59d23eab440-kube-api-access-cgvsz\") pod \"network-check-target-w9rwx\" (UID: \"1359e8cd-f362-45c5-8071-c59d23eab440\") " pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:06.237669 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:06.237647 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 10:08:06.237713 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:06.237679 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 10:08:06.237713 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:06.237694 2552 projected.go:196] Error preparing data for projected volume kube-api-access-cgvsz for pod openshift-network-diagnostics/network-check-target-w9rwx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:06.237780 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:06.237768 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1359e8cd-f362-45c5-8071-c59d23eab440-kube-api-access-cgvsz podName:1359e8cd-f362-45c5-8071-c59d23eab440 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:07.237748043 +0000 UTC m=+4.206159832 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cgvsz" (UniqueName: "kubernetes.io/projected/1359e8cd-f362-45c5-8071-c59d23eab440-kube-api-access-cgvsz") pod "network-check-target-w9rwx" (UID: "1359e8cd-f362-45c5-8071-c59d23eab440") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:06.460019 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:06.459931 2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 10:03:05 +0000 UTC" deadline="2028-03-24 01:11:47.950132136 +0000 UTC" Sep 07 10:08:06.460019 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:06.459969 2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13527h3m41.490167544s" Sep 07 10:08:06.541027 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:06.540850 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc06f0d60_b864_4616_82e6_3bf85f17249b.slice/crio-4c753e3ea0010f0989f4584f1fbffc5ae5acd0667b2bfcdd4c1e4b0adb19a103 WatchSource:0}: Error finding container 4c753e3ea0010f0989f4584f1fbffc5ae5acd0667b2bfcdd4c1e4b0adb19a103: Status 404 returned error can't find the container with id 4c753e3ea0010f0989f4584f1fbffc5ae5acd0667b2bfcdd4c1e4b0adb19a103 Sep 07 10:08:06.544031 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:06.544003 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2293164a_1a0c_49ea_95b1_d5d0b223d585.slice/crio-1b4786aa80818f996db7e7b395ef37ab7d15f6387b6eacc5454c09c6b0da9fd8 WatchSource:0}: Error finding container 1b4786aa80818f996db7e7b395ef37ab7d15f6387b6eacc5454c09c6b0da9fd8: Status 404 returned error can't find the container with id 1b4786aa80818f996db7e7b395ef37ab7d15f6387b6eacc5454c09c6b0da9fd8 Sep 07 10:08:06.545255 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:06.545218 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe03fc91_4aa7_46e8_a9a2_e78173df653b.slice/crio-94afe161f4896148043b069fe37317cc2d190fdb8d4dee6f9930efd951a3b97c WatchSource:0}: Error finding container 94afe161f4896148043b069fe37317cc2d190fdb8d4dee6f9930efd951a3b97c: Status 404 returned error can't find the container with id 94afe161f4896148043b069fe37317cc2d190fdb8d4dee6f9930efd951a3b97c Sep 07 10:08:06.565729 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:06.565707 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaff3118f_d191_4990_b60b_958c0c686aec.slice/crio-c804c79d6ade4ea0441d8bfa903eefb43f8159fe776e1bd65f35b460acc0e955 WatchSource:0}: Error finding container c804c79d6ade4ea0441d8bfa903eefb43f8159fe776e1bd65f35b460acc0e955: Status 404 returned error can't find the container with id c804c79d6ade4ea0441d8bfa903eefb43f8159fe776e1bd65f35b460acc0e955 Sep 07 10:08:06.566759 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:06.566739 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e3d4718_d092_48e6_86f6_f09e1793eac1.slice/crio-83427ed80c10cb876be36cc83bfc915c895ed9a19f5170ac0e8dea823a0c0a3a WatchSource:0}: Error finding container 83427ed80c10cb876be36cc83bfc915c895ed9a19f5170ac0e8dea823a0c0a3a: Status 404 returned error can't find the container with id 83427ed80c10cb876be36cc83bfc915c895ed9a19f5170ac0e8dea823a0c0a3a Sep 07 10:08:06.568002 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:06.567978 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda04783c7_15ce_4d0c_82a3_6b946c5ecda0.slice/crio-1d07dcc44f79a73a3691f1dea2577c87226553c8f59e2d59274aad05ecb54e45 WatchSource:0}: Error finding container 1d07dcc44f79a73a3691f1dea2577c87226553c8f59e2d59274aad05ecb54e45: Status 404 returned error can't find the container with id 1d07dcc44f79a73a3691f1dea2577c87226553c8f59e2d59274aad05ecb54e45 Sep 07 10:08:06.568708 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:06.568520 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9e1b00d_e190_452a_8322_333f9397c2fa.slice/crio-fb5056098fc7d10c99e3458b101feb3b21e00f987f016d1d711986fbc7a2bcd5 WatchSource:0}: Error finding container fb5056098fc7d10c99e3458b101feb3b21e00f987f016d1d711986fbc7a2bcd5: Status 404 returned error can't find the container with id fb5056098fc7d10c99e3458b101feb3b21e00f987f016d1d711986fbc7a2bcd5 Sep 07 10:08:07.144565 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:07.144474 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-original-pull-secret\") pod \"global-pull-secret-syncer-llgqf\" (UID: \"5c35d143-5a49-4636-b86b-c1dd3f6df6c3\") " pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:07.144565 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:07.144561 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs\") pod \"network-metrics-daemon-f5sf9\" (UID: \"6cae549b-87f4-4031-b984-83a7d64f9dd3\") " pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:07.144783 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:07.144751 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:07.144836 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:07.144817 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs podName:6cae549b-87f4-4031-b984-83a7d64f9dd3 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:09.144798189 +0000 UTC m=+6.113209971 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs") pod "network-metrics-daemon-f5sf9" (UID: "6cae549b-87f4-4031-b984-83a7d64f9dd3") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:07.145267 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:07.145225 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:07.145427 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:07.145294 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-original-pull-secret podName:5c35d143-5a49-4636-b86b-c1dd3f6df6c3 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:09.145278749 +0000 UTC m=+6.113690526 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-original-pull-secret") pod "global-pull-secret-syncer-llgqf" (UID: "5c35d143-5a49-4636-b86b-c1dd3f6df6c3") : object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:07.245616 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:07.245576 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgvsz\" (UniqueName: \"kubernetes.io/projected/1359e8cd-f362-45c5-8071-c59d23eab440-kube-api-access-cgvsz\") pod \"network-check-target-w9rwx\" (UID: \"1359e8cd-f362-45c5-8071-c59d23eab440\") " pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:07.245795 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:07.245777 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 10:08:07.245859 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:07.245799 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 10:08:07.245859 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:07.245812 2552 projected.go:196] Error preparing data for projected volume kube-api-access-cgvsz for pod openshift-network-diagnostics/network-check-target-w9rwx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:07.245948 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:07.245871 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1359e8cd-f362-45c5-8071-c59d23eab440-kube-api-access-cgvsz podName:1359e8cd-f362-45c5-8071-c59d23eab440 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:09.245852279 +0000 UTC m=+6.214264053 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cgvsz" (UniqueName: "kubernetes.io/projected/1359e8cd-f362-45c5-8071-c59d23eab440-kube-api-access-cgvsz") pod "network-check-target-w9rwx" (UID: "1359e8cd-f362-45c5-8071-c59d23eab440") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:07.460772 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:07.460619 2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 10:03:05 +0000 UTC" deadline="2028-06-20 19:53:58.677509628 +0000 UTC" Sep 07 10:08:07.460772 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:07.460681 2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15657h45m51.2168329s" Sep 07 10:08:07.521896 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:07.520649 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:07.521896 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:07.520779 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llgqf" podUID="5c35d143-5a49-4636-b86b-c1dd3f6df6c3" Sep 07 10:08:07.521896 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:07.521585 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:07.521896 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:07.521694 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5sf9" podUID="6cae549b-87f4-4031-b984-83a7d64f9dd3" Sep 07 10:08:07.521896 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:07.521775 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:07.521896 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:07.521842 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w9rwx" podUID="1359e8cd-f362-45c5-8071-c59d23eab440" Sep 07 10:08:07.551183 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:07.551087 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4qt9r" event={"ID":"0e3d4718-d092-48e6-86f6-f09e1793eac1","Type":"ContainerStarted","Data":"83427ed80c10cb876be36cc83bfc915c895ed9a19f5170ac0e8dea823a0c0a3a"} Sep 07 10:08:07.557340 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:07.557311 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vs9qt" event={"ID":"2293164a-1a0c-49ea-95b1-d5d0b223d585","Type":"ContainerStarted","Data":"1b4786aa80818f996db7e7b395ef37ab7d15f6387b6eacc5454c09c6b0da9fd8"} Sep 07 10:08:07.559713 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:07.559688 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" event={"ID":"c06f0d60-b864-4616-82e6-3bf85f17249b","Type":"ContainerStarted","Data":"4c753e3ea0010f0989f4584f1fbffc5ae5acd0667b2bfcdd4c1e4b0adb19a103"} Sep 07 10:08:07.563229 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:07.563207 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" event={"ID":"b2836d86-221b-4f6e-9d5e-5f0edf93e61b","Type":"ContainerStarted","Data":"769bc76edde0a3d25046be7aae94624c4509f66a0dc044fc1ff0306e6a455958"} Sep 07 10:08:07.574940 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:07.574912 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z27cp" event={"ID":"36ee18c1-8608-4bf7-84d8-f94c414d02bc","Type":"ContainerStarted","Data":"1b83659101e0f0444521b64cf0c9f5a46882df46812ca1a7e4e3bde7d3ffe087"} Sep 07 10:08:07.602191 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:07.601991 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kqh2w" event={"ID":"f9e1b00d-e190-452a-8322-333f9397c2fa","Type":"ContainerStarted","Data":"fb5056098fc7d10c99e3458b101feb3b21e00f987f016d1d711986fbc7a2bcd5"} Sep 07 10:08:07.609422 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:07.609355 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmbs2" event={"ID":"a04783c7-15ce-4d0c-82a3-6b946c5ecda0","Type":"ContainerStarted","Data":"1d07dcc44f79a73a3691f1dea2577c87226553c8f59e2d59274aad05ecb54e45"} Sep 07 10:08:07.616487 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:07.616401 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v9stm" event={"ID":"aff3118f-d191-4990-b60b-958c0c686aec","Type":"ContainerStarted","Data":"c804c79d6ade4ea0441d8bfa903eefb43f8159fe776e1bd65f35b460acc0e955"} Sep 07 10:08:07.635183 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:07.635151 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" event={"ID":"fe03fc91-4aa7-46e8-a9a2-e78173df653b","Type":"ContainerStarted","Data":"94afe161f4896148043b069fe37317cc2d190fdb8d4dee6f9930efd951a3b97c"} Sep 07 10:08:07.654958 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:07.654922 2552 generic.go:356] "Generic (PLEG): container finished" podID="615daa2f0d915b9ccc186d19a8658ab6" containerID="e91d87c00a59f229892d00065a601bd47d31657243b0f528e926548f183bbfc1" exitCode=0 Sep 07 10:08:07.655256 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:07.655216 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-225.ec2.internal" event={"ID":"615daa2f0d915b9ccc186d19a8658ab6","Type":"ContainerDied","Data":"e91d87c00a59f229892d00065a601bd47d31657243b0f528e926548f183bbfc1"} Sep 07 10:08:07.666474 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:07.666444 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-225.ec2.internal" event={"ID":"35886f0cd374d86381ddcbd8cc29c623","Type":"ContainerStarted","Data":"ff749b93e697805d4bf58687e9b147c98a28cf1caff418aa8d5cc92e7cd224e9"} Sep 07 10:08:08.717732 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:08.717396 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-225.ec2.internal" event={"ID":"615daa2f0d915b9ccc186d19a8658ab6","Type":"ContainerStarted","Data":"0ec6aee1ddabdc443620875dfc0852d0406e0b9c4db2aef737c61b5f84dcbd78"} Sep 07 10:08:08.733854 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:08.733422 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-225.ec2.internal" podStartSLOduration=4.73339996 podStartE2EDuration="4.73339996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:08:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:08:07.683362279 +0000 UTC m=+4.651774069" watchObservedRunningTime="2026-09-07 10:08:08.73339996 +0000 UTC m=+5.701811753" Sep 07 10:08:09.163718 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:09.162943 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-original-pull-secret\") pod \"global-pull-secret-syncer-llgqf\" (UID: \"5c35d143-5a49-4636-b86b-c1dd3f6df6c3\") " pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:09.163718 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:09.163021 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs\") pod \"network-metrics-daemon-f5sf9\" (UID: \"6cae549b-87f4-4031-b984-83a7d64f9dd3\") " pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:09.163718 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:09.163157 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:09.163718 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:09.163216 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs podName:6cae549b-87f4-4031-b984-83a7d64f9dd3 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:13.16319899 +0000 UTC m=+10.131610775 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs") pod "network-metrics-daemon-f5sf9" (UID: "6cae549b-87f4-4031-b984-83a7d64f9dd3") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:09.163718 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:09.163291 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:09.163718 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:09.163326 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-original-pull-secret podName:5c35d143-5a49-4636-b86b-c1dd3f6df6c3 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:13.163312552 +0000 UTC m=+10.131724327 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-original-pull-secret") pod "global-pull-secret-syncer-llgqf" (UID: "5c35d143-5a49-4636-b86b-c1dd3f6df6c3") : object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:09.264115 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:09.264080 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgvsz\" (UniqueName: \"kubernetes.io/projected/1359e8cd-f362-45c5-8071-c59d23eab440-kube-api-access-cgvsz\") pod \"network-check-target-w9rwx\" (UID: \"1359e8cd-f362-45c5-8071-c59d23eab440\") " pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:09.264447 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:09.264285 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 10:08:09.264447 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:09.264320 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 10:08:09.264447 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:09.264334 2552 projected.go:196] Error preparing data for projected volume kube-api-access-cgvsz for pod openshift-network-diagnostics/network-check-target-w9rwx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:09.264447 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:09.264398 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1359e8cd-f362-45c5-8071-c59d23eab440-kube-api-access-cgvsz podName:1359e8cd-f362-45c5-8071-c59d23eab440 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:13.26437441 +0000 UTC m=+10.232786198 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cgvsz" (UniqueName: "kubernetes.io/projected/1359e8cd-f362-45c5-8071-c59d23eab440-kube-api-access-cgvsz") pod "network-check-target-w9rwx" (UID: "1359e8cd-f362-45c5-8071-c59d23eab440") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:09.516425 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:09.516342 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:09.516425 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:09.516371 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:09.516654 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:09.516464 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w9rwx" podUID="1359e8cd-f362-45c5-8071-c59d23eab440" Sep 07 10:08:09.516654 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:09.516491 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:09.516654 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:09.516600 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5sf9" podUID="6cae549b-87f4-4031-b984-83a7d64f9dd3" Sep 07 10:08:09.516980 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:09.516698 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llgqf" podUID="5c35d143-5a49-4636-b86b-c1dd3f6df6c3" Sep 07 10:08:11.518473 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:11.518442 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:11.518941 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:11.518571 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llgqf" podUID="5c35d143-5a49-4636-b86b-c1dd3f6df6c3" Sep 07 10:08:11.519010 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:11.518967 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:11.519230 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:11.519069 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5sf9" podUID="6cae549b-87f4-4031-b984-83a7d64f9dd3" Sep 07 10:08:11.519230 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:11.519129 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:11.519230 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:11.519199 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w9rwx" podUID="1359e8cd-f362-45c5-8071-c59d23eab440" Sep 07 10:08:13.200062 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:13.200029 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs\") pod \"network-metrics-daemon-f5sf9\" (UID: \"6cae549b-87f4-4031-b984-83a7d64f9dd3\") " pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:13.200556 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:13.200105 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-original-pull-secret\") pod \"global-pull-secret-syncer-llgqf\" (UID: \"5c35d143-5a49-4636-b86b-c1dd3f6df6c3\") " pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:13.200556 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:13.200263 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:13.200556 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:13.200329 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-original-pull-secret podName:5c35d143-5a49-4636-b86b-c1dd3f6df6c3 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:21.200310509 +0000 UTC m=+18.168722293 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-original-pull-secret") pod "global-pull-secret-syncer-llgqf" (UID: "5c35d143-5a49-4636-b86b-c1dd3f6df6c3") : object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:13.200761 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:13.200734 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:13.200877 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:13.200801 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs podName:6cae549b-87f4-4031-b984-83a7d64f9dd3 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:21.200782605 +0000 UTC m=+18.169194381 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs") pod "network-metrics-daemon-f5sf9" (UID: "6cae549b-87f4-4031-b984-83a7d64f9dd3") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:13.301001 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:13.300963 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgvsz\" (UniqueName: \"kubernetes.io/projected/1359e8cd-f362-45c5-8071-c59d23eab440-kube-api-access-cgvsz\") pod \"network-check-target-w9rwx\" (UID: \"1359e8cd-f362-45c5-8071-c59d23eab440\") " pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:13.301157 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:13.301127 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 10:08:13.301157 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:13.301145 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 10:08:13.301157 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:13.301158 2552 projected.go:196] Error preparing data for projected volume kube-api-access-cgvsz for pod openshift-network-diagnostics/network-check-target-w9rwx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:13.301320 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:13.301223 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1359e8cd-f362-45c5-8071-c59d23eab440-kube-api-access-cgvsz podName:1359e8cd-f362-45c5-8071-c59d23eab440 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:21.301204096 +0000 UTC m=+18.269615884 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cgvsz" (UniqueName: "kubernetes.io/projected/1359e8cd-f362-45c5-8071-c59d23eab440-kube-api-access-cgvsz") pod "network-check-target-w9rwx" (UID: "1359e8cd-f362-45c5-8071-c59d23eab440") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:13.517990 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:13.517702 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:13.518149 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:13.518040 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llgqf" podUID="5c35d143-5a49-4636-b86b-c1dd3f6df6c3" Sep 07 10:08:13.518604 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:13.518446 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:13.518604 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:13.518557 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5sf9" podUID="6cae549b-87f4-4031-b984-83a7d64f9dd3" Sep 07 10:08:13.518973 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:13.518847 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:13.518973 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:13.518940 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w9rwx" podUID="1359e8cd-f362-45c5-8071-c59d23eab440" Sep 07 10:08:15.516340 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:15.516302 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:15.516340 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:15.516324 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:15.516813 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:15.516306 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:15.516813 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:15.516428 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w9rwx" podUID="1359e8cd-f362-45c5-8071-c59d23eab440" Sep 07 10:08:15.516813 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:15.516516 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llgqf" podUID="5c35d143-5a49-4636-b86b-c1dd3f6df6c3" Sep 07 10:08:15.516813 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:15.516612 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5sf9" podUID="6cae549b-87f4-4031-b984-83a7d64f9dd3" Sep 07 10:08:17.516962 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:17.516930 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:17.517447 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:17.517058 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:17.517447 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:17.516936 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:17.517447 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:17.517158 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w9rwx" podUID="1359e8cd-f362-45c5-8071-c59d23eab440" Sep 07 10:08:17.517447 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:17.517059 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llgqf" podUID="5c35d143-5a49-4636-b86b-c1dd3f6df6c3" Sep 07 10:08:17.517447 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:17.517288 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5sf9" podUID="6cae549b-87f4-4031-b984-83a7d64f9dd3" Sep 07 10:08:19.516183 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:19.516143 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:19.516183 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:19.516190 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:19.516667 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:19.516143 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:19.516667 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:19.516284 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w9rwx" podUID="1359e8cd-f362-45c5-8071-c59d23eab440" Sep 07 10:08:19.516667 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:19.516432 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llgqf" podUID="5c35d143-5a49-4636-b86b-c1dd3f6df6c3" Sep 07 10:08:19.516667 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:19.516560 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5sf9" podUID="6cae549b-87f4-4031-b984-83a7d64f9dd3" Sep 07 10:08:21.260352 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:21.260321 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs\") pod \"network-metrics-daemon-f5sf9\" (UID: \"6cae549b-87f4-4031-b984-83a7d64f9dd3\") " pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:21.260853 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:21.260402 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-original-pull-secret\") pod \"global-pull-secret-syncer-llgqf\" (UID: \"5c35d143-5a49-4636-b86b-c1dd3f6df6c3\") " pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:21.260853 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:21.260468 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:21.260853 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:21.260532 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:21.260853 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:21.260539 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs podName:6cae549b-87f4-4031-b984-83a7d64f9dd3 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:37.260518609 +0000 UTC m=+34.228930380 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs") pod "network-metrics-daemon-f5sf9" (UID: "6cae549b-87f4-4031-b984-83a7d64f9dd3") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:21.260853 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:21.260602 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-original-pull-secret podName:5c35d143-5a49-4636-b86b-c1dd3f6df6c3 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:37.260585415 +0000 UTC m=+34.228997193 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-original-pull-secret") pod "global-pull-secret-syncer-llgqf" (UID: "5c35d143-5a49-4636-b86b-c1dd3f6df6c3") : object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:21.361697 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:21.361665 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgvsz\" (UniqueName: \"kubernetes.io/projected/1359e8cd-f362-45c5-8071-c59d23eab440-kube-api-access-cgvsz\") pod \"network-check-target-w9rwx\" (UID: \"1359e8cd-f362-45c5-8071-c59d23eab440\") " pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:21.361878 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:21.361840 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 10:08:21.361878 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:21.361861 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 10:08:21.361878 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:21.361875 2552 projected.go:196] Error preparing data for projected volume kube-api-access-cgvsz for pod openshift-network-diagnostics/network-check-target-w9rwx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:21.362033 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:21.361963 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1359e8cd-f362-45c5-8071-c59d23eab440-kube-api-access-cgvsz podName:1359e8cd-f362-45c5-8071-c59d23eab440 nodeName:}" failed. No retries permitted until 2026-09-07 10:08:37.361945673 +0000 UTC m=+34.330357461 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cgvsz" (UniqueName: "kubernetes.io/projected/1359e8cd-f362-45c5-8071-c59d23eab440-kube-api-access-cgvsz") pod "network-check-target-w9rwx" (UID: "1359e8cd-f362-45c5-8071-c59d23eab440") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:21.516755 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:21.516660 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:21.516922 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:21.516660 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:21.516922 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:21.516793 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:21.516922 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:21.516786 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w9rwx" podUID="1359e8cd-f362-45c5-8071-c59d23eab440" Sep 07 10:08:21.516922 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:21.516890 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llgqf" podUID="5c35d143-5a49-4636-b86b-c1dd3f6df6c3" Sep 07 10:08:21.517149 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:21.516992 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5sf9" podUID="6cae549b-87f4-4031-b984-83a7d64f9dd3" Sep 07 10:08:23.516857 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:23.516825 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:23.517376 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:23.516938 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:23.517505 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:23.517388 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llgqf" podUID="5c35d143-5a49-4636-b86b-c1dd3f6df6c3" Sep 07 10:08:23.517505 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:23.516970 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:23.517617 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:23.517485 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5sf9" podUID="6cae549b-87f4-4031-b984-83a7d64f9dd3" Sep 07 10:08:23.517617 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:23.517534 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w9rwx" podUID="1359e8cd-f362-45c5-8071-c59d23eab440" Sep 07 10:08:24.750643 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:24.750379 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" event={"ID":"b2836d86-221b-4f6e-9d5e-5f0edf93e61b","Type":"ContainerStarted","Data":"75885f519578799255d610f6df19f527f5ef4c22514756055ccb5d5c9913c1ee"} Sep 07 10:08:24.752192 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:24.752006 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kqh2w" event={"ID":"f9e1b00d-e190-452a-8322-333f9397c2fa","Type":"ContainerStarted","Data":"400b3447f3135b4af3056b6a6cfe62338711a928f546418033a78ae0f754e4a3"} Sep 07 10:08:24.753830 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:24.753802 2552 generic.go:356] "Generic (PLEG): container finished" podID="a04783c7-15ce-4d0c-82a3-6b946c5ecda0" containerID="bbeeba07002b3d1a80da6fa8ff27261630dd1d1017b9612d8bc3130fb15b4a30" exitCode=0 Sep 07 10:08:24.753914 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:24.753891 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmbs2" event={"ID":"a04783c7-15ce-4d0c-82a3-6b946c5ecda0","Type":"ContainerDied","Data":"bbeeba07002b3d1a80da6fa8ff27261630dd1d1017b9612d8bc3130fb15b4a30"} Sep 07 10:08:24.756068 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:24.756043 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v9stm" event={"ID":"aff3118f-d191-4990-b60b-958c0c686aec","Type":"ContainerStarted","Data":"a424567ef932cfc6c3fd15430d6267dc682bef273554eca0857f77b1e232be35"} Sep 07 10:08:24.759637 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:24.759619 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/ovn-acl-logging/0.log" Sep 07 10:08:24.760305 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:24.760272 2552 generic.go:356] "Generic (PLEG): container finished" podID="fe03fc91-4aa7-46e8-a9a2-e78173df653b" containerID="2a825b27906d49fccf687993f264cb819b6c93772fb9a204b74bea5e9255e3e6" exitCode=1 Sep 07 10:08:24.760452 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:24.760328 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" event={"ID":"fe03fc91-4aa7-46e8-a9a2-e78173df653b","Type":"ContainerStarted","Data":"20a6399f8079b66f4d6921876016f8580a9e839ba4964e5866b25d6c1089a09f"} Sep 07 10:08:24.760452 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:24.760419 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" event={"ID":"fe03fc91-4aa7-46e8-a9a2-e78173df653b","Type":"ContainerStarted","Data":"a18833fa10469596bbe358511242c2cb619fa30adb2436e6a54bb7d618d5708f"} Sep 07 10:08:24.760562 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:24.760472 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" event={"ID":"fe03fc91-4aa7-46e8-a9a2-e78173df653b","Type":"ContainerStarted","Data":"619c45f90697709f35a81ea4956ae30ca25756ccf350c830ad27a761cbf97d59"} Sep 07 10:08:24.760562 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:24.760521 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" event={"ID":"fe03fc91-4aa7-46e8-a9a2-e78173df653b","Type":"ContainerDied","Data":"2a825b27906d49fccf687993f264cb819b6c93772fb9a204b74bea5e9255e3e6"} Sep 07 10:08:24.760666 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:24.760581 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" event={"ID":"fe03fc91-4aa7-46e8-a9a2-e78173df653b","Type":"ContainerStarted","Data":"a9cf3324cb422e060c477d8f86e69f99215ec9d8a587f33473f518568fe152f2"} Sep 07 10:08:24.761958 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:24.761927 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4qt9r" event={"ID":"0e3d4718-d092-48e6-86f6-f09e1793eac1","Type":"ContainerStarted","Data":"3aaa06fd67397cef5307462f0e7b3f74ea7cc404ecc2eab7cb2d383629d840fb"} Sep 07 10:08:24.763549 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:24.763521 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vs9qt" event={"ID":"2293164a-1a0c-49ea-95b1-d5d0b223d585","Type":"ContainerStarted","Data":"e05494ddd62a2a2456396fb539a9a5441e63f23f80c2e7986c7f2e2ea6624d14"} Sep 07 10:08:24.765211 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:24.765187 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" event={"ID":"c06f0d60-b864-4616-82e6-3bf85f17249b","Type":"ContainerStarted","Data":"39d7c8e804e119d802067b29167cce1c12e6f3bf00c1d212c83c3fc1a99ad554"} Sep 07 10:08:24.766889 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:24.766846 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kqh2w" podStartSLOduration=4.245068213 podStartE2EDuration="21.766833597s" totalImagesPullingTime="17.521765384s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:08:24.766698767 +0000 UTC m=+21.735110581" watchObservedRunningTime="2026-09-07 10:08:24.766833597 +0000 UTC m=+21.735245390" Sep 07 10:08:24.767394 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:24.767359 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-225.ec2.internal" podStartSLOduration=20.767350564 podStartE2EDuration="20.767350564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:08:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:08:08.733807002 +0000 UTC m=+5.702218797" watchObservedRunningTime="2026-09-07 10:08:24.767350564 +0000 UTC m=+21.735762357" Sep 07 10:08:24.792767 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:24.792714 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vs9qt" podStartSLOduration=3.802909231 podStartE2EDuration="21.792694849s" totalImagesPullingTime="17.989785618s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:08:24.792062552 +0000 UTC m=+21.760474344" watchObservedRunningTime="2026-09-07 10:08:24.792694849 +0000 UTC m=+21.761106646" Sep 07 10:08:24.811480 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:24.811432 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4qt9r" podStartSLOduration=7.942303892 podStartE2EDuration="21.811416231s" totalImagesPullingTime="13.869112339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:08:24.8108219 +0000 UTC m=+21.779233694" watchObservedRunningTime="2026-09-07 10:08:24.811416231 +0000 UTC m=+21.779828024" Sep 07 10:08:24.829674 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:24.829621 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-v9stm" podStartSLOduration=4.25617599 podStartE2EDuration="21.829599s" totalImagesPullingTime="17.57342301s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:08:24.829011327 +0000 UTC m=+21.797423120" watchObservedRunningTime="2026-09-07 10:08:24.829599 +0000 UTC m=+21.798010795" Sep 07 10:08:24.881517 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:24.881440 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4qt9r" Sep 07 10:08:24.881981 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:24.881964 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4qt9r" Sep 07 10:08:24.905135 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:24.905077 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9rj9f" podStartSLOduration=4.30529547 podStartE2EDuration="21.905057102s" totalImagesPullingTime="17.599761632s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:08:24.87835581 +0000 UTC m=+21.846767604" watchObservedRunningTime="2026-09-07 10:08:24.905057102 +0000 UTC m=+21.873468896" Sep 07 10:08:25.516722 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:25.515978 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:25.516722 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:25.516109 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w9rwx" podUID="1359e8cd-f362-45c5-8071-c59d23eab440" Sep 07 10:08:25.516722 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:25.516460 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:25.516722 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:25.516556 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llgqf" podUID="5c35d143-5a49-4636-b86b-c1dd3f6df6c3" Sep 07 10:08:25.516722 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:25.516618 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:25.516722 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:25.516690 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5sf9" podUID="6cae549b-87f4-4031-b984-83a7d64f9dd3" Sep 07 10:08:25.602679 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:25.602509 2552 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 07 10:08:25.771464 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:25.771362 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/ovn-acl-logging/0.log" Sep 07 10:08:25.771992 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:25.771810 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" event={"ID":"fe03fc91-4aa7-46e8-a9a2-e78173df653b","Type":"ContainerStarted","Data":"e96521f0725cff7673ce745b5090396b12a42202961ad1d4bdb427ce1026f58d"} Sep 07 10:08:25.773900 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:25.773869 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" event={"ID":"b2836d86-221b-4f6e-9d5e-5f0edf93e61b","Type":"ContainerStarted","Data":"04c0205c0b86cd3ba1eabb11c47b536d910f8d14dcf1944c155d64143482d6eb"} Sep 07 10:08:25.775489 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:25.775461 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z27cp" event={"ID":"36ee18c1-8608-4bf7-84d8-f94c414d02bc","Type":"ContainerStarted","Data":"d72ce315a0aac85b8aff60f4daca3e65310aaceec041192eeab1abdd3a4f4968"} Sep 07 10:08:25.793195 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:25.793130 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-z27cp" podStartSLOduration=5.220962071 podStartE2EDuration="22.793110561s" totalImagesPullingTime="17.57214849s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:08:25.792082689 +0000 UTC m=+22.760494482" watchObservedRunningTime="2026-09-07 10:08:25.793110561 +0000 UTC m=+22.761522355" Sep 07 10:08:26.487736 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:26.487618 2552 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-07T10:08:25.602675589Z","UUID":"f3fd1f4b-965d-4f35-9561-ab93005f19a4","Handler":null,"Name":"","Endpoint":""} Sep 07 10:08:26.489519 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:26.489483 2552 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 07 10:08:26.489519 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:26.489521 2552 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 07 10:08:26.781134 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:26.781114 2552 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 07 10:08:27.516103 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:27.516071 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:27.516291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:27.516075 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:27.516291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:27.516206 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:27.516291 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:27.516202 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w9rwx" podUID="1359e8cd-f362-45c5-8071-c59d23eab440" Sep 07 10:08:27.516291 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:27.516285 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llgqf" podUID="5c35d143-5a49-4636-b86b-c1dd3f6df6c3" Sep 07 10:08:27.516514 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:27.516343 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5sf9" podUID="6cae549b-87f4-4031-b984-83a7d64f9dd3" Sep 07 10:08:27.787659 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:27.787574 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/ovn-acl-logging/0.log" Sep 07 10:08:27.788218 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:27.788182 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" event={"ID":"fe03fc91-4aa7-46e8-a9a2-e78173df653b","Type":"ContainerStarted","Data":"6a9ab8c935e9277a2fe60506b514a8824fa8c5d37f4e2462efad4194b186e3ad"} Sep 07 10:08:27.790713 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:27.790686 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" event={"ID":"b2836d86-221b-4f6e-9d5e-5f0edf93e61b","Type":"ContainerStarted","Data":"d32daa211116ea320f614546d281d7b2d3329262ef96e6e158fce70c739d3487"} Sep 07 10:08:29.517027 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:29.516852 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:29.517547 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:29.516909 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:29.517547 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:29.517118 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llgqf" podUID="5c35d143-5a49-4636-b86b-c1dd3f6df6c3" Sep 07 10:08:29.517547 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:29.516958 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:29.517547 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:29.517182 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w9rwx" podUID="1359e8cd-f362-45c5-8071-c59d23eab440" Sep 07 10:08:29.517547 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:29.517307 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5sf9" podUID="6cae549b-87f4-4031-b984-83a7d64f9dd3" Sep 07 10:08:29.799367 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:29.799335 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/ovn-acl-logging/0.log" Sep 07 10:08:29.799778 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:29.799752 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" event={"ID":"fe03fc91-4aa7-46e8-a9a2-e78173df653b","Type":"ContainerStarted","Data":"36ce6728890176cf44cdb5311f25aa73fabcc42b04e7d61f43a312cf94f9d1fe"} Sep 07 10:08:29.800014 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:29.799984 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:29.800198 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:29.800181 2552 scope.go:117] "RemoveContainer" containerID="2a825b27906d49fccf687993f264cb819b6c93772fb9a204b74bea5e9255e3e6" Sep 07 10:08:29.801751 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:29.801731 2552 generic.go:356] "Generic (PLEG): container finished" podID="a04783c7-15ce-4d0c-82a3-6b946c5ecda0" containerID="93ea00cb3b6fa1404cfeab2283b612e9e2c43c8d00abea57680bc52751cc87da" exitCode=0 Sep 07 10:08:29.801840 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:29.801760 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmbs2" event={"ID":"a04783c7-15ce-4d0c-82a3-6b946c5ecda0","Type":"ContainerDied","Data":"93ea00cb3b6fa1404cfeab2283b612e9e2c43c8d00abea57680bc52751cc87da"} Sep 07 10:08:29.815619 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:29.815600 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:29.826627 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:29.826587 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m4tlq" podStartSLOduration=7.042358899 podStartE2EDuration="26.82657451s" totalImagesPullingTime="19.784215611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:08:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:08:27.807719994 +0000 UTC m=+24.776131788" watchObservedRunningTime="2026-09-07 10:08:29.82657451 +0000 UTC m=+26.794986302" Sep 07 10:08:29.872586 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:29.872555 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4qt9r" Sep 07 10:08:29.872722 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:29.872707 2552 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 07 10:08:29.873083 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:29.873069 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4qt9r" Sep 07 10:08:30.808345 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:30.808320 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/ovn-acl-logging/0.log" Sep 07 10:08:30.808999 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:30.808766 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" event={"ID":"fe03fc91-4aa7-46e8-a9a2-e78173df653b","Type":"ContainerStarted","Data":"5f9556e757fcaea1a0e7a25610e94babbe9c421f5c992cd301e275c56cdaedd9"} Sep 07 10:08:30.808999 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:30.808787 2552 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 07 10:08:30.808999 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:30.808887 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:30.823342 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:30.823315 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:30.836880 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:30.836836 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" podStartSLOduration=10.238369828 podStartE2EDuration="27.836821976s" totalImagesPullingTime="17.598452148s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:08:30.83561556 +0000 UTC m=+27.804027387" watchObservedRunningTime="2026-09-07 10:08:30.836821976 +0000 UTC m=+27.805233769" Sep 07 10:08:30.995848 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:30.995819 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-llgqf"] Sep 07 10:08:30.995973 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:30.995942 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:30.996079 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:30.996052 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llgqf" podUID="5c35d143-5a49-4636-b86b-c1dd3f6df6c3" Sep 07 10:08:30.998800 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:30.998484 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-f5sf9"] Sep 07 10:08:30.998800 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:30.998614 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:30.998800 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:30.998727 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5sf9" podUID="6cae549b-87f4-4031-b984-83a7d64f9dd3" Sep 07 10:08:30.999235 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:30.999212 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-w9rwx"] Sep 07 10:08:30.999371 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:30.999333 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:30.999456 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:30.999421 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w9rwx" podUID="1359e8cd-f362-45c5-8071-c59d23eab440" Sep 07 10:08:31.813616 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:31.813420 2552 generic.go:356] "Generic (PLEG): container finished" podID="a04783c7-15ce-4d0c-82a3-6b946c5ecda0" containerID="38d6d6ced4001319d68a9b67bfe5233d0b5687a0cfe821408b7311f21d76cfd8" exitCode=0 Sep 07 10:08:31.814072 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:31.813500 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmbs2" event={"ID":"a04783c7-15ce-4d0c-82a3-6b946c5ecda0","Type":"ContainerDied","Data":"38d6d6ced4001319d68a9b67bfe5233d0b5687a0cfe821408b7311f21d76cfd8"} Sep 07 10:08:31.814072 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:31.813745 2552 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 07 10:08:32.516881 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:32.516840 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:32.516881 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:32.516840 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:32.516881 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:32.516874 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:32.517182 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:32.516973 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llgqf" podUID="5c35d143-5a49-4636-b86b-c1dd3f6df6c3" Sep 07 10:08:32.517182 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:32.517081 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w9rwx" podUID="1359e8cd-f362-45c5-8071-c59d23eab440" Sep 07 10:08:32.517182 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:32.517158 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5sf9" podUID="6cae549b-87f4-4031-b984-83a7d64f9dd3" Sep 07 10:08:32.816456 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:32.816385 2552 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 07 10:08:32.842190 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:32.842166 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:08:33.821452 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:33.821422 2552 generic.go:356] "Generic (PLEG): container finished" podID="a04783c7-15ce-4d0c-82a3-6b946c5ecda0" containerID="96f541105d0e7c41901bf6a994bd425f6a5e60e4752ab4db7de97d9c08b63d0f" exitCode=0 Sep 07 10:08:33.821773 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:33.821552 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmbs2" event={"ID":"a04783c7-15ce-4d0c-82a3-6b946c5ecda0","Type":"ContainerDied","Data":"96f541105d0e7c41901bf6a994bd425f6a5e60e4752ab4db7de97d9c08b63d0f"} Sep 07 10:08:34.516501 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:34.516466 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:34.516501 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:34.516504 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:34.516772 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:34.516517 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:34.516772 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:34.516627 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w9rwx" podUID="1359e8cd-f362-45c5-8071-c59d23eab440" Sep 07 10:08:34.516772 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:34.516720 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5sf9" podUID="6cae549b-87f4-4031-b984-83a7d64f9dd3" Sep 07 10:08:34.516952 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:34.516813 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llgqf" podUID="5c35d143-5a49-4636-b86b-c1dd3f6df6c3" Sep 07 10:08:36.516776 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:36.516743 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:36.517216 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:36.516744 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:36.517216 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:36.516879 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-llgqf" podUID="5c35d143-5a49-4636-b86b-c1dd3f6df6c3" Sep 07 10:08:36.517216 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:36.516756 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:36.517216 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:36.516967 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w9rwx" podUID="1359e8cd-f362-45c5-8071-c59d23eab440" Sep 07 10:08:36.517216 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:36.517038 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f5sf9" podUID="6cae549b-87f4-4031-b984-83a7d64f9dd3" Sep 07 10:08:37.277795 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.277760 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-original-pull-secret\") pod \"global-pull-secret-syncer-llgqf\" (UID: \"5c35d143-5a49-4636-b86b-c1dd3f6df6c3\") " pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:37.277954 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.277823 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs\") pod \"network-metrics-daemon-f5sf9\" (UID: \"6cae549b-87f4-4031-b984-83a7d64f9dd3\") " pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:37.277954 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:37.277919 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:37.277954 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:37.277937 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:37.278117 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:37.277982 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-original-pull-secret podName:5c35d143-5a49-4636-b86b-c1dd3f6df6c3 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:09.277964245 +0000 UTC m=+66.246376035 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-original-pull-secret") pod "global-pull-secret-syncer-llgqf" (UID: "5c35d143-5a49-4636-b86b-c1dd3f6df6c3") : object "kube-system"/"original-pull-secret" not registered Sep 07 10:08:37.278117 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:37.277997 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs podName:6cae549b-87f4-4031-b984-83a7d64f9dd3 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:09.277990226 +0000 UTC m=+66.246401997 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs") pod "network-metrics-daemon-f5sf9" (UID: "6cae549b-87f4-4031-b984-83a7d64f9dd3") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 10:08:37.313379 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.313355 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-225.ec2.internal" event="NodeReady" Sep 07 10:08:37.313532 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.313504 2552 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 07 10:08:37.358047 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.358014 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-p5xgx"] Sep 07 10:08:37.378824 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.378712 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgvsz\" (UniqueName: \"kubernetes.io/projected/1359e8cd-f362-45c5-8071-c59d23eab440-kube-api-access-cgvsz\") pod \"network-check-target-w9rwx\" (UID: \"1359e8cd-f362-45c5-8071-c59d23eab440\") " pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:37.378966 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:37.378934 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 10:08:37.378966 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:37.378960 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 10:08:37.379189 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:37.378973 2552 projected.go:196] Error preparing data for projected volume kube-api-access-cgvsz for pod openshift-network-diagnostics/network-check-target-w9rwx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:37.379189 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:37.379032 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1359e8cd-f362-45c5-8071-c59d23eab440-kube-api-access-cgvsz podName:1359e8cd-f362-45c5-8071-c59d23eab440 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:09.379014988 +0000 UTC m=+66.347426779 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-cgvsz" (UniqueName: "kubernetes.io/projected/1359e8cd-f362-45c5-8071-c59d23eab440-kube-api-access-cgvsz") pod "network-check-target-w9rwx" (UID: "1359e8cd-f362-45c5-8071-c59d23eab440") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 10:08:37.392497 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.392474 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-cdkr7"] Sep 07 10:08:37.392638 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.392620 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p5xgx" Sep 07 10:08:37.395483 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.395458 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 07 10:08:37.395749 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.395719 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 07 10:08:37.395839 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.395723 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5dlqq\"" Sep 07 10:08:37.408470 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.408375 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-p5xgx"] Sep 07 10:08:37.408470 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.408414 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cdkr7"] Sep 07 10:08:37.408637 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.408503 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cdkr7" Sep 07 10:08:37.410996 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.410974 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 07 10:08:37.410996 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.410991 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 07 10:08:37.411163 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.411014 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-926dj\"" Sep 07 10:08:37.411219 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.411193 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 07 10:08:37.479657 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.479620 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/57c1502d-f287-4126-8d7a-ab800b49d84e-config-volume\") pod \"dns-default-p5xgx\" (UID: \"57c1502d-f287-4126-8d7a-ab800b49d84e\") " pod="openshift-dns/dns-default-p5xgx" Sep 07 10:08:37.479845 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.479698 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j2qj\" (UniqueName: \"kubernetes.io/projected/24404e76-4654-44b0-ab31-06822ef18722-kube-api-access-4j2qj\") pod \"ingress-canary-cdkr7\" (UID: \"24404e76-4654-44b0-ab31-06822ef18722\") " pod="openshift-ingress-canary/ingress-canary-cdkr7" Sep 07 10:08:37.479845 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.479762 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls\") pod \"dns-default-p5xgx\" (UID: \"57c1502d-f287-4126-8d7a-ab800b49d84e\") " pod="openshift-dns/dns-default-p5xgx" Sep 07 10:08:37.479845 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.479803 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24404e76-4654-44b0-ab31-06822ef18722-cert\") pod \"ingress-canary-cdkr7\" (UID: \"24404e76-4654-44b0-ab31-06822ef18722\") " pod="openshift-ingress-canary/ingress-canary-cdkr7" Sep 07 10:08:37.479993 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.479871 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57c1502d-f287-4126-8d7a-ab800b49d84e-tmp-dir\") pod \"dns-default-p5xgx\" (UID: \"57c1502d-f287-4126-8d7a-ab800b49d84e\") " pod="openshift-dns/dns-default-p5xgx" Sep 07 10:08:37.479993 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.479912 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cjq9\" (UniqueName: \"kubernetes.io/projected/57c1502d-f287-4126-8d7a-ab800b49d84e-kube-api-access-6cjq9\") pod \"dns-default-p5xgx\" (UID: \"57c1502d-f287-4126-8d7a-ab800b49d84e\") " pod="openshift-dns/dns-default-p5xgx" Sep 07 10:08:37.580459 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.580376 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls\") pod \"dns-default-p5xgx\" (UID: \"57c1502d-f287-4126-8d7a-ab800b49d84e\") " pod="openshift-dns/dns-default-p5xgx" Sep 07 10:08:37.580459 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.580417 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24404e76-4654-44b0-ab31-06822ef18722-cert\") pod \"ingress-canary-cdkr7\" (UID: \"24404e76-4654-44b0-ab31-06822ef18722\") " pod="openshift-ingress-canary/ingress-canary-cdkr7" Sep 07 10:08:37.580459 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.580435 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57c1502d-f287-4126-8d7a-ab800b49d84e-tmp-dir\") pod \"dns-default-p5xgx\" (UID: \"57c1502d-f287-4126-8d7a-ab800b49d84e\") " pod="openshift-dns/dns-default-p5xgx" Sep 07 10:08:37.580459 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.580456 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cjq9\" (UniqueName: \"kubernetes.io/projected/57c1502d-f287-4126-8d7a-ab800b49d84e-kube-api-access-6cjq9\") pod \"dns-default-p5xgx\" (UID: \"57c1502d-f287-4126-8d7a-ab800b49d84e\") " pod="openshift-dns/dns-default-p5xgx" Sep 07 10:08:37.581010 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.580514 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/57c1502d-f287-4126-8d7a-ab800b49d84e-config-volume\") pod \"dns-default-p5xgx\" (UID: \"57c1502d-f287-4126-8d7a-ab800b49d84e\") " pod="openshift-dns/dns-default-p5xgx" Sep 07 10:08:37.581010 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:37.580548 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 10:08:37.581010 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.580573 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4j2qj\" (UniqueName: \"kubernetes.io/projected/24404e76-4654-44b0-ab31-06822ef18722-kube-api-access-4j2qj\") pod \"ingress-canary-cdkr7\" (UID: \"24404e76-4654-44b0-ab31-06822ef18722\") " pod="openshift-ingress-canary/ingress-canary-cdkr7" Sep 07 10:08:37.581010 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:37.580635 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls podName:57c1502d-f287-4126-8d7a-ab800b49d84e nodeName:}" failed. No retries permitted until 2026-09-07 10:08:38.080612052 +0000 UTC m=+35.049023848 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls") pod "dns-default-p5xgx" (UID: "57c1502d-f287-4126-8d7a-ab800b49d84e") : secret "dns-default-metrics-tls" not found Sep 07 10:08:37.581010 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.580792 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57c1502d-f287-4126-8d7a-ab800b49d84e-tmp-dir\") pod \"dns-default-p5xgx\" (UID: \"57c1502d-f287-4126-8d7a-ab800b49d84e\") " pod="openshift-dns/dns-default-p5xgx" Sep 07 10:08:37.581339 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.581314 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/57c1502d-f287-4126-8d7a-ab800b49d84e-config-volume\") pod \"dns-default-p5xgx\" (UID: \"57c1502d-f287-4126-8d7a-ab800b49d84e\") " pod="openshift-dns/dns-default-p5xgx" Sep 07 10:08:37.584837 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.584689 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24404e76-4654-44b0-ab31-06822ef18722-cert\") pod \"ingress-canary-cdkr7\" (UID: \"24404e76-4654-44b0-ab31-06822ef18722\") " pod="openshift-ingress-canary/ingress-canary-cdkr7" Sep 07 10:08:37.589546 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.589524 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cjq9\" (UniqueName: \"kubernetes.io/projected/57c1502d-f287-4126-8d7a-ab800b49d84e-kube-api-access-6cjq9\") pod \"dns-default-p5xgx\" (UID: \"57c1502d-f287-4126-8d7a-ab800b49d84e\") " pod="openshift-dns/dns-default-p5xgx" Sep 07 10:08:37.589644 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.589531 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j2qj\" (UniqueName: \"kubernetes.io/projected/24404e76-4654-44b0-ab31-06822ef18722-kube-api-access-4j2qj\") pod \"ingress-canary-cdkr7\" (UID: \"24404e76-4654-44b0-ab31-06822ef18722\") " pod="openshift-ingress-canary/ingress-canary-cdkr7" Sep 07 10:08:37.720927 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:37.720887 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cdkr7" Sep 07 10:08:38.048869 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:38.048833 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cdkr7"] Sep 07 10:08:38.053268 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:08:38.053217 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24404e76_4654_44b0_ab31_06822ef18722.slice/crio-69b2bc24cc5beb12cc89c157588026a1db779d72ce4fd89edf435c9f6b2fe99a WatchSource:0}: Error finding container 69b2bc24cc5beb12cc89c157588026a1db779d72ce4fd89edf435c9f6b2fe99a: Status 404 returned error can't find the container with id 69b2bc24cc5beb12cc89c157588026a1db779d72ce4fd89edf435c9f6b2fe99a Sep 07 10:08:38.084652 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:38.084624 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls\") pod \"dns-default-p5xgx\" (UID: \"57c1502d-f287-4126-8d7a-ab800b49d84e\") " pod="openshift-dns/dns-default-p5xgx" Sep 07 10:08:38.084786 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:38.084768 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 10:08:38.084838 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:38.084828 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls podName:57c1502d-f287-4126-8d7a-ab800b49d84e nodeName:}" failed. No retries permitted until 2026-09-07 10:08:39.084811417 +0000 UTC m=+36.053223206 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls") pod "dns-default-p5xgx" (UID: "57c1502d-f287-4126-8d7a-ab800b49d84e") : secret "dns-default-metrics-tls" not found Sep 07 10:08:38.516390 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:38.516355 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:08:38.516390 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:38.516388 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:08:38.516632 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:38.516355 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:08:38.518866 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:38.518845 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 07 10:08:38.519028 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:38.518911 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 07 10:08:38.519155 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:38.519139 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 07 10:08:38.519882 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:38.519845 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 07 10:08:38.519981 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:38.519885 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2frx2\"" Sep 07 10:08:38.519981 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:38.519933 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5tggq\"" Sep 07 10:08:38.837813 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:38.837781 2552 generic.go:356] "Generic (PLEG): container finished" podID="a04783c7-15ce-4d0c-82a3-6b946c5ecda0" containerID="bc934cd57ca20f6e3fd990a7c5a9704467e6f8199a945e681e1837bdf7dfbd79" exitCode=0 Sep 07 10:08:38.838641 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:38.837868 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmbs2" event={"ID":"a04783c7-15ce-4d0c-82a3-6b946c5ecda0","Type":"ContainerDied","Data":"bc934cd57ca20f6e3fd990a7c5a9704467e6f8199a945e681e1837bdf7dfbd79"} Sep 07 10:08:38.839531 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:38.839509 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cdkr7" event={"ID":"24404e76-4654-44b0-ab31-06822ef18722","Type":"ContainerStarted","Data":"69b2bc24cc5beb12cc89c157588026a1db779d72ce4fd89edf435c9f6b2fe99a"} Sep 07 10:08:39.093827 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:39.093740 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls\") pod \"dns-default-p5xgx\" (UID: \"57c1502d-f287-4126-8d7a-ab800b49d84e\") " pod="openshift-dns/dns-default-p5xgx" Sep 07 10:08:39.093991 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:39.093914 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 10:08:39.094049 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:39.093993 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls podName:57c1502d-f287-4126-8d7a-ab800b49d84e nodeName:}" failed. No retries permitted until 2026-09-07 10:08:41.093972575 +0000 UTC m=+38.062384365 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls") pod "dns-default-p5xgx" (UID: "57c1502d-f287-4126-8d7a-ab800b49d84e") : secret "dns-default-metrics-tls" not found Sep 07 10:08:39.846092 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:39.846058 2552 generic.go:356] "Generic (PLEG): container finished" podID="a04783c7-15ce-4d0c-82a3-6b946c5ecda0" containerID="1013a85ee60281212787959d815c8fe12be3f60afb8c033834c3ef6204365eba" exitCode=0 Sep 07 10:08:39.846545 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:39.846131 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmbs2" event={"ID":"a04783c7-15ce-4d0c-82a3-6b946c5ecda0","Type":"ContainerDied","Data":"1013a85ee60281212787959d815c8fe12be3f60afb8c033834c3ef6204365eba"} Sep 07 10:08:40.852172 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:40.852091 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmbs2" event={"ID":"a04783c7-15ce-4d0c-82a3-6b946c5ecda0","Type":"ContainerStarted","Data":"f0da4e2d50cfe7ae95240c79780a73e0329aede20dfb9434d0a989570c55e940"} Sep 07 10:08:40.853613 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:40.853590 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cdkr7" event={"ID":"24404e76-4654-44b0-ab31-06822ef18722","Type":"ContainerStarted","Data":"21d76658d4f10bd2743d5226817fe216d22f76aa049f3271790ebd77033f3546"} Sep 07 10:08:40.874138 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:40.874094 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gmbs2" podStartSLOduration=9.224550011 podStartE2EDuration="37.874082467s" totalImagesPullingTime="28.649532456s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:08:03 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:08:40.872885597 +0000 UTC m=+37.841297381" watchObservedRunningTime="2026-09-07 10:08:40.874082467 +0000 UTC m=+37.842494260" Sep 07 10:08:40.889297 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:40.889256 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-cdkr7" podStartSLOduration=1.841085488 podStartE2EDuration="3.889230331s" totalImagesPullingTime="2.048144843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:08:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:08:40.888409095 +0000 UTC m=+37.856820878" watchObservedRunningTime="2026-09-07 10:08:40.889230331 +0000 UTC m=+37.857642123" Sep 07 10:08:41.110107 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:41.110047 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls\") pod \"dns-default-p5xgx\" (UID: \"57c1502d-f287-4126-8d7a-ab800b49d84e\") " pod="openshift-dns/dns-default-p5xgx" Sep 07 10:08:41.110207 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:41.110167 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 10:08:41.110266 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:41.110219 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls podName:57c1502d-f287-4126-8d7a-ab800b49d84e nodeName:}" failed. No retries permitted until 2026-09-07 10:08:45.11020407 +0000 UTC m=+42.078615841 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls") pod "dns-default-p5xgx" (UID: "57c1502d-f287-4126-8d7a-ab800b49d84e") : secret "dns-default-metrics-tls" not found Sep 07 10:08:45.137302 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:45.137258 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls\") pod \"dns-default-p5xgx\" (UID: \"57c1502d-f287-4126-8d7a-ab800b49d84e\") " pod="openshift-dns/dns-default-p5xgx" Sep 07 10:08:45.137687 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:45.137377 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 10:08:45.137687 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:45.137428 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls podName:57c1502d-f287-4126-8d7a-ab800b49d84e nodeName:}" failed. No retries permitted until 2026-09-07 10:08:53.137413861 +0000 UTC m=+50.105825645 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls") pod "dns-default-p5xgx" (UID: "57c1502d-f287-4126-8d7a-ab800b49d84e") : secret "dns-default-metrics-tls" not found Sep 07 10:08:53.187683 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:08:53.187638 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls\") pod \"dns-default-p5xgx\" (UID: \"57c1502d-f287-4126-8d7a-ab800b49d84e\") " pod="openshift-dns/dns-default-p5xgx" Sep 07 10:08:53.188213 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:53.187778 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 10:08:53.188213 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:08:53.187854 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls podName:57c1502d-f287-4126-8d7a-ab800b49d84e nodeName:}" failed. No retries permitted until 2026-09-07 10:09:09.187834449 +0000 UTC m=+66.156246240 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls") pod "dns-default-p5xgx" (UID: "57c1502d-f287-4126-8d7a-ab800b49d84e") : secret "dns-default-metrics-tls" not found Sep 07 10:09:03.831794 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:03.831763 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lcshb" Sep 07 10:09:09.197258 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:09.197193 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls\") pod \"dns-default-p5xgx\" (UID: \"57c1502d-f287-4126-8d7a-ab800b49d84e\") " pod="openshift-dns/dns-default-p5xgx" Sep 07 10:09:09.197714 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:09.197377 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 10:09:09.197714 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:09.197463 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls podName:57c1502d-f287-4126-8d7a-ab800b49d84e nodeName:}" failed. No retries permitted until 2026-09-07 10:09:41.197444614 +0000 UTC m=+98.165856385 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls") pod "dns-default-p5xgx" (UID: "57c1502d-f287-4126-8d7a-ab800b49d84e") : secret "dns-default-metrics-tls" not found Sep 07 10:09:09.298587 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:09.298547 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-original-pull-secret\") pod \"global-pull-secret-syncer-llgqf\" (UID: \"5c35d143-5a49-4636-b86b-c1dd3f6df6c3\") " pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:09:09.298745 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:09.298605 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs\") pod \"network-metrics-daemon-f5sf9\" (UID: \"6cae549b-87f4-4031-b984-83a7d64f9dd3\") " pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:09:09.300882 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:09.300861 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 07 10:09:09.300978 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:09.300959 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 07 10:09:09.308760 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:09.308737 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 07 10:09:09.308847 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:09.308810 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs podName:6cae549b-87f4-4031-b984-83a7d64f9dd3 nodeName:}" failed. No retries permitted until 2026-09-07 10:10:13.308791083 +0000 UTC m=+130.277202855 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs") pod "network-metrics-daemon-f5sf9" (UID: "6cae549b-87f4-4031-b984-83a7d64f9dd3") : secret "metrics-daemon-secret" not found Sep 07 10:09:09.311779 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:09.311757 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5c35d143-5a49-4636-b86b-c1dd3f6df6c3-original-pull-secret\") pod \"global-pull-secret-syncer-llgqf\" (UID: \"5c35d143-5a49-4636-b86b-c1dd3f6df6c3\") " pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:09:09.399205 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:09.399167 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgvsz\" (UniqueName: \"kubernetes.io/projected/1359e8cd-f362-45c5-8071-c59d23eab440-kube-api-access-cgvsz\") pod \"network-check-target-w9rwx\" (UID: \"1359e8cd-f362-45c5-8071-c59d23eab440\") " pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:09:09.401707 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:09.401689 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 07 10:09:09.411443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:09.411426 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 07 10:09:09.422256 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:09.422228 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgvsz\" (UniqueName: \"kubernetes.io/projected/1359e8cd-f362-45c5-8071-c59d23eab440-kube-api-access-cgvsz\") pod \"network-check-target-w9rwx\" (UID: \"1359e8cd-f362-45c5-8071-c59d23eab440\") " pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:09:09.429942 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:09.429923 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5tggq\"" Sep 07 10:09:09.438640 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:09.438623 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:09:09.440854 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:09.440836 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-llgqf" Sep 07 10:09:09.733766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:09.733691 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-w9rwx"] Sep 07 10:09:09.737195 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:09:09.737172 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1359e8cd_f362_45c5_8071_c59d23eab440.slice/crio-a601bc8e6dc72006cbf47ca1efd057fda47d58634030e6e0561101d9a0a3f463 WatchSource:0}: Error finding container a601bc8e6dc72006cbf47ca1efd057fda47d58634030e6e0561101d9a0a3f463: Status 404 returned error can't find the container with id a601bc8e6dc72006cbf47ca1efd057fda47d58634030e6e0561101d9a0a3f463 Sep 07 10:09:09.755140 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:09.755116 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-llgqf"] Sep 07 10:09:09.758128 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:09:09.758107 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c35d143_5a49_4636_b86b_c1dd3f6df6c3.slice/crio-68972397c3539e3687277da8e1465096d753ff2b33eacffdad2739f4d33cdcb0 WatchSource:0}: Error finding container 68972397c3539e3687277da8e1465096d753ff2b33eacffdad2739f4d33cdcb0: Status 404 returned error can't find the container with id 68972397c3539e3687277da8e1465096d753ff2b33eacffdad2739f4d33cdcb0 Sep 07 10:09:09.924070 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:09.924035 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-w9rwx" event={"ID":"1359e8cd-f362-45c5-8071-c59d23eab440","Type":"ContainerStarted","Data":"a601bc8e6dc72006cbf47ca1efd057fda47d58634030e6e0561101d9a0a3f463"} Sep 07 10:09:09.925169 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:09.925142 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-llgqf" event={"ID":"5c35d143-5a49-4636-b86b-c1dd3f6df6c3","Type":"ContainerStarted","Data":"68972397c3539e3687277da8e1465096d753ff2b33eacffdad2739f4d33cdcb0"} Sep 07 10:09:14.939966 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:14.939923 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-w9rwx" event={"ID":"1359e8cd-f362-45c5-8071-c59d23eab440","Type":"ContainerStarted","Data":"c7a3a37db1ba4cbe82eb6f2a74adef407e32b438f8df25ef7739d2d06e4c6140"} Sep 07 10:09:14.940486 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:14.940022 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:09:14.941314 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:14.941288 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-llgqf" event={"ID":"5c35d143-5a49-4636-b86b-c1dd3f6df6c3","Type":"ContainerStarted","Data":"f370e54488a58003fda1ee4004c347e9a0e295c5e3423c6c45250c00067a9798"} Sep 07 10:09:14.955604 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:14.955558 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-w9rwx" podStartSLOduration=67.581291712 podStartE2EDuration="1m11.955544698s" totalImagesPullingTime="4.374252986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:09:14.954832876 +0000 UTC m=+71.923244668" watchObservedRunningTime="2026-09-07 10:09:14.955544698 +0000 UTC m=+71.923956491" Sep 07 10:09:14.968979 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:14.968936 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-llgqf" podStartSLOduration=67.612521949 podStartE2EDuration="1m11.968922703s" totalImagesPullingTime="4.356400754s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:09:14.96842993 +0000 UTC m=+71.936841724" watchObservedRunningTime="2026-09-07 10:09:14.968922703 +0000 UTC m=+71.937334496" Sep 07 10:09:29.962650 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:29.962613 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-5xkkz"] Sep 07 10:09:29.965009 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:29.964993 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5xkkz" Sep 07 10:09:29.966965 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:29.966941 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-75cc47b89c-ts6dd"] Sep 07 10:09:29.967722 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:29.967693 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 07 10:09:29.967827 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:29.967746 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 07 10:09:29.967827 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:29.967765 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 07 10:09:29.967827 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:29.967746 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-h9kg7\"" Sep 07 10:09:29.967827 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:29.967814 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 07 10:09:29.968539 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:29.968519 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:29.972588 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:29.972567 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 07 10:09:29.972852 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:29.972837 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-djfnm\"" Sep 07 10:09:29.973063 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:29.973039 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 07 10:09:29.973123 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:29.973109 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 07 10:09:29.977553 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:29.977533 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 07 10:09:29.978091 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:29.978072 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-5xkkz"] Sep 07 10:09:29.987094 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:29.987066 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75cc47b89c-ts6dd"] Sep 07 10:09:30.034423 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.034396 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/29bcf16b-9878-4b5b-859e-d2cf6be8ff9d-config\") pod \"service-ca-operator-859f94c4b-5xkkz\" (UID: \"29bcf16b-9878-4b5b-859e-d2cf6be8ff9d\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5xkkz" Sep 07 10:09:30.034608 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.034432 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-image-registry-private-configuration\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:30.034608 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.034456 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdjwn\" (UniqueName: \"kubernetes.io/projected/29bcf16b-9878-4b5b-859e-d2cf6be8ff9d-kube-api-access-bdjwn\") pod \"service-ca-operator-859f94c4b-5xkkz\" (UID: \"29bcf16b-9878-4b5b-859e-d2cf6be8ff9d\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5xkkz" Sep 07 10:09:30.034608 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.034533 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-ca-trust-extracted\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:30.034608 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.034565 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-bound-sa-token\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:30.034608 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.034607 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-certificates\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:30.034850 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.034642 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/29bcf16b-9878-4b5b-859e-d2cf6be8ff9d-serving-cert\") pod \"service-ca-operator-859f94c4b-5xkkz\" (UID: \"29bcf16b-9878-4b5b-859e-d2cf6be8ff9d\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5xkkz" Sep 07 10:09:30.034850 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.034667 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-trusted-ca\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:30.034850 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.034700 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-tls\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:30.034850 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.034718 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-installation-pull-secrets\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:30.034850 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.034736 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw4j7\" (UniqueName: \"kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-kube-api-access-sw4j7\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:30.135942 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.135900 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-image-registry-private-configuration\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:30.135942 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.135947 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdjwn\" (UniqueName: \"kubernetes.io/projected/29bcf16b-9878-4b5b-859e-d2cf6be8ff9d-kube-api-access-bdjwn\") pod \"service-ca-operator-859f94c4b-5xkkz\" (UID: \"29bcf16b-9878-4b5b-859e-d2cf6be8ff9d\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5xkkz" Sep 07 10:09:30.136185 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.136108 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-ca-trust-extracted\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:30.136185 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.136147 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-bound-sa-token\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:30.136325 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.136193 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-certificates\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:30.136325 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.136236 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/29bcf16b-9878-4b5b-859e-d2cf6be8ff9d-serving-cert\") pod \"service-ca-operator-859f94c4b-5xkkz\" (UID: \"29bcf16b-9878-4b5b-859e-d2cf6be8ff9d\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5xkkz" Sep 07 10:09:30.136325 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.136288 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-trusted-ca\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:30.136472 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.136324 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-tls\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:30.136472 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.136347 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-installation-pull-secrets\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:30.136472 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.136376 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw4j7\" (UniqueName: \"kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-kube-api-access-sw4j7\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:30.136472 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.136420 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/29bcf16b-9878-4b5b-859e-d2cf6be8ff9d-config\") pod \"service-ca-operator-859f94c4b-5xkkz\" (UID: \"29bcf16b-9878-4b5b-859e-d2cf6be8ff9d\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5xkkz" Sep 07 10:09:30.136674 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.136558 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-ca-trust-extracted\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:30.136674 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:30.136599 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 07 10:09:30.136674 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:30.136628 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75cc47b89c-ts6dd: secret "image-registry-tls" not found Sep 07 10:09:30.136824 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:30.136697 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-tls podName:1e0a434b-6047-48f0-a1d9-0e9147ddb9b1 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:30.636678035 +0000 UTC m=+87.605089820 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-tls") pod "image-registry-75cc47b89c-ts6dd" (UID: "1e0a434b-6047-48f0-a1d9-0e9147ddb9b1") : secret "image-registry-tls" not found Sep 07 10:09:30.136886 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.136817 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-certificates\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:30.137065 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.137044 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/29bcf16b-9878-4b5b-859e-d2cf6be8ff9d-config\") pod \"service-ca-operator-859f94c4b-5xkkz\" (UID: \"29bcf16b-9878-4b5b-859e-d2cf6be8ff9d\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5xkkz" Sep 07 10:09:30.137348 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.137324 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-trusted-ca\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:30.138770 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.138750 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/29bcf16b-9878-4b5b-859e-d2cf6be8ff9d-serving-cert\") pod \"service-ca-operator-859f94c4b-5xkkz\" (UID: \"29bcf16b-9878-4b5b-859e-d2cf6be8ff9d\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5xkkz" Sep 07 10:09:30.138987 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.138962 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-image-registry-private-configuration\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:30.139048 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.138989 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-installation-pull-secrets\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:30.144288 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.144265 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdjwn\" (UniqueName: \"kubernetes.io/projected/29bcf16b-9878-4b5b-859e-d2cf6be8ff9d-kube-api-access-bdjwn\") pod \"service-ca-operator-859f94c4b-5xkkz\" (UID: \"29bcf16b-9878-4b5b-859e-d2cf6be8ff9d\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5xkkz" Sep 07 10:09:30.145574 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.145553 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-bound-sa-token\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:30.145743 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.145721 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw4j7\" (UniqueName: \"kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-kube-api-access-sw4j7\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:30.276546 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.276513 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5xkkz" Sep 07 10:09:30.541624 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.541537 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-5xkkz"] Sep 07 10:09:30.544123 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:09:30.544101 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29bcf16b_9878_4b5b_859e_d2cf6be8ff9d.slice/crio-f4fe32ad545d3340dd418d2838a6d1629443aa2b401026279cc9f20c226c81ef WatchSource:0}: Error finding container f4fe32ad545d3340dd418d2838a6d1629443aa2b401026279cc9f20c226c81ef: Status 404 returned error can't find the container with id f4fe32ad545d3340dd418d2838a6d1629443aa2b401026279cc9f20c226c81ef Sep 07 10:09:30.640701 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.640653 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-tls\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:30.640865 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:30.640807 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 07 10:09:30.640865 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:30.640824 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75cc47b89c-ts6dd: secret "image-registry-tls" not found Sep 07 10:09:30.640937 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:30.640890 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-tls podName:1e0a434b-6047-48f0-a1d9-0e9147ddb9b1 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:31.640873911 +0000 UTC m=+88.609285705 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-tls") pod "image-registry-75cc47b89c-ts6dd" (UID: "1e0a434b-6047-48f0-a1d9-0e9147ddb9b1") : secret "image-registry-tls" not found Sep 07 10:09:30.981728 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:30.981694 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5xkkz" event={"ID":"29bcf16b-9878-4b5b-859e-d2cf6be8ff9d","Type":"ContainerStarted","Data":"f4fe32ad545d3340dd418d2838a6d1629443aa2b401026279cc9f20c226c81ef"} Sep 07 10:09:31.648436 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:31.648390 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-tls\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:31.648620 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:31.648552 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 07 10:09:31.648620 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:31.648576 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75cc47b89c-ts6dd: secret "image-registry-tls" not found Sep 07 10:09:31.648705 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:31.648655 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-tls podName:1e0a434b-6047-48f0-a1d9-0e9147ddb9b1 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:33.648632726 +0000 UTC m=+90.617044502 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-tls") pod "image-registry-75cc47b89c-ts6dd" (UID: "1e0a434b-6047-48f0-a1d9-0e9147ddb9b1") : secret "image-registry-tls" not found Sep 07 10:09:32.989210 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:32.989173 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5xkkz" event={"ID":"29bcf16b-9878-4b5b-859e-d2cf6be8ff9d","Type":"ContainerStarted","Data":"5ff0039f1daa39c4355cd10d0fba8af6dc027b8af92323f866deea8ce99d7113"} Sep 07 10:09:33.006438 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:33.006383 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5xkkz" podStartSLOduration=2.104220657 podStartE2EDuration="4.006367137s" totalImagesPullingTime="1.90214648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:09:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:09:33.005763315 +0000 UTC m=+89.974175110" watchObservedRunningTime="2026-09-07 10:09:33.006367137 +0000 UTC m=+89.974778929" Sep 07 10:09:33.664916 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:33.662031 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-tls\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:33.664916 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:33.662298 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 07 10:09:33.664916 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:33.662314 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75cc47b89c-ts6dd: secret "image-registry-tls" not found Sep 07 10:09:33.664916 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:33.662431 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-tls podName:1e0a434b-6047-48f0-a1d9-0e9147ddb9b1 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:37.662372528 +0000 UTC m=+94.630784301 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-tls") pod "image-registry-75cc47b89c-ts6dd" (UID: "1e0a434b-6047-48f0-a1d9-0e9147ddb9b1") : secret "image-registry-tls" not found Sep 07 10:09:33.924484 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:33.924399 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9klwx"] Sep 07 10:09:33.926600 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:33.926583 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:33.928844 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:33.928820 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 07 10:09:33.928844 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:33.928832 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 07 10:09:33.929012 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:33.928885 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 07 10:09:33.929744 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:33.929725 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rmr27\"" Sep 07 10:09:33.929813 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:33.929760 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 07 10:09:33.939373 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:33.939352 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9klwx"] Sep 07 10:09:33.964044 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:33.964017 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/da9be304-bff3-4215-89ae-04bee504e10e-data-volume\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:33.964220 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:33.964050 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/da9be304-bff3-4215-89ae-04bee504e10e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:33.964220 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:33.964089 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/da9be304-bff3-4215-89ae-04bee504e10e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:33.964220 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:33.964200 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/da9be304-bff3-4215-89ae-04bee504e10e-crio-socket\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:33.964393 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:33.964287 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxl8j\" (UniqueName: \"kubernetes.io/projected/da9be304-bff3-4215-89ae-04bee504e10e-kube-api-access-gxl8j\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:34.064976 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:34.064930 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/da9be304-bff3-4215-89ae-04bee504e10e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:34.064976 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:34.064988 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/da9be304-bff3-4215-89ae-04bee504e10e-crio-socket\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:34.065514 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:34.065033 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxl8j\" (UniqueName: \"kubernetes.io/projected/da9be304-bff3-4215-89ae-04bee504e10e-kube-api-access-gxl8j\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:34.065514 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:34.065061 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 07 10:09:34.065514 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:34.065076 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/da9be304-bff3-4215-89ae-04bee504e10e-crio-socket\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:34.065514 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:34.065066 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/da9be304-bff3-4215-89ae-04bee504e10e-data-volume\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:34.065514 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:34.065164 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/da9be304-bff3-4215-89ae-04bee504e10e-kube-rbac-proxy-cm podName:da9be304-bff3-4215-89ae-04bee504e10e nodeName:}" failed. No retries permitted until 2026-09-07 10:09:34.565139711 +0000 UTC m=+91.533551508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/da9be304-bff3-4215-89ae-04bee504e10e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9klwx" (UID: "da9be304-bff3-4215-89ae-04bee504e10e") : configmap "kube-rbac-proxy" not found Sep 07 10:09:34.065514 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:34.065198 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/da9be304-bff3-4215-89ae-04bee504e10e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:34.065514 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:34.065340 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 10:09:34.065514 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:34.065382 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/da9be304-bff3-4215-89ae-04bee504e10e-data-volume\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:34.065514 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:34.065391 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da9be304-bff3-4215-89ae-04bee504e10e-insights-runtime-extractor-tls podName:da9be304-bff3-4215-89ae-04bee504e10e nodeName:}" failed. No retries permitted until 2026-09-07 10:09:34.56537651 +0000 UTC m=+91.533788316 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/da9be304-bff3-4215-89ae-04bee504e10e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9klwx" (UID: "da9be304-bff3-4215-89ae-04bee504e10e") : secret "insights-runtime-extractor-tls" not found Sep 07 10:09:34.072856 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:34.072831 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxl8j\" (UniqueName: \"kubernetes.io/projected/da9be304-bff3-4215-89ae-04bee504e10e-kube-api-access-gxl8j\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:34.568887 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:34.568841 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/da9be304-bff3-4215-89ae-04bee504e10e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:34.569069 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:34.568931 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/da9be304-bff3-4215-89ae-04bee504e10e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:34.569069 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:34.568972 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 07 10:09:34.569069 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:34.569034 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 10:09:34.569069 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:34.569053 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/da9be304-bff3-4215-89ae-04bee504e10e-kube-rbac-proxy-cm podName:da9be304-bff3-4215-89ae-04bee504e10e nodeName:}" failed. No retries permitted until 2026-09-07 10:09:35.569037613 +0000 UTC m=+92.537449383 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/da9be304-bff3-4215-89ae-04bee504e10e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9klwx" (UID: "da9be304-bff3-4215-89ae-04bee504e10e") : configmap "kube-rbac-proxy" not found Sep 07 10:09:34.569205 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:34.569075 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da9be304-bff3-4215-89ae-04bee504e10e-insights-runtime-extractor-tls podName:da9be304-bff3-4215-89ae-04bee504e10e nodeName:}" failed. No retries permitted until 2026-09-07 10:09:35.569063319 +0000 UTC m=+92.537475090 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/da9be304-bff3-4215-89ae-04bee504e10e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9klwx" (UID: "da9be304-bff3-4215-89ae-04bee504e10e") : secret "insights-runtime-extractor-tls" not found Sep 07 10:09:34.995476 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:34.995445 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-l9jh8"] Sep 07 10:09:34.997359 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:34.997344 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-l9jh8" Sep 07 10:09:34.999720 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:34.999688 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-5n2fx\"" Sep 07 10:09:35.000457 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:35.000430 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Sep 07 10:09:35.000457 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:35.000456 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Sep 07 10:09:35.009386 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:35.009360 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-l9jh8"] Sep 07 10:09:35.073552 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:35.073517 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpj9w\" (UniqueName: \"kubernetes.io/projected/24fbe6c1-0754-4fce-a795-334557b3cc16-kube-api-access-vpj9w\") pod \"migrator-fdbb8fdff-l9jh8\" (UID: \"24fbe6c1-0754-4fce-a795-334557b3cc16\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-l9jh8" Sep 07 10:09:35.174739 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:35.174685 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpj9w\" (UniqueName: \"kubernetes.io/projected/24fbe6c1-0754-4fce-a795-334557b3cc16-kube-api-access-vpj9w\") pod \"migrator-fdbb8fdff-l9jh8\" (UID: \"24fbe6c1-0754-4fce-a795-334557b3cc16\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-l9jh8" Sep 07 10:09:35.181959 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:35.181938 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpj9w\" (UniqueName: \"kubernetes.io/projected/24fbe6c1-0754-4fce-a795-334557b3cc16-kube-api-access-vpj9w\") pod \"migrator-fdbb8fdff-l9jh8\" (UID: \"24fbe6c1-0754-4fce-a795-334557b3cc16\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-l9jh8" Sep 07 10:09:35.306636 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:35.306542 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-l9jh8" Sep 07 10:09:35.572045 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:35.571968 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-l9jh8"] Sep 07 10:09:35.575379 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:09:35.575352 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24fbe6c1_0754_4fce_a795_334557b3cc16.slice/crio-1f31dd27350aafd3d703aa4996bc66ed859be64bf4520a3630b12de595aee0f1 WatchSource:0}: Error finding container 1f31dd27350aafd3d703aa4996bc66ed859be64bf4520a3630b12de595aee0f1: Status 404 returned error can't find the container with id 1f31dd27350aafd3d703aa4996bc66ed859be64bf4520a3630b12de595aee0f1 Sep 07 10:09:35.576847 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:35.576826 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/da9be304-bff3-4215-89ae-04bee504e10e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:35.576951 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:35.576883 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/da9be304-bff3-4215-89ae-04bee504e10e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:35.576951 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:35.576942 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 10:09:35.577044 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:35.576993 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 07 10:09:35.577044 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:35.577001 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da9be304-bff3-4215-89ae-04bee504e10e-insights-runtime-extractor-tls podName:da9be304-bff3-4215-89ae-04bee504e10e nodeName:}" failed. No retries permitted until 2026-09-07 10:09:37.576982228 +0000 UTC m=+94.545394005 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/da9be304-bff3-4215-89ae-04bee504e10e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9klwx" (UID: "da9be304-bff3-4215-89ae-04bee504e10e") : secret "insights-runtime-extractor-tls" not found Sep 07 10:09:35.577044 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:35.577043 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/da9be304-bff3-4215-89ae-04bee504e10e-kube-rbac-proxy-cm podName:da9be304-bff3-4215-89ae-04bee504e10e nodeName:}" failed. No retries permitted until 2026-09-07 10:09:37.577023456 +0000 UTC m=+94.545435249 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/da9be304-bff3-4215-89ae-04bee504e10e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9klwx" (UID: "da9be304-bff3-4215-89ae-04bee504e10e") : configmap "kube-rbac-proxy" not found Sep 07 10:09:35.659232 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:35.659206 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kqh2w_f9e1b00d-e190-452a-8322-333f9397c2fa/dns-node-resolver/0.log" Sep 07 10:09:35.998142 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:35.998106 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-l9jh8" event={"ID":"24fbe6c1-0754-4fce-a795-334557b3cc16","Type":"ContainerStarted","Data":"1f31dd27350aafd3d703aa4996bc66ed859be64bf4520a3630b12de595aee0f1"} Sep 07 10:09:36.457594 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:36.457516 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-86dvg"] Sep 07 10:09:36.459423 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:36.459404 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-86dvg" Sep 07 10:09:36.461582 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:36.461557 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 07 10:09:36.461677 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:36.461596 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 07 10:09:36.462384 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:36.462363 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-qqqcq\"" Sep 07 10:09:36.462480 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:36.462381 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 07 10:09:36.462480 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:36.462372 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 07 10:09:36.467163 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:36.467142 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-86dvg"] Sep 07 10:09:36.483823 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:36.483797 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nszg6\" (UniqueName: \"kubernetes.io/projected/76df16bc-648e-49d3-8496-23293f551fe2-kube-api-access-nszg6\") pod \"service-ca-75d99c484-86dvg\" (UID: \"76df16bc-648e-49d3-8496-23293f551fe2\") " pod="openshift-service-ca/service-ca-75d99c484-86dvg" Sep 07 10:09:36.483946 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:36.483852 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/76df16bc-648e-49d3-8496-23293f551fe2-signing-cabundle\") pod \"service-ca-75d99c484-86dvg\" (UID: \"76df16bc-648e-49d3-8496-23293f551fe2\") " pod="openshift-service-ca/service-ca-75d99c484-86dvg" Sep 07 10:09:36.484019 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:36.483964 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/76df16bc-648e-49d3-8496-23293f551fe2-signing-key\") pod \"service-ca-75d99c484-86dvg\" (UID: \"76df16bc-648e-49d3-8496-23293f551fe2\") " pod="openshift-service-ca/service-ca-75d99c484-86dvg" Sep 07 10:09:36.585319 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:36.585282 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/76df16bc-648e-49d3-8496-23293f551fe2-signing-cabundle\") pod \"service-ca-75d99c484-86dvg\" (UID: \"76df16bc-648e-49d3-8496-23293f551fe2\") " pod="openshift-service-ca/service-ca-75d99c484-86dvg" Sep 07 10:09:36.585483 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:36.585398 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/76df16bc-648e-49d3-8496-23293f551fe2-signing-key\") pod \"service-ca-75d99c484-86dvg\" (UID: \"76df16bc-648e-49d3-8496-23293f551fe2\") " pod="openshift-service-ca/service-ca-75d99c484-86dvg" Sep 07 10:09:36.585483 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:36.585463 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nszg6\" (UniqueName: \"kubernetes.io/projected/76df16bc-648e-49d3-8496-23293f551fe2-kube-api-access-nszg6\") pod \"service-ca-75d99c484-86dvg\" (UID: \"76df16bc-648e-49d3-8496-23293f551fe2\") " pod="openshift-service-ca/service-ca-75d99c484-86dvg" Sep 07 10:09:36.586069 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:36.586039 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/76df16bc-648e-49d3-8496-23293f551fe2-signing-cabundle\") pod \"service-ca-75d99c484-86dvg\" (UID: \"76df16bc-648e-49d3-8496-23293f551fe2\") " pod="openshift-service-ca/service-ca-75d99c484-86dvg" Sep 07 10:09:36.587905 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:36.587886 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/76df16bc-648e-49d3-8496-23293f551fe2-signing-key\") pod \"service-ca-75d99c484-86dvg\" (UID: \"76df16bc-648e-49d3-8496-23293f551fe2\") " pod="openshift-service-ca/service-ca-75d99c484-86dvg" Sep 07 10:09:36.594168 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:36.594143 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nszg6\" (UniqueName: \"kubernetes.io/projected/76df16bc-648e-49d3-8496-23293f551fe2-kube-api-access-nszg6\") pod \"service-ca-75d99c484-86dvg\" (UID: \"76df16bc-648e-49d3-8496-23293f551fe2\") " pod="openshift-service-ca/service-ca-75d99c484-86dvg" Sep 07 10:09:36.768517 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:36.768479 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-86dvg" Sep 07 10:09:37.003140 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:37.003105 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-l9jh8" event={"ID":"24fbe6c1-0754-4fce-a795-334557b3cc16","Type":"ContainerStarted","Data":"44c073d5d250cf905daf206fd5d4f139bd98bf762ccad4670fdc816180bb4f45"} Sep 07 10:09:37.003323 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:37.003190 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-l9jh8" event={"ID":"24fbe6c1-0754-4fce-a795-334557b3cc16","Type":"ContainerStarted","Data":"857beb89eb47eed560cd70cffa400bf8926d736ad62731452057893615d452b3"} Sep 07 10:09:37.020042 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:37.019948 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-l9jh8" podStartSLOduration=1.806212515 podStartE2EDuration="3.019931349s" totalImagesPullingTime="1.213718834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:09:37.018728779 +0000 UTC m=+93.987140571" watchObservedRunningTime="2026-09-07 10:09:37.019931349 +0000 UTC m=+93.988343141" Sep 07 10:09:37.050866 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:37.050835 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-86dvg"] Sep 07 10:09:37.053580 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:09:37.053548 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76df16bc_648e_49d3_8496_23293f551fe2.slice/crio-9aba8947dc6d38c7cba638fcf37d409ac13da00eb88daf8c35190b96d83e25ab WatchSource:0}: Error finding container 9aba8947dc6d38c7cba638fcf37d409ac13da00eb88daf8c35190b96d83e25ab: Status 404 returned error can't find the container with id 9aba8947dc6d38c7cba638fcf37d409ac13da00eb88daf8c35190b96d83e25ab Sep 07 10:09:37.057965 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:37.057947 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v9stm_aff3118f-d191-4990-b60b-958c0c686aec/node-ca/0.log" Sep 07 10:09:37.458583 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:37.458512 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cdkr7_24404e76-4654-44b0-ab31-06822ef18722/serve-healthcheck-canary/0.log" Sep 07 10:09:37.594032 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:37.594004 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/da9be304-bff3-4215-89ae-04bee504e10e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:37.594188 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:37.594049 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/da9be304-bff3-4215-89ae-04bee504e10e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:37.594188 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:37.594126 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 07 10:09:37.594188 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:37.594159 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 10:09:37.594188 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:37.594178 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/da9be304-bff3-4215-89ae-04bee504e10e-kube-rbac-proxy-cm podName:da9be304-bff3-4215-89ae-04bee504e10e nodeName:}" failed. No retries permitted until 2026-09-07 10:09:41.594164743 +0000 UTC m=+98.562576515 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/da9be304-bff3-4215-89ae-04bee504e10e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9klwx" (UID: "da9be304-bff3-4215-89ae-04bee504e10e") : configmap "kube-rbac-proxy" not found Sep 07 10:09:37.594352 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:37.594210 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da9be304-bff3-4215-89ae-04bee504e10e-insights-runtime-extractor-tls podName:da9be304-bff3-4215-89ae-04bee504e10e nodeName:}" failed. No retries permitted until 2026-09-07 10:09:41.594195895 +0000 UTC m=+98.562607666 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/da9be304-bff3-4215-89ae-04bee504e10e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9klwx" (UID: "da9be304-bff3-4215-89ae-04bee504e10e") : secret "insights-runtime-extractor-tls" not found Sep 07 10:09:37.695189 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:37.695149 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-tls\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:37.695370 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:37.695291 2552 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 07 10:09:37.695370 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:37.695304 2552 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75cc47b89c-ts6dd: secret "image-registry-tls" not found Sep 07 10:09:37.695370 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:37.695352 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-tls podName:1e0a434b-6047-48f0-a1d9-0e9147ddb9b1 nodeName:}" failed. No retries permitted until 2026-09-07 10:09:45.695339103 +0000 UTC m=+102.663750875 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-tls") pod "image-registry-75cc47b89c-ts6dd" (UID: "1e0a434b-6047-48f0-a1d9-0e9147ddb9b1") : secret "image-registry-tls" not found Sep 07 10:09:38.007692 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:38.007654 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-86dvg" event={"ID":"76df16bc-648e-49d3-8496-23293f551fe2","Type":"ContainerStarted","Data":"7b719bff3952891f5f391749ece3db6ecbd87223dc0f511114d949064cb2534e"} Sep 07 10:09:38.007876 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:38.007712 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-86dvg" event={"ID":"76df16bc-648e-49d3-8496-23293f551fe2","Type":"ContainerStarted","Data":"9aba8947dc6d38c7cba638fcf37d409ac13da00eb88daf8c35190b96d83e25ab"} Sep 07 10:09:38.026519 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:38.026462 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-86dvg" podStartSLOduration=2.026444576 podStartE2EDuration="2.026444576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:09:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:09:38.025858254 +0000 UTC m=+94.994270047" watchObservedRunningTime="2026-09-07 10:09:38.026444576 +0000 UTC m=+94.994856368" Sep 07 10:09:41.224559 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:41.224517 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls\") pod \"dns-default-p5xgx\" (UID: \"57c1502d-f287-4126-8d7a-ab800b49d84e\") " pod="openshift-dns/dns-default-p5xgx" Sep 07 10:09:41.225022 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:41.224647 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 07 10:09:41.225022 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:41.224703 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls podName:57c1502d-f287-4126-8d7a-ab800b49d84e nodeName:}" failed. No retries permitted until 2026-09-07 10:10:45.224688264 +0000 UTC m=+162.193100036 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls") pod "dns-default-p5xgx" (UID: "57c1502d-f287-4126-8d7a-ab800b49d84e") : secret "dns-default-metrics-tls" not found Sep 07 10:09:41.626626 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:41.626537 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/da9be304-bff3-4215-89ae-04bee504e10e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:41.626771 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:41.626674 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 07 10:09:41.626771 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:41.626694 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/da9be304-bff3-4215-89ae-04bee504e10e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:41.626771 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:41.626743 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/da9be304-bff3-4215-89ae-04bee504e10e-kube-rbac-proxy-cm podName:da9be304-bff3-4215-89ae-04bee504e10e nodeName:}" failed. No retries permitted until 2026-09-07 10:09:49.626727924 +0000 UTC m=+106.595139696 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/da9be304-bff3-4215-89ae-04bee504e10e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9klwx" (UID: "da9be304-bff3-4215-89ae-04bee504e10e") : configmap "kube-rbac-proxy" not found Sep 07 10:09:41.626877 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:41.626790 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 07 10:09:41.626877 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:09:41.626832 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da9be304-bff3-4215-89ae-04bee504e10e-insights-runtime-extractor-tls podName:da9be304-bff3-4215-89ae-04bee504e10e nodeName:}" failed. No retries permitted until 2026-09-07 10:09:49.626821168 +0000 UTC m=+106.595232938 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/da9be304-bff3-4215-89ae-04bee504e10e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9klwx" (UID: "da9be304-bff3-4215-89ae-04bee504e10e") : secret "insights-runtime-extractor-tls" not found Sep 07 10:09:45.756081 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:45.756039 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-tls\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:45.758531 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:45.758499 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-tls\") pod \"image-registry-75cc47b89c-ts6dd\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:45.882174 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:45.882133 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:45.947133 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:45.947099 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-w9rwx" Sep 07 10:09:46.185399 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:46.185301 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75cc47b89c-ts6dd"] Sep 07 10:09:46.187858 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:09:46.187827 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e0a434b_6047_48f0_a1d9_0e9147ddb9b1.slice/crio-c3073102d7fa0f27b95f154184064c7423cf853f0f60b1b1a470a0b3b1c1f16a WatchSource:0}: Error finding container c3073102d7fa0f27b95f154184064c7423cf853f0f60b1b1a470a0b3b1c1f16a: Status 404 returned error can't find the container with id c3073102d7fa0f27b95f154184064c7423cf853f0f60b1b1a470a0b3b1c1f16a Sep 07 10:09:47.037187 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:47.037147 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" event={"ID":"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1","Type":"ContainerStarted","Data":"cf7f1e72b001dac085b03716f0ab23dc6db165da9833dd49a1e9b02bb1ec2c7a"} Sep 07 10:09:47.037606 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:47.037208 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" event={"ID":"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1","Type":"ContainerStarted","Data":"c3073102d7fa0f27b95f154184064c7423cf853f0f60b1b1a470a0b3b1c1f16a"} Sep 07 10:09:47.037606 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:47.037285 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:09:47.056533 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:47.056486 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" podStartSLOduration=18.056468578 podStartE2EDuration="18.056468578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:09:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:09:47.055858636 +0000 UTC m=+104.024270429" watchObservedRunningTime="2026-09-07 10:09:47.056468578 +0000 UTC m=+104.024880371" Sep 07 10:09:49.687474 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:49.687432 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/da9be304-bff3-4215-89ae-04bee504e10e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:49.687838 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:49.687488 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/da9be304-bff3-4215-89ae-04bee504e10e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:49.687970 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:49.687946 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/da9be304-bff3-4215-89ae-04bee504e10e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:49.689760 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:49.689743 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/da9be304-bff3-4215-89ae-04bee504e10e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9klwx\" (UID: \"da9be304-bff3-4215-89ae-04bee504e10e\") " pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:49.835629 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:49.835596 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9klwx" Sep 07 10:09:50.104310 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:50.104274 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9klwx"] Sep 07 10:09:50.107013 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:09:50.106981 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda9be304_bff3_4215_89ae_04bee504e10e.slice/crio-c1cf29bb9918289dac069687439e4ffe6c1f259633cd694daf4a9190a31c51f6 WatchSource:0}: Error finding container c1cf29bb9918289dac069687439e4ffe6c1f259633cd694daf4a9190a31c51f6: Status 404 returned error can't find the container with id c1cf29bb9918289dac069687439e4ffe6c1f259633cd694daf4a9190a31c51f6 Sep 07 10:09:51.051232 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:51.051194 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9klwx" event={"ID":"da9be304-bff3-4215-89ae-04bee504e10e","Type":"ContainerStarted","Data":"1a039629420572f780c799de1887bf17e3b38d41d714869b9dcd7a310f80a5c9"} Sep 07 10:09:51.051607 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:51.051274 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9klwx" event={"ID":"da9be304-bff3-4215-89ae-04bee504e10e","Type":"ContainerStarted","Data":"7c5a1b0ae2776b648bed180180f38e22c9f67d91ef3aed9c35f3c38750526844"} Sep 07 10:09:51.051607 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:51.051297 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9klwx" event={"ID":"da9be304-bff3-4215-89ae-04bee504e10e","Type":"ContainerStarted","Data":"c1cf29bb9918289dac069687439e4ffe6c1f259633cd694daf4a9190a31c51f6"} Sep 07 10:09:53.059869 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:53.059834 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9klwx" event={"ID":"da9be304-bff3-4215-89ae-04bee504e10e","Type":"ContainerStarted","Data":"baa62b6cd7f00f6bde4c69416ab2c508eb19ec233a11894c2d8b892f0002f587"} Sep 07 10:09:53.076916 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:53.076864 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9klwx" podStartSLOduration=17.995483489 podStartE2EDuration="20.076849061s" totalImagesPullingTime="2.081365572s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:09:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:09:53.075214713 +0000 UTC m=+110.043626506" watchObservedRunningTime="2026-09-07 10:09:53.076849061 +0000 UTC m=+110.045260853" Sep 07 10:09:55.129261 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.129212 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw"] Sep 07 10:09:55.132197 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.132179 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" Sep 07 10:09:55.134412 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.134385 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 07 10:09:55.134412 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.134406 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Sep 07 10:09:55.134608 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.134437 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Sep 07 10:09:55.134608 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.134540 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Sep 07 10:09:55.134713 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.134696 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 07 10:09:55.134990 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.134976 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 07 10:09:55.135452 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.135433 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Sep 07 10:09:55.144324 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.144303 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw"] Sep 07 10:09:55.184945 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.184920 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75cc47b89c-ts6dd"] Sep 07 10:09:55.228824 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.228793 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b1962d54-9916-4f7c-b3a4-5f8c037a31b9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6678c7fc9d-rt7pw\" (UID: \"b1962d54-9916-4f7c-b3a4-5f8c037a31b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" Sep 07 10:09:55.228972 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.228830 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1962d54-9916-4f7c-b3a4-5f8c037a31b9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6678c7fc9d-rt7pw\" (UID: \"b1962d54-9916-4f7c-b3a4-5f8c037a31b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" Sep 07 10:09:55.228972 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.228852 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b1962d54-9916-4f7c-b3a4-5f8c037a31b9-hub\") pod \"cluster-proxy-proxy-agent-6678c7fc9d-rt7pw\" (UID: \"b1962d54-9916-4f7c-b3a4-5f8c037a31b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" Sep 07 10:09:55.228972 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.228873 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b1962d54-9916-4f7c-b3a4-5f8c037a31b9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6678c7fc9d-rt7pw\" (UID: \"b1962d54-9916-4f7c-b3a4-5f8c037a31b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" Sep 07 10:09:55.228972 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.228914 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td22z\" (UniqueName: \"kubernetes.io/projected/b1962d54-9916-4f7c-b3a4-5f8c037a31b9-kube-api-access-td22z\") pod \"cluster-proxy-proxy-agent-6678c7fc9d-rt7pw\" (UID: \"b1962d54-9916-4f7c-b3a4-5f8c037a31b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" Sep 07 10:09:55.228972 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.228940 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b1962d54-9916-4f7c-b3a4-5f8c037a31b9-ca\") pod \"cluster-proxy-proxy-agent-6678c7fc9d-rt7pw\" (UID: \"b1962d54-9916-4f7c-b3a4-5f8c037a31b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" Sep 07 10:09:55.330194 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.330160 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b1962d54-9916-4f7c-b3a4-5f8c037a31b9-ca\") pod \"cluster-proxy-proxy-agent-6678c7fc9d-rt7pw\" (UID: \"b1962d54-9916-4f7c-b3a4-5f8c037a31b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" Sep 07 10:09:55.330403 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.330225 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b1962d54-9916-4f7c-b3a4-5f8c037a31b9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6678c7fc9d-rt7pw\" (UID: \"b1962d54-9916-4f7c-b3a4-5f8c037a31b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" Sep 07 10:09:55.330403 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.330370 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1962d54-9916-4f7c-b3a4-5f8c037a31b9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6678c7fc9d-rt7pw\" (UID: \"b1962d54-9916-4f7c-b3a4-5f8c037a31b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" Sep 07 10:09:55.330517 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.330413 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b1962d54-9916-4f7c-b3a4-5f8c037a31b9-hub\") pod \"cluster-proxy-proxy-agent-6678c7fc9d-rt7pw\" (UID: \"b1962d54-9916-4f7c-b3a4-5f8c037a31b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" Sep 07 10:09:55.330517 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.330430 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b1962d54-9916-4f7c-b3a4-5f8c037a31b9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6678c7fc9d-rt7pw\" (UID: \"b1962d54-9916-4f7c-b3a4-5f8c037a31b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" Sep 07 10:09:55.330517 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.330452 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td22z\" (UniqueName: \"kubernetes.io/projected/b1962d54-9916-4f7c-b3a4-5f8c037a31b9-kube-api-access-td22z\") pod \"cluster-proxy-proxy-agent-6678c7fc9d-rt7pw\" (UID: \"b1962d54-9916-4f7c-b3a4-5f8c037a31b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" Sep 07 10:09:55.331079 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.331053 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1962d54-9916-4f7c-b3a4-5f8c037a31b9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6678c7fc9d-rt7pw\" (UID: \"b1962d54-9916-4f7c-b3a4-5f8c037a31b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" Sep 07 10:09:55.332796 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.332773 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b1962d54-9916-4f7c-b3a4-5f8c037a31b9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6678c7fc9d-rt7pw\" (UID: \"b1962d54-9916-4f7c-b3a4-5f8c037a31b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" Sep 07 10:09:55.332904 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.332805 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b1962d54-9916-4f7c-b3a4-5f8c037a31b9-ca\") pod \"cluster-proxy-proxy-agent-6678c7fc9d-rt7pw\" (UID: \"b1962d54-9916-4f7c-b3a4-5f8c037a31b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" Sep 07 10:09:55.333110 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.333094 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b1962d54-9916-4f7c-b3a4-5f8c037a31b9-hub\") pod \"cluster-proxy-proxy-agent-6678c7fc9d-rt7pw\" (UID: \"b1962d54-9916-4f7c-b3a4-5f8c037a31b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" Sep 07 10:09:55.333348 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.333328 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b1962d54-9916-4f7c-b3a4-5f8c037a31b9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6678c7fc9d-rt7pw\" (UID: \"b1962d54-9916-4f7c-b3a4-5f8c037a31b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" Sep 07 10:09:55.339650 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.339628 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td22z\" (UniqueName: \"kubernetes.io/projected/b1962d54-9916-4f7c-b3a4-5f8c037a31b9-kube-api-access-td22z\") pod \"cluster-proxy-proxy-agent-6678c7fc9d-rt7pw\" (UID: \"b1962d54-9916-4f7c-b3a4-5f8c037a31b9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" Sep 07 10:09:55.455294 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.455200 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" Sep 07 10:09:55.724943 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:55.724915 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw"] Sep 07 10:09:55.728037 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:09:55.728008 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb1962d54_9916_4f7c_b3a4_5f8c037a31b9.slice/crio-5e358b84e1b2b32a03da6f60304dcab7b727bf781aa0c0b63b9424b7e7c18e1d WatchSource:0}: Error finding container 5e358b84e1b2b32a03da6f60304dcab7b727bf781aa0c0b63b9424b7e7c18e1d: Status 404 returned error can't find the container with id 5e358b84e1b2b32a03da6f60304dcab7b727bf781aa0c0b63b9424b7e7c18e1d Sep 07 10:09:56.069990 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:09:56.069903 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" event={"ID":"b1962d54-9916-4f7c-b3a4-5f8c037a31b9","Type":"ContainerStarted","Data":"5e358b84e1b2b32a03da6f60304dcab7b727bf781aa0c0b63b9424b7e7c18e1d"} Sep 07 10:10:00.085908 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:00.085874 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" event={"ID":"b1962d54-9916-4f7c-b3a4-5f8c037a31b9","Type":"ContainerStarted","Data":"3ad5db80de2c3d94d2a0501017b1088637d2cd0f6fd767dc93719546e7bf2e2e"} Sep 07 10:10:00.085908 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:00.085915 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" event={"ID":"b1962d54-9916-4f7c-b3a4-5f8c037a31b9","Type":"ContainerStarted","Data":"a7215dd30ba74706158cb47ef76be102211409993f2d418dbf19aa3638ef9081"} Sep 07 10:10:00.086353 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:00.085927 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" event={"ID":"b1962d54-9916-4f7c-b3a4-5f8c037a31b9","Type":"ContainerStarted","Data":"e5502f0fca3725957b542b6ca79dfc9e1f7341cdd05d0c61546f51e980f70692"} Sep 07 10:10:00.103096 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:00.103050 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6678c7fc9d-rt7pw" podStartSLOduration=1.664187004 podStartE2EDuration="5.103037466s" totalImagesPullingTime="3.438850462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:09:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:10:00.101922201 +0000 UTC m=+117.070333993" watchObservedRunningTime="2026-09-07 10:10:00.103037466 +0000 UTC m=+117.071449258" Sep 07 10:10:02.553405 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.553372 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hwpns"] Sep 07 10:10:02.555949 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.555915 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.558514 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.558493 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 07 10:10:02.558721 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.558502 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 07 10:10:02.558860 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.558621 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 07 10:10:02.558920 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.558627 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 07 10:10:02.559414 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.559393 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 07 10:10:02.559511 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.559417 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8phjk\"" Sep 07 10:10:02.559511 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.559430 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 07 10:10:02.688899 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.688856 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b5f2976e-e1b6-4659-80a2-555511cc90e8-node-exporter-textfile\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.689077 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.688930 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b5f2976e-e1b6-4659-80a2-555511cc90e8-node-exporter-tls\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.689077 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.689013 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt8vz\" (UniqueName: \"kubernetes.io/projected/b5f2976e-e1b6-4659-80a2-555511cc90e8-kube-api-access-xt8vz\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.689077 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.689060 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5f2976e-e1b6-4659-80a2-555511cc90e8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.689217 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.689099 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5f2976e-e1b6-4659-80a2-555511cc90e8-sys\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.689217 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.689129 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b5f2976e-e1b6-4659-80a2-555511cc90e8-root\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.689217 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.689208 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5f2976e-e1b6-4659-80a2-555511cc90e8-metrics-client-ca\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.689361 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.689258 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b5f2976e-e1b6-4659-80a2-555511cc90e8-node-exporter-wtmp\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.689361 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.689349 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b5f2976e-e1b6-4659-80a2-555511cc90e8-node-exporter-accelerators-collector-config\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.790218 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.790177 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt8vz\" (UniqueName: \"kubernetes.io/projected/b5f2976e-e1b6-4659-80a2-555511cc90e8-kube-api-access-xt8vz\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.790218 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.790221 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5f2976e-e1b6-4659-80a2-555511cc90e8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.790488 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.790266 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5f2976e-e1b6-4659-80a2-555511cc90e8-sys\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.790488 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.790294 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b5f2976e-e1b6-4659-80a2-555511cc90e8-root\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.790488 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.790330 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5f2976e-e1b6-4659-80a2-555511cc90e8-metrics-client-ca\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.790488 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.790352 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b5f2976e-e1b6-4659-80a2-555511cc90e8-node-exporter-wtmp\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.790488 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.790387 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b5f2976e-e1b6-4659-80a2-555511cc90e8-node-exporter-accelerators-collector-config\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.790488 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.790406 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b5f2976e-e1b6-4659-80a2-555511cc90e8-root\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.790788 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.790582 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5f2976e-e1b6-4659-80a2-555511cc90e8-sys\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.790788 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.790416 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b5f2976e-e1b6-4659-80a2-555511cc90e8-node-exporter-textfile\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.790788 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.790621 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b5f2976e-e1b6-4659-80a2-555511cc90e8-node-exporter-wtmp\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.790788 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.790664 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b5f2976e-e1b6-4659-80a2-555511cc90e8-node-exporter-tls\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.790788 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.790724 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b5f2976e-e1b6-4659-80a2-555511cc90e8-node-exporter-textfile\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.790788 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:10:02.790778 2552 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 07 10:10:02.791064 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:10:02.790834 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5f2976e-e1b6-4659-80a2-555511cc90e8-node-exporter-tls podName:b5f2976e-e1b6-4659-80a2-555511cc90e8 nodeName:}" failed. No retries permitted until 2026-09-07 10:10:03.290816993 +0000 UTC m=+120.259228780 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b5f2976e-e1b6-4659-80a2-555511cc90e8-node-exporter-tls") pod "node-exporter-hwpns" (UID: "b5f2976e-e1b6-4659-80a2-555511cc90e8") : secret "node-exporter-tls" not found Sep 07 10:10:02.791064 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.791010 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5f2976e-e1b6-4659-80a2-555511cc90e8-metrics-client-ca\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.791134 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.791113 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b5f2976e-e1b6-4659-80a2-555511cc90e8-node-exporter-accelerators-collector-config\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.793234 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.793212 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5f2976e-e1b6-4659-80a2-555511cc90e8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:02.800226 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:02.800197 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt8vz\" (UniqueName: \"kubernetes.io/projected/b5f2976e-e1b6-4659-80a2-555511cc90e8-kube-api-access-xt8vz\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:03.294019 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:03.293975 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b5f2976e-e1b6-4659-80a2-555511cc90e8-node-exporter-tls\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:03.296260 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:03.296216 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b5f2976e-e1b6-4659-80a2-555511cc90e8-node-exporter-tls\") pod \"node-exporter-hwpns\" (UID: \"b5f2976e-e1b6-4659-80a2-555511cc90e8\") " pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:03.467498 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:03.467472 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8phjk\"" Sep 07 10:10:03.476556 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:03.476526 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hwpns" Sep 07 10:10:03.486376 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:10:03.486347 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5f2976e_e1b6_4659_80a2_555511cc90e8.slice/crio-262ac0da2c8216c53e8560b9db846f342e0f27181ce3d5022750e746aea1a83b WatchSource:0}: Error finding container 262ac0da2c8216c53e8560b9db846f342e0f27181ce3d5022750e746aea1a83b: Status 404 returned error can't find the container with id 262ac0da2c8216c53e8560b9db846f342e0f27181ce3d5022750e746aea1a83b Sep 07 10:10:04.100308 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:04.100272 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hwpns" event={"ID":"b5f2976e-e1b6-4659-80a2-555511cc90e8","Type":"ContainerStarted","Data":"262ac0da2c8216c53e8560b9db846f342e0f27181ce3d5022750e746aea1a83b"} Sep 07 10:10:05.104581 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:05.104544 2552 generic.go:356] "Generic (PLEG): container finished" podID="b5f2976e-e1b6-4659-80a2-555511cc90e8" containerID="3607d1407abb5b4ef903d96ee8c90ae817b6e0b727e3610df3f0842cdc00e52a" exitCode=0 Sep 07 10:10:05.105056 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:05.104632 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hwpns" event={"ID":"b5f2976e-e1b6-4659-80a2-555511cc90e8","Type":"ContainerDied","Data":"3607d1407abb5b4ef903d96ee8c90ae817b6e0b727e3610df3f0842cdc00e52a"} Sep 07 10:10:05.190811 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:05.190784 2552 patch_prober.go:28] interesting pod/image-registry-75cc47b89c-ts6dd container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 07 10:10:05.190916 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:05.190835 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" podUID="1e0a434b-6047-48f0-a1d9-0e9147ddb9b1" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 07 10:10:06.110390 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:06.110352 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hwpns" event={"ID":"b5f2976e-e1b6-4659-80a2-555511cc90e8","Type":"ContainerStarted","Data":"86b3a1004283c1cfc905331f094e190d18be8b4e4888d2737c6c271a364b3fa9"} Sep 07 10:10:06.110761 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:06.110404 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hwpns" event={"ID":"b5f2976e-e1b6-4659-80a2-555511cc90e8","Type":"ContainerStarted","Data":"1f6b602ac47931f85d85f37f3d71a033785f7e86033d7bcd59eccd0776bb6e4b"} Sep 07 10:10:06.126984 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:06.126925 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hwpns" podStartSLOduration=3.030736989 podStartE2EDuration="4.126908303s" totalImagesPullingTime="1.034812055s" totalInitContainerRuntime="61.359259ms" isStatefulPod=true podCreationTimestamp="2026-09-07 10:10:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:10:06.126620657 +0000 UTC m=+123.095032475" watchObservedRunningTime="2026-09-07 10:10:06.126908303 +0000 UTC m=+123.095320100" Sep 07 10:10:13.373003 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:13.372958 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs\") pod \"network-metrics-daemon-f5sf9\" (UID: \"6cae549b-87f4-4031-b984-83a7d64f9dd3\") " pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:10:13.375423 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:13.375395 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cae549b-87f4-4031-b984-83a7d64f9dd3-metrics-certs\") pod \"network-metrics-daemon-f5sf9\" (UID: \"6cae549b-87f4-4031-b984-83a7d64f9dd3\") " pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:10:13.636636 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:13.636541 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2frx2\"" Sep 07 10:10:13.645483 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:13.645449 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f5sf9" Sep 07 10:10:13.914147 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:13.914053 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-f5sf9"] Sep 07 10:10:13.918133 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:10:13.918094 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cae549b_87f4_4031_b984_83a7d64f9dd3.slice/crio-b044460efe90c1ae433b06085046c1f21206863625a8534fb33c4e438328e2cd WatchSource:0}: Error finding container b044460efe90c1ae433b06085046c1f21206863625a8534fb33c4e438328e2cd: Status 404 returned error can't find the container with id b044460efe90c1ae433b06085046c1f21206863625a8534fb33c4e438328e2cd Sep 07 10:10:14.136190 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:14.136152 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f5sf9" event={"ID":"6cae549b-87f4-4031-b984-83a7d64f9dd3","Type":"ContainerStarted","Data":"b044460efe90c1ae433b06085046c1f21206863625a8534fb33c4e438328e2cd"} Sep 07 10:10:15.141517 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:15.141426 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f5sf9" event={"ID":"6cae549b-87f4-4031-b984-83a7d64f9dd3","Type":"ContainerStarted","Data":"a070e208d24144f9213ca2ecb9635a310f125be579f1164a0b59e4b4e960f133"} Sep 07 10:10:15.141517 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:15.141484 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f5sf9" event={"ID":"6cae549b-87f4-4031-b984-83a7d64f9dd3","Type":"ContainerStarted","Data":"3f2d05b57ff514d63c38354dd1a599d2ec29e2cfbbdc3487ca3a2413c0e6efb3"} Sep 07 10:10:15.157585 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:15.157523 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-f5sf9" podStartSLOduration=131.259220531 podStartE2EDuration="2m12.157500328s" totalImagesPullingTime="898.279797ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:10:15.156321008 +0000 UTC m=+132.124732802" watchObservedRunningTime="2026-09-07 10:10:15.157500328 +0000 UTC m=+132.125912122" Sep 07 10:10:15.189489 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:15.189457 2552 patch_prober.go:28] interesting pod/image-registry-75cc47b89c-ts6dd container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 07 10:10:15.189626 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:15.189507 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" podUID="1e0a434b-6047-48f0-a1d9-0e9147ddb9b1" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 07 10:10:20.203808 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.203761 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" podUID="1e0a434b-6047-48f0-a1d9-0e9147ddb9b1" containerName="registry" containerID="cri-o://cf7f1e72b001dac085b03716f0ab23dc6db165da9833dd49a1e9b02bb1ec2c7a" gracePeriod=30 Sep 07 10:10:20.433481 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.433458 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:10:20.528094 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.528054 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-bound-sa-token\") pod \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " Sep 07 10:10:20.528303 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.528113 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-installation-pull-secrets\") pod \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " Sep 07 10:10:20.528303 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.528143 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sw4j7\" (UniqueName: \"kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-kube-api-access-sw4j7\") pod \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " Sep 07 10:10:20.528303 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.528170 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-trusted-ca\") pod \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " Sep 07 10:10:20.528303 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.528200 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-image-registry-private-configuration\") pod \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " Sep 07 10:10:20.528303 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.528226 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-ca-trust-extracted\") pod \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " Sep 07 10:10:20.528545 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.528311 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-certificates\") pod \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " Sep 07 10:10:20.528545 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.528350 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-tls\") pod \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\" (UID: \"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1\") " Sep 07 10:10:20.528781 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.528748 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "1e0a434b-6047-48f0-a1d9-0e9147ddb9b1" (UID: "1e0a434b-6047-48f0-a1d9-0e9147ddb9b1"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 10:10:20.528876 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.528779 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "1e0a434b-6047-48f0-a1d9-0e9147ddb9b1" (UID: "1e0a434b-6047-48f0-a1d9-0e9147ddb9b1"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 10:10:20.530671 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.530634 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "1e0a434b-6047-48f0-a1d9-0e9147ddb9b1" (UID: "1e0a434b-6047-48f0-a1d9-0e9147ddb9b1"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:10:20.530671 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.530650 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1e0a434b-6047-48f0-a1d9-0e9147ddb9b1" (UID: "1e0a434b-6047-48f0-a1d9-0e9147ddb9b1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:10:20.530813 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.530702 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "1e0a434b-6047-48f0-a1d9-0e9147ddb9b1" (UID: "1e0a434b-6047-48f0-a1d9-0e9147ddb9b1"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:10:20.530923 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.530900 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "1e0a434b-6047-48f0-a1d9-0e9147ddb9b1" (UID: "1e0a434b-6047-48f0-a1d9-0e9147ddb9b1"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:10:20.531094 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.531066 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-kube-api-access-sw4j7" (OuterVolumeSpecName: "kube-api-access-sw4j7") pod "1e0a434b-6047-48f0-a1d9-0e9147ddb9b1" (UID: "1e0a434b-6047-48f0-a1d9-0e9147ddb9b1"). InnerVolumeSpecName "kube-api-access-sw4j7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:10:20.537500 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.537475 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "1e0a434b-6047-48f0-a1d9-0e9147ddb9b1" (UID: "1e0a434b-6047-48f0-a1d9-0e9147ddb9b1"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 10:10:20.629128 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.629088 2552 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-image-registry-private-configuration\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:10:20.629128 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.629117 2552 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-ca-trust-extracted\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:10:20.629128 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.629126 2552 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-certificates\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:10:20.629128 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.629138 2552 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-registry-tls\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:10:20.629395 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.629147 2552 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-bound-sa-token\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:10:20.629395 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.629155 2552 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-installation-pull-secrets\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:10:20.629395 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.629163 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sw4j7\" (UniqueName: \"kubernetes.io/projected/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-kube-api-access-sw4j7\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:10:20.629395 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:20.629171 2552 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1-trusted-ca\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:10:21.164283 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:21.164234 2552 generic.go:356] "Generic (PLEG): container finished" podID="1e0a434b-6047-48f0-a1d9-0e9147ddb9b1" containerID="cf7f1e72b001dac085b03716f0ab23dc6db165da9833dd49a1e9b02bb1ec2c7a" exitCode=0 Sep 07 10:10:21.164476 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:21.164302 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" event={"ID":"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1","Type":"ContainerDied","Data":"cf7f1e72b001dac085b03716f0ab23dc6db165da9833dd49a1e9b02bb1ec2c7a"} Sep 07 10:10:21.164476 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:21.164309 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" Sep 07 10:10:21.164476 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:21.164350 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75cc47b89c-ts6dd" event={"ID":"1e0a434b-6047-48f0-a1d9-0e9147ddb9b1","Type":"ContainerDied","Data":"c3073102d7fa0f27b95f154184064c7423cf853f0f60b1b1a470a0b3b1c1f16a"} Sep 07 10:10:21.164476 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:21.164377 2552 scope.go:117] "RemoveContainer" containerID="cf7f1e72b001dac085b03716f0ab23dc6db165da9833dd49a1e9b02bb1ec2c7a" Sep 07 10:10:21.172475 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:21.172458 2552 scope.go:117] "RemoveContainer" containerID="cf7f1e72b001dac085b03716f0ab23dc6db165da9833dd49a1e9b02bb1ec2c7a" Sep 07 10:10:21.172774 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:10:21.172753 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf7f1e72b001dac085b03716f0ab23dc6db165da9833dd49a1e9b02bb1ec2c7a\": container with ID starting with cf7f1e72b001dac085b03716f0ab23dc6db165da9833dd49a1e9b02bb1ec2c7a not found: ID does not exist" containerID="cf7f1e72b001dac085b03716f0ab23dc6db165da9833dd49a1e9b02bb1ec2c7a" Sep 07 10:10:21.172826 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:21.172781 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf7f1e72b001dac085b03716f0ab23dc6db165da9833dd49a1e9b02bb1ec2c7a"} err="failed to get container status \"cf7f1e72b001dac085b03716f0ab23dc6db165da9833dd49a1e9b02bb1ec2c7a\": rpc error: code = NotFound desc = could not find container \"cf7f1e72b001dac085b03716f0ab23dc6db165da9833dd49a1e9b02bb1ec2c7a\": container with ID starting with cf7f1e72b001dac085b03716f0ab23dc6db165da9833dd49a1e9b02bb1ec2c7a not found: ID does not exist" Sep 07 10:10:21.183456 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:21.183433 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75cc47b89c-ts6dd"] Sep 07 10:10:21.189168 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:21.189148 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-75cc47b89c-ts6dd"] Sep 07 10:10:21.521164 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:21.521135 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e0a434b-6047-48f0-a1d9-0e9147ddb9b1" path="/var/lib/kubelet/pods/1e0a434b-6047-48f0-a1d9-0e9147ddb9b1/volumes" Sep 07 10:10:40.403700 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:10:40.403641 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-p5xgx" podUID="57c1502d-f287-4126-8d7a-ab800b49d84e" Sep 07 10:10:41.231745 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:41.231710 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p5xgx" Sep 07 10:10:44.902496 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:44.902470 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hwpns_b5f2976e-e1b6-4659-80a2-555511cc90e8/init-textfile/0.log" Sep 07 10:10:45.103659 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:45.103628 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hwpns_b5f2976e-e1b6-4659-80a2-555511cc90e8/node-exporter/0.log" Sep 07 10:10:45.302484 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:45.302454 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hwpns_b5f2976e-e1b6-4659-80a2-555511cc90e8/kube-rbac-proxy/0.log" Sep 07 10:10:45.303480 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:45.303458 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls\") pod \"dns-default-p5xgx\" (UID: \"57c1502d-f287-4126-8d7a-ab800b49d84e\") " pod="openshift-dns/dns-default-p5xgx" Sep 07 10:10:45.305749 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:45.305726 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57c1502d-f287-4126-8d7a-ab800b49d84e-metrics-tls\") pod \"dns-default-p5xgx\" (UID: \"57c1502d-f287-4126-8d7a-ab800b49d84e\") " pod="openshift-dns/dns-default-p5xgx" Sep 07 10:10:45.434569 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:45.434534 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5dlqq\"" Sep 07 10:10:45.443331 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:45.443312 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p5xgx" Sep 07 10:10:45.714680 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:45.714604 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-p5xgx"] Sep 07 10:10:45.717521 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:10:45.717494 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57c1502d_f287_4126_8d7a_ab800b49d84e.slice/crio-6a93f36458d15feb8f7823c4fdead88cd084c4083dde47cde429fd7678ecf2cc WatchSource:0}: Error finding container 6a93f36458d15feb8f7823c4fdead88cd084c4083dde47cde429fd7678ecf2cc: Status 404 returned error can't find the container with id 6a93f36458d15feb8f7823c4fdead88cd084c4083dde47cde429fd7678ecf2cc Sep 07 10:10:46.250714 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:46.250680 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p5xgx" event={"ID":"57c1502d-f287-4126-8d7a-ab800b49d84e","Type":"ContainerStarted","Data":"6a93f36458d15feb8f7823c4fdead88cd084c4083dde47cde429fd7678ecf2cc"} Sep 07 10:10:48.259073 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:48.259034 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p5xgx" event={"ID":"57c1502d-f287-4126-8d7a-ab800b49d84e","Type":"ContainerStarted","Data":"6e8f5106131f5d20a2d20723559cb62514b7e1ac7c7382e0945ded96b01d51e5"} Sep 07 10:10:48.259528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:48.259089 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p5xgx" event={"ID":"57c1502d-f287-4126-8d7a-ab800b49d84e","Type":"ContainerStarted","Data":"db330102641c958b464ea1ae78edb263b4cd0a32af141a264cf71360f7b9f18e"} Sep 07 10:10:48.259528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:48.259192 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-p5xgx" Sep 07 10:10:48.275642 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:48.275587 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-p5xgx" podStartSLOduration=129.821874706 podStartE2EDuration="2m11.275571755s" totalImagesPullingTime="1.453697049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:08:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:10:48.273772718 +0000 UTC m=+165.242184512" watchObservedRunningTime="2026-09-07 10:10:48.275571755 +0000 UTC m=+165.243983579" Sep 07 10:10:49.264057 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:49.264023 2552 generic.go:356] "Generic (PLEG): container finished" podID="29bcf16b-9878-4b5b-859e-d2cf6be8ff9d" containerID="5ff0039f1daa39c4355cd10d0fba8af6dc027b8af92323f866deea8ce99d7113" exitCode=0 Sep 07 10:10:49.264460 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:49.264092 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5xkkz" event={"ID":"29bcf16b-9878-4b5b-859e-d2cf6be8ff9d","Type":"ContainerDied","Data":"5ff0039f1daa39c4355cd10d0fba8af6dc027b8af92323f866deea8ce99d7113"} Sep 07 10:10:49.264536 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:49.264517 2552 scope.go:117] "RemoveContainer" containerID="5ff0039f1daa39c4355cd10d0fba8af6dc027b8af92323f866deea8ce99d7113" Sep 07 10:10:50.269791 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:50.269755 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5xkkz" event={"ID":"29bcf16b-9878-4b5b-859e-d2cf6be8ff9d","Type":"ContainerStarted","Data":"4c41405e505f79ab745b1a24e8e9b071210aa3180fc0115dd08cc6646427f43a"} Sep 07 10:10:52.102856 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:52.102828 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-p5xgx_57c1502d-f287-4126-8d7a-ab800b49d84e/dns/0.log" Sep 07 10:10:52.302942 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:52.302912 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-p5xgx_57c1502d-f287-4126-8d7a-ab800b49d84e/kube-rbac-proxy/0.log" Sep 07 10:10:52.502448 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:52.502418 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kqh2w_f9e1b00d-e190-452a-8322-333f9397c2fa/dns-node-resolver/0.log" Sep 07 10:10:53.902857 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:53.902824 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v9stm_aff3118f-d191-4990-b60b-958c0c686aec/node-ca/0.log" Sep 07 10:10:54.306752 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:54.306723 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cdkr7_24404e76-4654-44b0-ab31-06822ef18722/serve-healthcheck-canary/0.log" Sep 07 10:10:58.266331 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:10:58.266228 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-p5xgx" Sep 07 10:13:03.425487 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:13:03.425457 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/ovn-acl-logging/0.log" Sep 07 10:13:03.433174 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:13:03.433137 2552 kubelet.go:1648] "Image garbage collection succeeded" Sep 07 10:13:03.436904 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:13:03.436882 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/ovn-acl-logging/0.log" Sep 07 10:14:50.415630 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:14:50.415594 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-85bbffd6d8-vkwgv"] Sep 07 10:14:50.416023 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:14:50.415822 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e0a434b-6047-48f0-a1d9-0e9147ddb9b1" containerName="registry" Sep 07 10:14:50.416023 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:14:50.415833 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e0a434b-6047-48f0-a1d9-0e9147ddb9b1" containerName="registry" Sep 07 10:14:50.416023 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:14:50.415879 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e0a434b-6047-48f0-a1d9-0e9147ddb9b1" containerName="registry" Sep 07 10:14:50.418558 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:14:50.418539 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-85bbffd6d8-vkwgv" Sep 07 10:14:50.420834 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:14:50.420803 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 07 10:14:50.421559 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:14:50.421538 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-759dl\"" Sep 07 10:14:50.421559 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:14:50.421550 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 07 10:14:50.428466 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:14:50.428442 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-85bbffd6d8-vkwgv"] Sep 07 10:14:50.527725 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:14:50.527692 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvj74\" (UniqueName: \"kubernetes.io/projected/3f5a496d-f7aa-4a13-be93-e662b0242a4e-kube-api-access-rvj74\") pod \"feast-operator-controller-manager-85bbffd6d8-vkwgv\" (UID: \"3f5a496d-f7aa-4a13-be93-e662b0242a4e\") " pod="feast-operator-system/feast-operator-controller-manager-85bbffd6d8-vkwgv" Sep 07 10:14:50.628550 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:14:50.628511 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvj74\" (UniqueName: \"kubernetes.io/projected/3f5a496d-f7aa-4a13-be93-e662b0242a4e-kube-api-access-rvj74\") pod \"feast-operator-controller-manager-85bbffd6d8-vkwgv\" (UID: \"3f5a496d-f7aa-4a13-be93-e662b0242a4e\") " pod="feast-operator-system/feast-operator-controller-manager-85bbffd6d8-vkwgv" Sep 07 10:14:50.635746 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:14:50.635724 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvj74\" (UniqueName: \"kubernetes.io/projected/3f5a496d-f7aa-4a13-be93-e662b0242a4e-kube-api-access-rvj74\") pod \"feast-operator-controller-manager-85bbffd6d8-vkwgv\" (UID: \"3f5a496d-f7aa-4a13-be93-e662b0242a4e\") " pod="feast-operator-system/feast-operator-controller-manager-85bbffd6d8-vkwgv" Sep 07 10:14:50.728051 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:14:50.728012 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-85bbffd6d8-vkwgv" Sep 07 10:14:50.994363 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:14:50.994282 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-85bbffd6d8-vkwgv"] Sep 07 10:14:50.997213 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:14:50.997190 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f5a496d_f7aa_4a13_be93_e662b0242a4e.slice/crio-6a4a8bf1b81eadefef3e4c49a8e4145a96a7d57913b6aa8ce090b370308ddb78 WatchSource:0}: Error finding container 6a4a8bf1b81eadefef3e4c49a8e4145a96a7d57913b6aa8ce090b370308ddb78: Status 404 returned error can't find the container with id 6a4a8bf1b81eadefef3e4c49a8e4145a96a7d57913b6aa8ce090b370308ddb78 Sep 07 10:14:50.999659 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:14:50.999643 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 07 10:14:51.062232 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:14:51.062197 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-85bbffd6d8-vkwgv" event={"ID":"3f5a496d-f7aa-4a13-be93-e662b0242a4e","Type":"ContainerStarted","Data":"6a4a8bf1b81eadefef3e4c49a8e4145a96a7d57913b6aa8ce090b370308ddb78"} Sep 07 10:14:54.075155 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:14:54.075120 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-85bbffd6d8-vkwgv" event={"ID":"3f5a496d-f7aa-4a13-be93-e662b0242a4e","Type":"ContainerStarted","Data":"15015645feb7dea70aa03ef881207175227b899e7313b5541b4038df0043d746"} Sep 07 10:14:54.075523 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:14:54.075270 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-85bbffd6d8-vkwgv" Sep 07 10:14:54.089090 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:14:54.089044 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-85bbffd6d8-vkwgv" podStartSLOduration=1.769319732 podStartE2EDuration="4.089031023s" totalImagesPullingTime="2.319711291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:14:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:14:54.088463772 +0000 UTC m=+411.056875564" watchObservedRunningTime="2026-09-07 10:14:54.089031023 +0000 UTC m=+411.057442815" Sep 07 10:15:05.081400 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:05.081363 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-85bbffd6d8-vkwgv" Sep 07 10:15:26.529362 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.529272 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb"] Sep 07 10:15:26.532834 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.532809 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:15:26.534951 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.534920 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 07 10:15:26.535077 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.534999 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 07 10:15:26.535219 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.535196 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-8pk5l\"" Sep 07 10:15:26.535321 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.535304 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 07 10:15:26.535408 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.535391 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 07 10:15:26.535522 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.535508 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 07 10:15:26.535583 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.535569 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 07 10:15:26.535697 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.535683 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 07 10:15:26.547762 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.547737 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb"] Sep 07 10:15:26.683518 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.683480 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-offline-tls\") pod \"feast-simple-feast-setup-7d48777b49-dmrzb\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:15:26.683719 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.683533 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-ui-tls\") pod \"feast-simple-feast-setup-7d48777b49-dmrzb\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:15:26.683719 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.683585 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-registry-tls\") pod \"feast-simple-feast-setup-7d48777b49-dmrzb\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:15:26.683719 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.683609 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xtd2\" (UniqueName: \"kubernetes.io/projected/68528213-3bd2-4add-a80a-f96edf64be9d-kube-api-access-9xtd2\") pod \"feast-simple-feast-setup-7d48777b49-dmrzb\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:15:26.683719 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.683651 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/68528213-3bd2-4add-a80a-f96edf64be9d-feast-data\") pod \"feast-simple-feast-setup-7d48777b49-dmrzb\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:15:26.683719 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.683687 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-online-tls\") pod \"feast-simple-feast-setup-7d48777b49-dmrzb\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:15:26.784314 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.784211 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-online-tls\") pod \"feast-simple-feast-setup-7d48777b49-dmrzb\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:15:26.784314 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.784308 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-offline-tls\") pod \"feast-simple-feast-setup-7d48777b49-dmrzb\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:15:26.784539 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.784333 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-ui-tls\") pod \"feast-simple-feast-setup-7d48777b49-dmrzb\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:15:26.784539 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.784351 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-registry-tls\") pod \"feast-simple-feast-setup-7d48777b49-dmrzb\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:15:26.784539 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.784368 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xtd2\" (UniqueName: \"kubernetes.io/projected/68528213-3bd2-4add-a80a-f96edf64be9d-kube-api-access-9xtd2\") pod \"feast-simple-feast-setup-7d48777b49-dmrzb\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:15:26.784539 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:15:26.784394 2552 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Sep 07 10:15:26.784539 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:15:26.784474 2552 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 07 10:15:26.784539 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:15:26.784480 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-online-tls podName:68528213-3bd2-4add-a80a-f96edf64be9d nodeName:}" failed. No retries permitted until 2026-09-07 10:15:27.284458343 +0000 UTC m=+444.252870128 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-online-tls") pod "feast-simple-feast-setup-7d48777b49-dmrzb" (UID: "68528213-3bd2-4add-a80a-f96edf64be9d") : secret "feast-simple-feast-setup-online-tls" not found Sep 07 10:15:26.784539 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:15:26.784551 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-ui-tls podName:68528213-3bd2-4add-a80a-f96edf64be9d nodeName:}" failed. No retries permitted until 2026-09-07 10:15:27.284533714 +0000 UTC m=+444.252945498 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-ui-tls") pod "feast-simple-feast-setup-7d48777b49-dmrzb" (UID: "68528213-3bd2-4add-a80a-f96edf64be9d") : secret "feast-simple-feast-setup-ui-tls" not found Sep 07 10:15:26.784539 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.784398 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/68528213-3bd2-4add-a80a-f96edf64be9d-feast-data\") pod \"feast-simple-feast-setup-7d48777b49-dmrzb\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:15:26.784978 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.784953 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/68528213-3bd2-4add-a80a-f96edf64be9d-feast-data\") pod \"feast-simple-feast-setup-7d48777b49-dmrzb\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:15:26.786993 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.786962 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-registry-tls\") pod \"feast-simple-feast-setup-7d48777b49-dmrzb\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:15:26.787283 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.787266 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-offline-tls\") pod \"feast-simple-feast-setup-7d48777b49-dmrzb\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:15:26.811511 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:26.811475 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xtd2\" (UniqueName: \"kubernetes.io/projected/68528213-3bd2-4add-a80a-f96edf64be9d-kube-api-access-9xtd2\") pod \"feast-simple-feast-setup-7d48777b49-dmrzb\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:15:27.288711 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:27.288672 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-ui-tls\") pod \"feast-simple-feast-setup-7d48777b49-dmrzb\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:15:27.288897 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:27.288748 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-online-tls\") pod \"feast-simple-feast-setup-7d48777b49-dmrzb\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:15:27.291153 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:27.291123 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-ui-tls\") pod \"feast-simple-feast-setup-7d48777b49-dmrzb\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:15:27.291284 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:27.291150 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-online-tls\") pod \"feast-simple-feast-setup-7d48777b49-dmrzb\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:15:27.443136 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:27.443098 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:15:27.720131 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:27.720097 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb"] Sep 07 10:15:27.722323 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:15:27.722293 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod68528213_3bd2_4add_a80a_f96edf64be9d.slice/crio-24c1d9c3d728e4b1236f7c4017f40d1bde33f210650ee1e130270b6b49266f2b WatchSource:0}: Error finding container 24c1d9c3d728e4b1236f7c4017f40d1bde33f210650ee1e130270b6b49266f2b: Status 404 returned error can't find the container with id 24c1d9c3d728e4b1236f7c4017f40d1bde33f210650ee1e130270b6b49266f2b Sep 07 10:15:28.187566 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:28.187473 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" event={"ID":"68528213-3bd2-4add-a80a-f96edf64be9d","Type":"ContainerStarted","Data":"24c1d9c3d728e4b1236f7c4017f40d1bde33f210650ee1e130270b6b49266f2b"} Sep 07 10:15:51.278707 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:51.278667 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" event={"ID":"68528213-3bd2-4add-a80a-f96edf64be9d","Type":"ContainerStarted","Data":"dcf8ee383b1b6fa7dbf6d1588d4f8858f2babb0208ac2544c9e49d32bbccdf8f"} Sep 07 10:15:56.297567 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:56.297537 2552 generic.go:356] "Generic (PLEG): container finished" podID="68528213-3bd2-4add-a80a-f96edf64be9d" containerID="dcf8ee383b1b6fa7dbf6d1588d4f8858f2babb0208ac2544c9e49d32bbccdf8f" exitCode=0 Sep 07 10:15:56.298044 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:56.297617 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" event={"ID":"68528213-3bd2-4add-a80a-f96edf64be9d","Type":"ContainerDied","Data":"dcf8ee383b1b6fa7dbf6d1588d4f8858f2babb0208ac2544c9e49d32bbccdf8f"} Sep 07 10:15:57.302857 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:15:57.302822 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" event={"ID":"68528213-3bd2-4add-a80a-f96edf64be9d","Type":"ContainerStarted","Data":"aef981a01e414dcf548b386ae6640d7a34c3ae2cd1bfaddd271377e33b21be1b"} Sep 07 10:16:02.325183 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:02.325150 2552 generic.go:356] "Generic (PLEG): container finished" podID="68528213-3bd2-4add-a80a-f96edf64be9d" containerID="aef981a01e414dcf548b386ae6640d7a34c3ae2cd1bfaddd271377e33b21be1b" exitCode=0 Sep 07 10:16:02.325584 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:02.325218 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" event={"ID":"68528213-3bd2-4add-a80a-f96edf64be9d","Type":"ContainerDied","Data":"aef981a01e414dcf548b386ae6640d7a34c3ae2cd1bfaddd271377e33b21be1b"} Sep 07 10:16:03.335201 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:03.335156 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" event={"ID":"68528213-3bd2-4add-a80a-f96edf64be9d","Type":"ContainerStarted","Data":"52398106f2aba9a153419676f7f802e401f891f42f9c1cf447a9d851dc5d6081"} Sep 07 10:16:03.335701 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:03.335222 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" event={"ID":"68528213-3bd2-4add-a80a-f96edf64be9d","Type":"ContainerStarted","Data":"0a559b1027d49d0f7fe64ce7b97b487c32f84b66df9cb05634cf21a7274732ea"} Sep 07 10:16:03.335701 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:03.335264 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" event={"ID":"68528213-3bd2-4add-a80a-f96edf64be9d","Type":"ContainerStarted","Data":"ec0e5f50030852c2632a2fed3bcb6da2961ee86bfba47bc1318024cfe34c7b2a"} Sep 07 10:16:03.335701 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:03.335284 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" event={"ID":"68528213-3bd2-4add-a80a-f96edf64be9d","Type":"ContainerStarted","Data":"a67d8df0db74ebfa90c829e1406639939e329f238757cb063e7952a342dae787"} Sep 07 10:16:03.357933 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:03.357415 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" podStartSLOduration=14.229732434 podStartE2EDuration="37.357392597s" totalImagesPullingTime="23.127660163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:15:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:16:03.356360891 +0000 UTC m=+480.324772687" watchObservedRunningTime="2026-09-07 10:16:03.357392597 +0000 UTC m=+480.325804393" Sep 07 10:16:03.443990 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:03.443930 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:16:03.444178 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:03.444004 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:16:03.444178 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:03.444025 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:16:03.444178 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:03.444046 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:16:03.446132 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:03.446085 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.15:8816: connect: connection refused" Sep 07 10:16:03.446293 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:03.446174 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.15:6571: connect: connection refused" Sep 07 10:16:03.446293 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:03.446194 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.15:6567/health\": dial tcp 10.134.0.15:6567: connect: connection refused" Sep 07 10:16:03.446293 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:03.446279 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.15:8443: connect: connection refused" Sep 07 10:16:06.443834 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:06.443786 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.15:8816: connect: connection refused" Sep 07 10:16:06.444434 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:06.443792 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.15:8443: connect: connection refused" Sep 07 10:16:06.444635 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:06.443801 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.15:6567/health\": dial tcp 10.134.0.15:6567: connect: connection refused" Sep 07 10:16:06.444777 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:06.443786 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.15:6571: connect: connection refused" Sep 07 10:16:07.443675 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:07.443631 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:16:07.443971 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:07.443944 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:16:07.443971 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:07.443975 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:16:07.444438 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:07.443995 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:16:09.443468 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:09.443417 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.15:8443: connect: connection refused" Sep 07 10:16:09.444055 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:09.443417 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.15:8816: connect: connection refused" Sep 07 10:16:09.444055 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:09.443417 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.15:6571: connect: connection refused" Sep 07 10:16:09.444055 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:09.444006 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.15:6567/health\": dial tcp 10.134.0.15:6567: connect: connection refused" Sep 07 10:16:12.445017 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:12.444979 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:16:12.445505 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:12.445078 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:16:12.445505 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:12.445309 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:16:12.445969 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:12.445946 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:16:12.446070 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:12.446003 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:16:12.446143 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:12.446122 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:16:12.450540 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:12.450521 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:16:13.385704 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:16:13.385673 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:17:23.814170 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:23.814094 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb"] Sep 07 10:17:23.814698 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:23.814408 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="registry" containerID="cri-o://a67d8df0db74ebfa90c829e1406639939e329f238757cb063e7952a342dae787" gracePeriod=30 Sep 07 10:17:23.814698 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:23.814528 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="online" containerID="cri-o://ec0e5f50030852c2632a2fed3bcb6da2961ee86bfba47bc1318024cfe34c7b2a" gracePeriod=30 Sep 07 10:17:23.814698 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:23.814531 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="ui" containerID="cri-o://52398106f2aba9a153419676f7f802e401f891f42f9c1cf447a9d851dc5d6081" gracePeriod=30 Sep 07 10:17:23.814698 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:23.814529 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="offline" containerID="cri-o://0a559b1027d49d0f7fe64ce7b97b487c32f84b66df9cb05634cf21a7274732ea" gracePeriod=30 Sep 07 10:17:24.186084 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.185978 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m"] Sep 07 10:17:24.189432 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.189400 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:24.192016 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.191988 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-d9dnh\"" Sep 07 10:17:24.204850 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.203425 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m"] Sep 07 10:17:24.224504 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.221769 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-offline-tls\") pod \"feast-simple-feast-setup-7d48777b49-2sb8m\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:24.224504 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.221831 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/21fb0061-410a-4e3d-a45a-18de4d4a31b9-feast-data\") pod \"feast-simple-feast-setup-7d48777b49-2sb8m\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:24.224504 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.221949 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-online-tls\") pod \"feast-simple-feast-setup-7d48777b49-2sb8m\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:24.224504 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.222054 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-registry-tls\") pod \"feast-simple-feast-setup-7d48777b49-2sb8m\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:24.224504 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.222121 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnf9z\" (UniqueName: \"kubernetes.io/projected/21fb0061-410a-4e3d-a45a-18de4d4a31b9-kube-api-access-rnf9z\") pod \"feast-simple-feast-setup-7d48777b49-2sb8m\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:24.224504 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.222156 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-ui-tls\") pod \"feast-simple-feast-setup-7d48777b49-2sb8m\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:24.323429 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.323386 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-online-tls\") pod \"feast-simple-feast-setup-7d48777b49-2sb8m\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:24.323645 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.323451 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-registry-tls\") pod \"feast-simple-feast-setup-7d48777b49-2sb8m\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:24.323645 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.323495 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnf9z\" (UniqueName: \"kubernetes.io/projected/21fb0061-410a-4e3d-a45a-18de4d4a31b9-kube-api-access-rnf9z\") pod \"feast-simple-feast-setup-7d48777b49-2sb8m\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:24.323645 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:17:24.323613 2552 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 07 10:17:24.323791 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:17:24.323699 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-registry-tls podName:21fb0061-410a-4e3d-a45a-18de4d4a31b9 nodeName:}" failed. No retries permitted until 2026-09-07 10:17:24.823677517 +0000 UTC m=+561.792089303 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-registry-tls") pod "feast-simple-feast-setup-7d48777b49-2sb8m" (UID: "21fb0061-410a-4e3d-a45a-18de4d4a31b9") : secret "feast-simple-feast-setup-registry-tls" not found Sep 07 10:17:24.323886 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.323838 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-ui-tls\") pod \"feast-simple-feast-setup-7d48777b49-2sb8m\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:24.323959 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.323906 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-offline-tls\") pod \"feast-simple-feast-setup-7d48777b49-2sb8m\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:24.323959 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.323930 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/21fb0061-410a-4e3d-a45a-18de4d4a31b9-feast-data\") pod \"feast-simple-feast-setup-7d48777b49-2sb8m\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:24.324056 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:17:24.323979 2552 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 07 10:17:24.324056 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:17:24.324042 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-ui-tls podName:21fb0061-410a-4e3d-a45a-18de4d4a31b9 nodeName:}" failed. No retries permitted until 2026-09-07 10:17:24.824022039 +0000 UTC m=+561.792433812 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-ui-tls") pod "feast-simple-feast-setup-7d48777b49-2sb8m" (UID: "21fb0061-410a-4e3d-a45a-18de4d4a31b9") : secret "feast-simple-feast-setup-ui-tls" not found Sep 07 10:17:24.324442 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.324418 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/21fb0061-410a-4e3d-a45a-18de4d4a31b9-feast-data\") pod \"feast-simple-feast-setup-7d48777b49-2sb8m\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:24.326514 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.326488 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-online-tls\") pod \"feast-simple-feast-setup-7d48777b49-2sb8m\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:24.326929 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.326906 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-offline-tls\") pod \"feast-simple-feast-setup-7d48777b49-2sb8m\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:24.335075 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.335022 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnf9z\" (UniqueName: \"kubernetes.io/projected/21fb0061-410a-4e3d-a45a-18de4d4a31b9-kube-api-access-rnf9z\") pod \"feast-simple-feast-setup-7d48777b49-2sb8m\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:24.635037 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.635003 2552 generic.go:356] "Generic (PLEG): container finished" podID="68528213-3bd2-4add-a80a-f96edf64be9d" containerID="52398106f2aba9a153419676f7f802e401f891f42f9c1cf447a9d851dc5d6081" exitCode=0 Sep 07 10:17:24.635219 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.635079 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" event={"ID":"68528213-3bd2-4add-a80a-f96edf64be9d","Type":"ContainerDied","Data":"52398106f2aba9a153419676f7f802e401f891f42f9c1cf447a9d851dc5d6081"} Sep 07 10:17:24.827801 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.827719 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-ui-tls\") pod \"feast-simple-feast-setup-7d48777b49-2sb8m\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:24.827801 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.827787 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-registry-tls\") pod \"feast-simple-feast-setup-7d48777b49-2sb8m\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:24.828171 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:17:24.827890 2552 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 07 10:17:24.828171 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:17:24.827959 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-registry-tls podName:21fb0061-410a-4e3d-a45a-18de4d4a31b9 nodeName:}" failed. No retries permitted until 2026-09-07 10:17:25.827942652 +0000 UTC m=+562.796354423 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-registry-tls") pod "feast-simple-feast-setup-7d48777b49-2sb8m" (UID: "21fb0061-410a-4e3d-a45a-18de4d4a31b9") : secret "feast-simple-feast-setup-registry-tls" not found Sep 07 10:17:24.830114 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:24.830092 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-ui-tls\") pod \"feast-simple-feast-setup-7d48777b49-2sb8m\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:25.640958 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:25.640927 2552 generic.go:356] "Generic (PLEG): container finished" podID="68528213-3bd2-4add-a80a-f96edf64be9d" containerID="ec0e5f50030852c2632a2fed3bcb6da2961ee86bfba47bc1318024cfe34c7b2a" exitCode=0 Sep 07 10:17:25.641115 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:25.640971 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" event={"ID":"68528213-3bd2-4add-a80a-f96edf64be9d","Type":"ContainerDied","Data":"ec0e5f50030852c2632a2fed3bcb6da2961ee86bfba47bc1318024cfe34c7b2a"} Sep 07 10:17:25.835699 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:25.835659 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-registry-tls\") pod \"feast-simple-feast-setup-7d48777b49-2sb8m\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:25.838016 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:25.837995 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-registry-tls\") pod \"feast-simple-feast-setup-7d48777b49-2sb8m\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:26.013523 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:26.013486 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:26.286053 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:26.286028 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m"] Sep 07 10:17:26.288545 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:17:26.288517 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21fb0061_410a_4e3d_a45a_18de4d4a31b9.slice/crio-859b39e0899b91b9550cda29210573e1bc06d66c6317512eb8dd12f5e90a15ad WatchSource:0}: Error finding container 859b39e0899b91b9550cda29210573e1bc06d66c6317512eb8dd12f5e90a15ad: Status 404 returned error can't find the container with id 859b39e0899b91b9550cda29210573e1bc06d66c6317512eb8dd12f5e90a15ad Sep 07 10:17:26.646287 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:26.646192 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" event={"ID":"21fb0061-410a-4e3d-a45a-18de4d4a31b9","Type":"ContainerStarted","Data":"f3d7b3374dac14af3193cce0ab19f22c9da4cc2914ef873585728bfe221d40f0"} Sep 07 10:17:26.646287 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:26.646237 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" event={"ID":"21fb0061-410a-4e3d-a45a-18de4d4a31b9","Type":"ContainerStarted","Data":"859b39e0899b91b9550cda29210573e1bc06d66c6317512eb8dd12f5e90a15ad"} Sep 07 10:17:31.671213 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:31.671177 2552 generic.go:356] "Generic (PLEG): container finished" podID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerID="f3d7b3374dac14af3193cce0ab19f22c9da4cc2914ef873585728bfe221d40f0" exitCode=0 Sep 07 10:17:31.671629 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:31.671232 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" event={"ID":"21fb0061-410a-4e3d-a45a-18de4d4a31b9","Type":"ContainerDied","Data":"f3d7b3374dac14af3193cce0ab19f22c9da4cc2914ef873585728bfe221d40f0"} Sep 07 10:17:32.445959 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:32.445912 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.15:8443: connect: connection refused" Sep 07 10:17:32.676380 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:32.676346 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" event={"ID":"21fb0061-410a-4e3d-a45a-18de4d4a31b9","Type":"ContainerStarted","Data":"c4a69362746a16aa91d0142c3dfd76c98c00b1a7bf5c7c38376b70b4ba4e896c"} Sep 07 10:17:33.382619 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:33.382577 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.15:6567/health\": dial tcp 10.134.0.15:6567: connect: connection refused" Sep 07 10:17:37.699409 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:37.699375 2552 generic.go:356] "Generic (PLEG): container finished" podID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerID="c4a69362746a16aa91d0142c3dfd76c98c00b1a7bf5c7c38376b70b4ba4e896c" exitCode=0 Sep 07 10:17:37.699775 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:37.699431 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" event={"ID":"21fb0061-410a-4e3d-a45a-18de4d4a31b9","Type":"ContainerDied","Data":"c4a69362746a16aa91d0142c3dfd76c98c00b1a7bf5c7c38376b70b4ba4e896c"} Sep 07 10:17:38.709192 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:38.709133 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" event={"ID":"21fb0061-410a-4e3d-a45a-18de4d4a31b9","Type":"ContainerStarted","Data":"636aa17b687fcdb6ca91d401172d97b8e62f82544c3ca5487c2ac387ded3a162"} Sep 07 10:17:38.709715 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:38.709212 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" event={"ID":"21fb0061-410a-4e3d-a45a-18de4d4a31b9","Type":"ContainerStarted","Data":"3e760dd7a02f493693d0bf3b414dd770440786299f9eb6e5ef4db23f22872e95"} Sep 07 10:17:38.709715 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:38.709261 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" event={"ID":"21fb0061-410a-4e3d-a45a-18de4d4a31b9","Type":"ContainerStarted","Data":"3bee0674ba73a3abbc4feb0aa8dc87d3cb5f9680681627629cbd4e7114289f2a"} Sep 07 10:17:38.709715 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:38.709285 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" event={"ID":"21fb0061-410a-4e3d-a45a-18de4d4a31b9","Type":"ContainerStarted","Data":"41db28eb567f1f2637cdfe440df0e6365b3102ec6de5f4ffedd51db514f714e2"} Sep 07 10:17:38.732424 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:38.732357 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" podStartSLOduration=14.732336137 podStartE2EDuration="14.732336137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:17:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:17:38.731637965 +0000 UTC m=+575.700049792" watchObservedRunningTime="2026-09-07 10:17:38.732336137 +0000 UTC m=+575.700747930" Sep 07 10:17:41.014330 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:41.014163 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:41.014330 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:41.014259 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:41.014330 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:41.014283 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:41.014330 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:41.014301 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:41.016421 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:41.016272 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.16:8443: connect: connection refused" Sep 07 10:17:41.016421 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:41.016307 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.16:6571: connect: connection refused" Sep 07 10:17:41.016421 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:41.016333 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.16:8816: connect: connection refused" Sep 07 10:17:41.016421 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:41.016316 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.16:6567/health\": dial tcp 10.134.0.16:6567: connect: connection refused" Sep 07 10:17:42.446657 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:42.446608 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.15:8443: connect: connection refused" Sep 07 10:17:43.383080 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:43.383024 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.15:6567/health\": dial tcp 10.134.0.15:6567: connect: connection refused" Sep 07 10:17:44.014433 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:44.014385 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.16:8443: connect: connection refused" Sep 07 10:17:44.014925 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:44.014472 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.16:8816: connect: connection refused" Sep 07 10:17:44.014925 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:44.014399 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.16:6567/health\": dial tcp 10.134.0.16:6567: connect: connection refused" Sep 07 10:17:44.014925 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:44.014385 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.16:6571: connect: connection refused" Sep 07 10:17:46.013811 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:46.013772 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:46.014191 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:46.013830 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:46.014191 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:46.013852 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:46.014191 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:46.013868 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:47.015141 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:47.015108 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:47.015580 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:47.015349 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:47.015580 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:47.015467 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:47.020360 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:47.020338 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:47.759449 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:47.759414 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:47.759650 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:47.759497 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:47.759650 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:47.759545 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:47.762498 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:47.762475 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:17:52.446184 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:52.446142 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.15:8443: connect: connection refused" Sep 07 10:17:52.446727 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:52.446320 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:17:53.382698 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:53.382657 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.15:6567/health\": dial tcp 10.134.0.15:6567: connect: connection refused" Sep 07 10:17:53.382875 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:53.382765 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:17:54.462399 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.462372 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:17:54.558657 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.558616 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xtd2\" (UniqueName: \"kubernetes.io/projected/68528213-3bd2-4add-a80a-f96edf64be9d-kube-api-access-9xtd2\") pod \"68528213-3bd2-4add-a80a-f96edf64be9d\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " Sep 07 10:17:54.558657 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.558664 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-registry-tls\") pod \"68528213-3bd2-4add-a80a-f96edf64be9d\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " Sep 07 10:17:54.558905 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.558683 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/68528213-3bd2-4add-a80a-f96edf64be9d-feast-data\") pod \"68528213-3bd2-4add-a80a-f96edf64be9d\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " Sep 07 10:17:54.558905 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.558716 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-ui-tls\") pod \"68528213-3bd2-4add-a80a-f96edf64be9d\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " Sep 07 10:17:54.558905 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.558742 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-offline-tls\") pod \"68528213-3bd2-4add-a80a-f96edf64be9d\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " Sep 07 10:17:54.558905 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.558766 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-online-tls\") pod \"68528213-3bd2-4add-a80a-f96edf64be9d\" (UID: \"68528213-3bd2-4add-a80a-f96edf64be9d\") " Sep 07 10:17:54.559328 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.559293 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68528213-3bd2-4add-a80a-f96edf64be9d-feast-data" (OuterVolumeSpecName: "feast-data") pod "68528213-3bd2-4add-a80a-f96edf64be9d" (UID: "68528213-3bd2-4add-a80a-f96edf64be9d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 10:17:54.561106 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.561074 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-online-tls" (OuterVolumeSpecName: "online-tls") pod "68528213-3bd2-4add-a80a-f96edf64be9d" (UID: "68528213-3bd2-4add-a80a-f96edf64be9d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:17:54.561203 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.561096 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "68528213-3bd2-4add-a80a-f96edf64be9d" (UID: "68528213-3bd2-4add-a80a-f96edf64be9d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:17:54.561203 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.561114 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "68528213-3bd2-4add-a80a-f96edf64be9d" (UID: "68528213-3bd2-4add-a80a-f96edf64be9d"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:17:54.561334 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.561268 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "68528213-3bd2-4add-a80a-f96edf64be9d" (UID: "68528213-3bd2-4add-a80a-f96edf64be9d"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:17:54.561334 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.561282 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68528213-3bd2-4add-a80a-f96edf64be9d-kube-api-access-9xtd2" (OuterVolumeSpecName: "kube-api-access-9xtd2") pod "68528213-3bd2-4add-a80a-f96edf64be9d" (UID: "68528213-3bd2-4add-a80a-f96edf64be9d"). InnerVolumeSpecName "kube-api-access-9xtd2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:17:54.659819 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.659739 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xtd2\" (UniqueName: \"kubernetes.io/projected/68528213-3bd2-4add-a80a-f96edf64be9d-kube-api-access-9xtd2\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:17:54.659819 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.659769 2552 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-registry-tls\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:17:54.659819 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.659779 2552 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/68528213-3bd2-4add-a80a-f96edf64be9d-feast-data\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:17:54.659819 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.659787 2552 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-ui-tls\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:17:54.659819 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.659795 2552 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-offline-tls\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:17:54.659819 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.659802 2552 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/68528213-3bd2-4add-a80a-f96edf64be9d-online-tls\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:17:54.787331 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.787301 2552 generic.go:356] "Generic (PLEG): container finished" podID="68528213-3bd2-4add-a80a-f96edf64be9d" containerID="0a559b1027d49d0f7fe64ce7b97b487c32f84b66df9cb05634cf21a7274732ea" exitCode=137 Sep 07 10:17:54.787331 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.787325 2552 generic.go:356] "Generic (PLEG): container finished" podID="68528213-3bd2-4add-a80a-f96edf64be9d" containerID="a67d8df0db74ebfa90c829e1406639939e329f238757cb063e7952a342dae787" exitCode=137 Sep 07 10:17:54.787584 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.787372 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" Sep 07 10:17:54.787584 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.787385 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" event={"ID":"68528213-3bd2-4add-a80a-f96edf64be9d","Type":"ContainerDied","Data":"0a559b1027d49d0f7fe64ce7b97b487c32f84b66df9cb05634cf21a7274732ea"} Sep 07 10:17:54.787584 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.787444 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" event={"ID":"68528213-3bd2-4add-a80a-f96edf64be9d","Type":"ContainerDied","Data":"a67d8df0db74ebfa90c829e1406639939e329f238757cb063e7952a342dae787"} Sep 07 10:17:54.787584 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.787466 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb" event={"ID":"68528213-3bd2-4add-a80a-f96edf64be9d","Type":"ContainerDied","Data":"24c1d9c3d728e4b1236f7c4017f40d1bde33f210650ee1e130270b6b49266f2b"} Sep 07 10:17:54.787584 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.787490 2552 scope.go:117] "RemoveContainer" containerID="52398106f2aba9a153419676f7f802e401f891f42f9c1cf447a9d851dc5d6081" Sep 07 10:17:54.795609 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.795593 2552 scope.go:117] "RemoveContainer" containerID="0a559b1027d49d0f7fe64ce7b97b487c32f84b66df9cb05634cf21a7274732ea" Sep 07 10:17:54.802763 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.802748 2552 scope.go:117] "RemoveContainer" containerID="ec0e5f50030852c2632a2fed3bcb6da2961ee86bfba47bc1318024cfe34c7b2a" Sep 07 10:17:54.807079 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.807059 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb"] Sep 07 10:17:54.810260 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.810233 2552 scope.go:117] "RemoveContainer" containerID="a67d8df0db74ebfa90c829e1406639939e329f238757cb063e7952a342dae787" Sep 07 10:17:54.813299 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.813269 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7d48777b49-dmrzb"] Sep 07 10:17:54.817431 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.817411 2552 scope.go:117] "RemoveContainer" containerID="aef981a01e414dcf548b386ae6640d7a34c3ae2cd1bfaddd271377e33b21be1b" Sep 07 10:17:54.823926 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.823910 2552 scope.go:117] "RemoveContainer" containerID="dcf8ee383b1b6fa7dbf6d1588d4f8858f2babb0208ac2544c9e49d32bbccdf8f" Sep 07 10:17:54.830410 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.830380 2552 scope.go:117] "RemoveContainer" containerID="52398106f2aba9a153419676f7f802e401f891f42f9c1cf447a9d851dc5d6081" Sep 07 10:17:54.830648 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:17:54.830631 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52398106f2aba9a153419676f7f802e401f891f42f9c1cf447a9d851dc5d6081\": container with ID starting with 52398106f2aba9a153419676f7f802e401f891f42f9c1cf447a9d851dc5d6081 not found: ID does not exist" containerID="52398106f2aba9a153419676f7f802e401f891f42f9c1cf447a9d851dc5d6081" Sep 07 10:17:54.830699 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.830665 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52398106f2aba9a153419676f7f802e401f891f42f9c1cf447a9d851dc5d6081"} err="failed to get container status \"52398106f2aba9a153419676f7f802e401f891f42f9c1cf447a9d851dc5d6081\": rpc error: code = NotFound desc = could not find container \"52398106f2aba9a153419676f7f802e401f891f42f9c1cf447a9d851dc5d6081\": container with ID starting with 52398106f2aba9a153419676f7f802e401f891f42f9c1cf447a9d851dc5d6081 not found: ID does not exist" Sep 07 10:17:54.830699 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.830681 2552 scope.go:117] "RemoveContainer" containerID="0a559b1027d49d0f7fe64ce7b97b487c32f84b66df9cb05634cf21a7274732ea" Sep 07 10:17:54.830908 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:17:54.830891 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a559b1027d49d0f7fe64ce7b97b487c32f84b66df9cb05634cf21a7274732ea\": container with ID starting with 0a559b1027d49d0f7fe64ce7b97b487c32f84b66df9cb05634cf21a7274732ea not found: ID does not exist" containerID="0a559b1027d49d0f7fe64ce7b97b487c32f84b66df9cb05634cf21a7274732ea" Sep 07 10:17:54.830979 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.830917 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a559b1027d49d0f7fe64ce7b97b487c32f84b66df9cb05634cf21a7274732ea"} err="failed to get container status \"0a559b1027d49d0f7fe64ce7b97b487c32f84b66df9cb05634cf21a7274732ea\": rpc error: code = NotFound desc = could not find container \"0a559b1027d49d0f7fe64ce7b97b487c32f84b66df9cb05634cf21a7274732ea\": container with ID starting with 0a559b1027d49d0f7fe64ce7b97b487c32f84b66df9cb05634cf21a7274732ea not found: ID does not exist" Sep 07 10:17:54.830979 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.830939 2552 scope.go:117] "RemoveContainer" containerID="ec0e5f50030852c2632a2fed3bcb6da2961ee86bfba47bc1318024cfe34c7b2a" Sep 07 10:17:54.831158 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:17:54.831141 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec0e5f50030852c2632a2fed3bcb6da2961ee86bfba47bc1318024cfe34c7b2a\": container with ID starting with ec0e5f50030852c2632a2fed3bcb6da2961ee86bfba47bc1318024cfe34c7b2a not found: ID does not exist" containerID="ec0e5f50030852c2632a2fed3bcb6da2961ee86bfba47bc1318024cfe34c7b2a" Sep 07 10:17:54.831201 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.831162 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec0e5f50030852c2632a2fed3bcb6da2961ee86bfba47bc1318024cfe34c7b2a"} err="failed to get container status \"ec0e5f50030852c2632a2fed3bcb6da2961ee86bfba47bc1318024cfe34c7b2a\": rpc error: code = NotFound desc = could not find container \"ec0e5f50030852c2632a2fed3bcb6da2961ee86bfba47bc1318024cfe34c7b2a\": container with ID starting with ec0e5f50030852c2632a2fed3bcb6da2961ee86bfba47bc1318024cfe34c7b2a not found: ID does not exist" Sep 07 10:17:54.831201 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.831175 2552 scope.go:117] "RemoveContainer" containerID="a67d8df0db74ebfa90c829e1406639939e329f238757cb063e7952a342dae787" Sep 07 10:17:54.831413 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:17:54.831399 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a67d8df0db74ebfa90c829e1406639939e329f238757cb063e7952a342dae787\": container with ID starting with a67d8df0db74ebfa90c829e1406639939e329f238757cb063e7952a342dae787 not found: ID does not exist" containerID="a67d8df0db74ebfa90c829e1406639939e329f238757cb063e7952a342dae787" Sep 07 10:17:54.831453 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.831416 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a67d8df0db74ebfa90c829e1406639939e329f238757cb063e7952a342dae787"} err="failed to get container status \"a67d8df0db74ebfa90c829e1406639939e329f238757cb063e7952a342dae787\": rpc error: code = NotFound desc = could not find container \"a67d8df0db74ebfa90c829e1406639939e329f238757cb063e7952a342dae787\": container with ID starting with a67d8df0db74ebfa90c829e1406639939e329f238757cb063e7952a342dae787 not found: ID does not exist" Sep 07 10:17:54.831453 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.831427 2552 scope.go:117] "RemoveContainer" containerID="aef981a01e414dcf548b386ae6640d7a34c3ae2cd1bfaddd271377e33b21be1b" Sep 07 10:17:54.831668 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:17:54.831652 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aef981a01e414dcf548b386ae6640d7a34c3ae2cd1bfaddd271377e33b21be1b\": container with ID starting with aef981a01e414dcf548b386ae6640d7a34c3ae2cd1bfaddd271377e33b21be1b not found: ID does not exist" containerID="aef981a01e414dcf548b386ae6640d7a34c3ae2cd1bfaddd271377e33b21be1b" Sep 07 10:17:54.831715 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.831671 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aef981a01e414dcf548b386ae6640d7a34c3ae2cd1bfaddd271377e33b21be1b"} err="failed to get container status \"aef981a01e414dcf548b386ae6640d7a34c3ae2cd1bfaddd271377e33b21be1b\": rpc error: code = NotFound desc = could not find container \"aef981a01e414dcf548b386ae6640d7a34c3ae2cd1bfaddd271377e33b21be1b\": container with ID starting with aef981a01e414dcf548b386ae6640d7a34c3ae2cd1bfaddd271377e33b21be1b not found: ID does not exist" Sep 07 10:17:54.831715 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.831683 2552 scope.go:117] "RemoveContainer" containerID="dcf8ee383b1b6fa7dbf6d1588d4f8858f2babb0208ac2544c9e49d32bbccdf8f" Sep 07 10:17:54.831910 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:17:54.831891 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcf8ee383b1b6fa7dbf6d1588d4f8858f2babb0208ac2544c9e49d32bbccdf8f\": container with ID starting with dcf8ee383b1b6fa7dbf6d1588d4f8858f2babb0208ac2544c9e49d32bbccdf8f not found: ID does not exist" containerID="dcf8ee383b1b6fa7dbf6d1588d4f8858f2babb0208ac2544c9e49d32bbccdf8f" Sep 07 10:17:54.831951 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.831917 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcf8ee383b1b6fa7dbf6d1588d4f8858f2babb0208ac2544c9e49d32bbccdf8f"} err="failed to get container status \"dcf8ee383b1b6fa7dbf6d1588d4f8858f2babb0208ac2544c9e49d32bbccdf8f\": rpc error: code = NotFound desc = could not find container \"dcf8ee383b1b6fa7dbf6d1588d4f8858f2babb0208ac2544c9e49d32bbccdf8f\": container with ID starting with dcf8ee383b1b6fa7dbf6d1588d4f8858f2babb0208ac2544c9e49d32bbccdf8f not found: ID does not exist" Sep 07 10:17:54.831951 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.831935 2552 scope.go:117] "RemoveContainer" containerID="52398106f2aba9a153419676f7f802e401f891f42f9c1cf447a9d851dc5d6081" Sep 07 10:17:54.832149 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.832133 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52398106f2aba9a153419676f7f802e401f891f42f9c1cf447a9d851dc5d6081"} err="failed to get container status \"52398106f2aba9a153419676f7f802e401f891f42f9c1cf447a9d851dc5d6081\": rpc error: code = NotFound desc = could not find container \"52398106f2aba9a153419676f7f802e401f891f42f9c1cf447a9d851dc5d6081\": container with ID starting with 52398106f2aba9a153419676f7f802e401f891f42f9c1cf447a9d851dc5d6081 not found: ID does not exist" Sep 07 10:17:54.832198 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.832148 2552 scope.go:117] "RemoveContainer" containerID="0a559b1027d49d0f7fe64ce7b97b487c32f84b66df9cb05634cf21a7274732ea" Sep 07 10:17:54.832406 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.832381 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a559b1027d49d0f7fe64ce7b97b487c32f84b66df9cb05634cf21a7274732ea"} err="failed to get container status \"0a559b1027d49d0f7fe64ce7b97b487c32f84b66df9cb05634cf21a7274732ea\": rpc error: code = NotFound desc = could not find container \"0a559b1027d49d0f7fe64ce7b97b487c32f84b66df9cb05634cf21a7274732ea\": container with ID starting with 0a559b1027d49d0f7fe64ce7b97b487c32f84b66df9cb05634cf21a7274732ea not found: ID does not exist" Sep 07 10:17:54.832455 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.832408 2552 scope.go:117] "RemoveContainer" containerID="ec0e5f50030852c2632a2fed3bcb6da2961ee86bfba47bc1318024cfe34c7b2a" Sep 07 10:17:54.832636 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.832621 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec0e5f50030852c2632a2fed3bcb6da2961ee86bfba47bc1318024cfe34c7b2a"} err="failed to get container status \"ec0e5f50030852c2632a2fed3bcb6da2961ee86bfba47bc1318024cfe34c7b2a\": rpc error: code = NotFound desc = could not find container \"ec0e5f50030852c2632a2fed3bcb6da2961ee86bfba47bc1318024cfe34c7b2a\": container with ID starting with ec0e5f50030852c2632a2fed3bcb6da2961ee86bfba47bc1318024cfe34c7b2a not found: ID does not exist" Sep 07 10:17:54.832636 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.832636 2552 scope.go:117] "RemoveContainer" containerID="a67d8df0db74ebfa90c829e1406639939e329f238757cb063e7952a342dae787" Sep 07 10:17:54.832854 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.832835 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a67d8df0db74ebfa90c829e1406639939e329f238757cb063e7952a342dae787"} err="failed to get container status \"a67d8df0db74ebfa90c829e1406639939e329f238757cb063e7952a342dae787\": rpc error: code = NotFound desc = could not find container \"a67d8df0db74ebfa90c829e1406639939e329f238757cb063e7952a342dae787\": container with ID starting with a67d8df0db74ebfa90c829e1406639939e329f238757cb063e7952a342dae787 not found: ID does not exist" Sep 07 10:17:54.832895 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.832854 2552 scope.go:117] "RemoveContainer" containerID="aef981a01e414dcf548b386ae6640d7a34c3ae2cd1bfaddd271377e33b21be1b" Sep 07 10:17:54.833077 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.833059 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aef981a01e414dcf548b386ae6640d7a34c3ae2cd1bfaddd271377e33b21be1b"} err="failed to get container status \"aef981a01e414dcf548b386ae6640d7a34c3ae2cd1bfaddd271377e33b21be1b\": rpc error: code = NotFound desc = could not find container \"aef981a01e414dcf548b386ae6640d7a34c3ae2cd1bfaddd271377e33b21be1b\": container with ID starting with aef981a01e414dcf548b386ae6640d7a34c3ae2cd1bfaddd271377e33b21be1b not found: ID does not exist" Sep 07 10:17:54.833147 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.833079 2552 scope.go:117] "RemoveContainer" containerID="dcf8ee383b1b6fa7dbf6d1588d4f8858f2babb0208ac2544c9e49d32bbccdf8f" Sep 07 10:17:54.833307 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:54.833287 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcf8ee383b1b6fa7dbf6d1588d4f8858f2babb0208ac2544c9e49d32bbccdf8f"} err="failed to get container status \"dcf8ee383b1b6fa7dbf6d1588d4f8858f2babb0208ac2544c9e49d32bbccdf8f\": rpc error: code = NotFound desc = could not find container \"dcf8ee383b1b6fa7dbf6d1588d4f8858f2babb0208ac2544c9e49d32bbccdf8f\": container with ID starting with dcf8ee383b1b6fa7dbf6d1588d4f8858f2babb0208ac2544c9e49d32bbccdf8f not found: ID does not exist" Sep 07 10:17:55.520661 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:17:55.520629 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" path="/var/lib/kubelet/pods/68528213-3bd2-4add-a80a-f96edf64be9d/volumes" Sep 07 10:18:03.452508 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:03.452480 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/ovn-acl-logging/0.log" Sep 07 10:18:03.457931 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:03.457911 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/ovn-acl-logging/0.log" Sep 07 10:18:51.159202 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.159164 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct"] Sep 07 10:18:51.159766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.159557 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="feast-apply" Sep 07 10:18:51.159766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.159572 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="feast-apply" Sep 07 10:18:51.159766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.159596 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="online" Sep 07 10:18:51.159766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.159605 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="online" Sep 07 10:18:51.159766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.159621 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="feast-init" Sep 07 10:18:51.159766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.159630 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="feast-init" Sep 07 10:18:51.159766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.159640 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="ui" Sep 07 10:18:51.159766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.159648 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="ui" Sep 07 10:18:51.159766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.159659 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="offline" Sep 07 10:18:51.159766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.159667 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="offline" Sep 07 10:18:51.159766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.159677 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="registry" Sep 07 10:18:51.159766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.159685 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="registry" Sep 07 10:18:51.159766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.159747 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="online" Sep 07 10:18:51.159766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.159755 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="registry" Sep 07 10:18:51.159766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.159764 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="offline" Sep 07 10:18:51.159766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.159772 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="68528213-3bd2-4add-a80a-f96edf64be9d" containerName="ui" Sep 07 10:18:51.162141 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.162120 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:18:51.164339 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.164231 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 07 10:18:51.164460 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.164380 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 07 10:18:51.164460 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.164424 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 07 10:18:51.164577 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.164233 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 07 10:18:51.164577 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.164482 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Sep 07 10:18:51.165009 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.164989 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 07 10:18:51.165118 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.165083 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-kv596\"" Sep 07 10:18:51.165177 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.165152 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 07 10:18:51.173730 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.173700 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct"] Sep 07 10:18:51.354094 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.354056 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/234a1439-4520-4f37-9cd7-e16951c08a69-registry-tls\") pod \"feast-simple-feast-remote-setup-6b8ddf69f6-hvwct\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:18:51.354094 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.354105 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/234a1439-4520-4f37-9cd7-e16951c08a69-offline-tls\") pod \"feast-simple-feast-remote-setup-6b8ddf69f6-hvwct\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:18:51.354326 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.354136 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/234a1439-4520-4f37-9cd7-e16951c08a69-ui-tls\") pod \"feast-simple-feast-remote-setup-6b8ddf69f6-hvwct\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:18:51.354326 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.354192 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zwf7\" (UniqueName: \"kubernetes.io/projected/234a1439-4520-4f37-9cd7-e16951c08a69-kube-api-access-8zwf7\") pod \"feast-simple-feast-remote-setup-6b8ddf69f6-hvwct\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:18:51.354326 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.354230 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/234a1439-4520-4f37-9cd7-e16951c08a69-online-tls\") pod \"feast-simple-feast-remote-setup-6b8ddf69f6-hvwct\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:18:51.354445 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.354321 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/234a1439-4520-4f37-9cd7-e16951c08a69-feast-data\") pod \"feast-simple-feast-remote-setup-6b8ddf69f6-hvwct\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:18:51.454898 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.454804 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/234a1439-4520-4f37-9cd7-e16951c08a69-online-tls\") pod \"feast-simple-feast-remote-setup-6b8ddf69f6-hvwct\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:18:51.454898 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.454865 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/234a1439-4520-4f37-9cd7-e16951c08a69-feast-data\") pod \"feast-simple-feast-remote-setup-6b8ddf69f6-hvwct\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:18:51.455133 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.454936 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/234a1439-4520-4f37-9cd7-e16951c08a69-registry-tls\") pod \"feast-simple-feast-remote-setup-6b8ddf69f6-hvwct\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:18:51.455133 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.454965 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/234a1439-4520-4f37-9cd7-e16951c08a69-offline-tls\") pod \"feast-simple-feast-remote-setup-6b8ddf69f6-hvwct\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:18:51.455133 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.454990 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/234a1439-4520-4f37-9cd7-e16951c08a69-ui-tls\") pod \"feast-simple-feast-remote-setup-6b8ddf69f6-hvwct\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:18:51.455133 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.455030 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zwf7\" (UniqueName: \"kubernetes.io/projected/234a1439-4520-4f37-9cd7-e16951c08a69-kube-api-access-8zwf7\") pod \"feast-simple-feast-remote-setup-6b8ddf69f6-hvwct\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:18:51.455365 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.455305 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/234a1439-4520-4f37-9cd7-e16951c08a69-feast-data\") pod \"feast-simple-feast-remote-setup-6b8ddf69f6-hvwct\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:18:51.455365 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:18:51.455330 2552 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found Sep 07 10:18:51.455463 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:18:51.455392 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/234a1439-4520-4f37-9cd7-e16951c08a69-offline-tls podName:234a1439-4520-4f37-9cd7-e16951c08a69 nodeName:}" failed. No retries permitted until 2026-09-07 10:18:51.95537212 +0000 UTC m=+648.923783903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/234a1439-4520-4f37-9cd7-e16951c08a69-offline-tls") pod "feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" (UID: "234a1439-4520-4f37-9cd7-e16951c08a69") : secret "feast-simple-feast-remote-setup-offline-tls" not found Sep 07 10:18:51.455463 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:18:51.455402 2552 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 07 10:18:51.455463 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:18:51.455455 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/234a1439-4520-4f37-9cd7-e16951c08a69-ui-tls podName:234a1439-4520-4f37-9cd7-e16951c08a69 nodeName:}" failed. No retries permitted until 2026-09-07 10:18:51.955437556 +0000 UTC m=+648.923849344 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/234a1439-4520-4f37-9cd7-e16951c08a69-ui-tls") pod "feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" (UID: "234a1439-4520-4f37-9cd7-e16951c08a69") : secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 07 10:18:51.455714 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.455694 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/234a1439-4520-4f37-9cd7-e16951c08a69-registry-tls\") pod \"feast-simple-feast-remote-setup-6b8ddf69f6-hvwct\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:18:51.457413 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.457390 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/234a1439-4520-4f37-9cd7-e16951c08a69-online-tls\") pod \"feast-simple-feast-remote-setup-6b8ddf69f6-hvwct\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:18:51.462779 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.462749 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zwf7\" (UniqueName: \"kubernetes.io/projected/234a1439-4520-4f37-9cd7-e16951c08a69-kube-api-access-8zwf7\") pod \"feast-simple-feast-remote-setup-6b8ddf69f6-hvwct\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:18:51.959653 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.959618 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/234a1439-4520-4f37-9cd7-e16951c08a69-offline-tls\") pod \"feast-simple-feast-remote-setup-6b8ddf69f6-hvwct\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:18:51.959653 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.959657 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/234a1439-4520-4f37-9cd7-e16951c08a69-ui-tls\") pod \"feast-simple-feast-remote-setup-6b8ddf69f6-hvwct\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:18:51.962102 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.962070 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/234a1439-4520-4f37-9cd7-e16951c08a69-ui-tls\") pod \"feast-simple-feast-remote-setup-6b8ddf69f6-hvwct\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:18:51.962220 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:51.962108 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/234a1439-4520-4f37-9cd7-e16951c08a69-offline-tls\") pod \"feast-simple-feast-remote-setup-6b8ddf69f6-hvwct\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:18:52.073721 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:52.073683 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:18:52.342858 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:52.342826 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct"] Sep 07 10:18:52.346016 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:18:52.345993 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod234a1439_4520_4f37_9cd7_e16951c08a69.slice/crio-74121dd362867c7cfae6eddf4efd243fa2de508482246aa38b89f15a1e1a4f0b WatchSource:0}: Error finding container 74121dd362867c7cfae6eddf4efd243fa2de508482246aa38b89f15a1e1a4f0b: Status 404 returned error can't find the container with id 74121dd362867c7cfae6eddf4efd243fa2de508482246aa38b89f15a1e1a4f0b Sep 07 10:18:52.993774 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:52.993730 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" event={"ID":"234a1439-4520-4f37-9cd7-e16951c08a69","Type":"ContainerStarted","Data":"2e0514135eed1e72f36f3edee38e001943960be1072f9f8808c59612b6767b76"} Sep 07 10:18:52.993954 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:52.993788 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" event={"ID":"234a1439-4520-4f37-9cd7-e16951c08a69","Type":"ContainerStarted","Data":"74121dd362867c7cfae6eddf4efd243fa2de508482246aa38b89f15a1e1a4f0b"} Sep 07 10:18:58.016250 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:58.016204 2552 generic.go:356] "Generic (PLEG): container finished" podID="234a1439-4520-4f37-9cd7-e16951c08a69" containerID="2e0514135eed1e72f36f3edee38e001943960be1072f9f8808c59612b6767b76" exitCode=0 Sep 07 10:18:58.016729 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:58.016291 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" event={"ID":"234a1439-4520-4f37-9cd7-e16951c08a69","Type":"ContainerDied","Data":"2e0514135eed1e72f36f3edee38e001943960be1072f9f8808c59612b6767b76"} Sep 07 10:18:59.021472 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:18:59.021427 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" event={"ID":"234a1439-4520-4f37-9cd7-e16951c08a69","Type":"ContainerStarted","Data":"f57c7574fc732727ba200cb49a5b6e941b3336f58cf5e335c2ffc1b6e63bf166"} Sep 07 10:19:18.091089 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:18.091001 2552 generic.go:356] "Generic (PLEG): container finished" podID="234a1439-4520-4f37-9cd7-e16951c08a69" containerID="f57c7574fc732727ba200cb49a5b6e941b3336f58cf5e335c2ffc1b6e63bf166" exitCode=0 Sep 07 10:19:18.091089 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:18.091067 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" event={"ID":"234a1439-4520-4f37-9cd7-e16951c08a69","Type":"ContainerDied","Data":"f57c7574fc732727ba200cb49a5b6e941b3336f58cf5e335c2ffc1b6e63bf166"} Sep 07 10:19:18.095275 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:18.095235 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Sep 07 10:19:19.099669 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:19.099628 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" event={"ID":"234a1439-4520-4f37-9cd7-e16951c08a69","Type":"ContainerStarted","Data":"1c31d652f55ed88a073cbfa5b0cdf54fdfb8fa6217c83e25d2a6a10faf10a1f1"} Sep 07 10:19:19.100037 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:19.099691 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" event={"ID":"234a1439-4520-4f37-9cd7-e16951c08a69","Type":"ContainerStarted","Data":"7fda11a90fb7d22067c78e15ab4763109821bd23fd33080734c4c15b4af1f71f"} Sep 07 10:19:19.100037 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:19.099711 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" event={"ID":"234a1439-4520-4f37-9cd7-e16951c08a69","Type":"ContainerStarted","Data":"34a1a741ae2e832363fd0f34b5d99255645ad9d5239f364a24879c401ee101c2"} Sep 07 10:19:19.122105 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:19.122039 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" podStartSLOduration=28.12201662 podStartE2EDuration="28.12201662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:18:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:19:19.119965496 +0000 UTC m=+676.088377290" watchObservedRunningTime="2026-09-07 10:19:19.12201662 +0000 UTC m=+676.090428414" Sep 07 10:19:22.074041 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:22.073996 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:19:22.074636 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:22.074607 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:19:22.074636 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:22.074645 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:19:22.074827 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:22.074666 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:19:22.074827 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:22.074685 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:19:22.074827 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:22.074703 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:19:22.075904 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:22.075870 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.17:8443: connect: connection refused" Sep 07 10:19:22.076022 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:22.075899 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.17:8816: connect: connection refused" Sep 07 10:19:22.076022 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:22.075917 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="online" probeResult="failure" output="Get \"https://10.134.0.17:6567/health\": dial tcp 10.134.0.17:6567: connect: connection refused" Sep 07 10:19:25.074523 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:25.074466 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.17:8443: connect: connection refused" Sep 07 10:19:25.075101 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:25.074479 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.17:8816: connect: connection refused" Sep 07 10:19:25.075101 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:25.074479 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="online" probeResult="failure" output="Get \"https://10.134.0.17:6567/health\": dial tcp 10.134.0.17:6567: connect: connection refused" Sep 07 10:19:28.075516 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:28.075476 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:19:28.076007 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:28.075654 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:19:28.076237 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:28.076215 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:19:28.076375 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:28.076360 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:19:28.094499 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:28.094474 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:19:28.195676 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:19:28.195639 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:20:29.355664 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:29.355633 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct"] Sep 07 10:20:29.356200 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:29.355873 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="online" containerID="cri-o://34a1a741ae2e832363fd0f34b5d99255645ad9d5239f364a24879c401ee101c2" gracePeriod=30 Sep 07 10:20:29.356200 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:29.355954 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="offline" containerID="cri-o://7fda11a90fb7d22067c78e15ab4763109821bd23fd33080734c4c15b4af1f71f" gracePeriod=30 Sep 07 10:20:29.356200 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:29.356180 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="ui" containerID="cri-o://1c31d652f55ed88a073cbfa5b0cdf54fdfb8fa6217c83e25d2a6a10faf10a1f1" gracePeriod=30 Sep 07 10:20:29.467224 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:29.467190 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m"] Sep 07 10:20:29.467520 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:29.467471 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="registry" containerID="cri-o://41db28eb567f1f2637cdfe440df0e6365b3102ec6de5f4ffedd51db514f714e2" gracePeriod=30 Sep 07 10:20:29.467628 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:29.467512 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="ui" containerID="cri-o://636aa17b687fcdb6ca91d401172d97b8e62f82544c3ca5487c2ac387ded3a162" gracePeriod=30 Sep 07 10:20:29.467628 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:29.467531 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="online" containerID="cri-o://3bee0674ba73a3abbc4feb0aa8dc87d3cb5f9680681627629cbd4e7114289f2a" gracePeriod=30 Sep 07 10:20:29.467742 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:29.467637 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="offline" containerID="cri-o://3e760dd7a02f493693d0bf3b414dd770440786299f9eb6e5ef4db23f22872e95" gracePeriod=30 Sep 07 10:20:30.373933 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:30.373904 2552 generic.go:356] "Generic (PLEG): container finished" podID="234a1439-4520-4f37-9cd7-e16951c08a69" containerID="1c31d652f55ed88a073cbfa5b0cdf54fdfb8fa6217c83e25d2a6a10faf10a1f1" exitCode=0 Sep 07 10:20:30.373933 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:30.373931 2552 generic.go:356] "Generic (PLEG): container finished" podID="234a1439-4520-4f37-9cd7-e16951c08a69" containerID="34a1a741ae2e832363fd0f34b5d99255645ad9d5239f364a24879c401ee101c2" exitCode=0 Sep 07 10:20:30.374438 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:30.373983 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" event={"ID":"234a1439-4520-4f37-9cd7-e16951c08a69","Type":"ContainerDied","Data":"1c31d652f55ed88a073cbfa5b0cdf54fdfb8fa6217c83e25d2a6a10faf10a1f1"} Sep 07 10:20:30.374438 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:30.374026 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" event={"ID":"234a1439-4520-4f37-9cd7-e16951c08a69","Type":"ContainerDied","Data":"34a1a741ae2e832363fd0f34b5d99255645ad9d5239f364a24879c401ee101c2"} Sep 07 10:20:30.377181 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:30.377158 2552 generic.go:356] "Generic (PLEG): container finished" podID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerID="636aa17b687fcdb6ca91d401172d97b8e62f82544c3ca5487c2ac387ded3a162" exitCode=0 Sep 07 10:20:30.377311 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:30.377227 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" event={"ID":"21fb0061-410a-4e3d-a45a-18de4d4a31b9","Type":"ContainerDied","Data":"636aa17b687fcdb6ca91d401172d97b8e62f82544c3ca5487c2ac387ded3a162"} Sep 07 10:20:31.383587 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:31.383554 2552 generic.go:356] "Generic (PLEG): container finished" podID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerID="3bee0674ba73a3abbc4feb0aa8dc87d3cb5f9680681627629cbd4e7114289f2a" exitCode=0 Sep 07 10:20:31.383949 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:31.383606 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" event={"ID":"21fb0061-410a-4e3d-a45a-18de4d4a31b9","Type":"ContainerDied","Data":"3bee0674ba73a3abbc4feb0aa8dc87d3cb5f9680681627629cbd4e7114289f2a"} Sep 07 10:20:37.759697 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:37.759642 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.16:8443: connect: connection refused" Sep 07 10:20:37.759697 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:37.759678 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.16:6567/health\": dial tcp 10.134.0.16:6567: connect: connection refused" Sep 07 10:20:38.077073 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:38.076974 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.17:8443: connect: connection refused" Sep 07 10:20:38.146713 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:38.146671 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="online" probeResult="failure" output="Get \"https://10.134.0.17:6567/health\": dial tcp 10.134.0.17:6567: connect: connection refused" Sep 07 10:20:47.760106 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:47.760050 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.16:8443: connect: connection refused" Sep 07 10:20:47.760106 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:47.760087 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.16:6567/health\": dial tcp 10.134.0.16:6567: connect: connection refused" Sep 07 10:20:48.076827 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:48.076735 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.17:8443: connect: connection refused" Sep 07 10:20:48.146407 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:48.146361 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="online" probeResult="failure" output="Get \"https://10.134.0.17:6567/health\": dial tcp 10.134.0.17:6567: connect: connection refused" Sep 07 10:20:57.759633 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:57.759584 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.16:6567/health\": dial tcp 10.134.0.16:6567: connect: connection refused" Sep 07 10:20:57.760096 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:57.759648 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.16:8443: connect: connection refused" Sep 07 10:20:57.760096 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:57.759724 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:20:57.760096 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:57.759791 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:20:58.076814 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:58.076714 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.17:8443: connect: connection refused" Sep 07 10:20:58.076982 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:58.076869 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:20:58.145954 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:58.145907 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="online" probeResult="failure" output="Get \"https://10.134.0.17:6567/health\": dial tcp 10.134.0.17:6567: connect: connection refused" Sep 07 10:20:58.146133 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:58.146046 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:20:59.491308 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:59.491277 2552 generic.go:356] "Generic (PLEG): container finished" podID="234a1439-4520-4f37-9cd7-e16951c08a69" containerID="7fda11a90fb7d22067c78e15ab4763109821bd23fd33080734c4c15b4af1f71f" exitCode=137 Sep 07 10:20:59.491638 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:59.491321 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" event={"ID":"234a1439-4520-4f37-9cd7-e16951c08a69","Type":"ContainerDied","Data":"7fda11a90fb7d22067c78e15ab4763109821bd23fd33080734c4c15b4af1f71f"} Sep 07 10:20:59.577725 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:59.577696 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:20:59.702795 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:59.702695 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/234a1439-4520-4f37-9cd7-e16951c08a69-registry-tls\") pod \"234a1439-4520-4f37-9cd7-e16951c08a69\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " Sep 07 10:20:59.702795 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:59.702750 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/234a1439-4520-4f37-9cd7-e16951c08a69-offline-tls\") pod \"234a1439-4520-4f37-9cd7-e16951c08a69\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " Sep 07 10:20:59.702795 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:59.702770 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zwf7\" (UniqueName: \"kubernetes.io/projected/234a1439-4520-4f37-9cd7-e16951c08a69-kube-api-access-8zwf7\") pod \"234a1439-4520-4f37-9cd7-e16951c08a69\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " Sep 07 10:20:59.703011 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:59.702836 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/234a1439-4520-4f37-9cd7-e16951c08a69-online-tls\") pod \"234a1439-4520-4f37-9cd7-e16951c08a69\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " Sep 07 10:20:59.703011 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:59.702853 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/234a1439-4520-4f37-9cd7-e16951c08a69-ui-tls\") pod \"234a1439-4520-4f37-9cd7-e16951c08a69\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " Sep 07 10:20:59.703011 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:59.702883 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/234a1439-4520-4f37-9cd7-e16951c08a69-feast-data\") pod \"234a1439-4520-4f37-9cd7-e16951c08a69\" (UID: \"234a1439-4520-4f37-9cd7-e16951c08a69\") " Sep 07 10:20:59.703207 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:59.703169 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/234a1439-4520-4f37-9cd7-e16951c08a69-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "234a1439-4520-4f37-9cd7-e16951c08a69" (UID: "234a1439-4520-4f37-9cd7-e16951c08a69"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 10:20:59.703592 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:59.703553 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/234a1439-4520-4f37-9cd7-e16951c08a69-feast-data" (OuterVolumeSpecName: "feast-data") pod "234a1439-4520-4f37-9cd7-e16951c08a69" (UID: "234a1439-4520-4f37-9cd7-e16951c08a69"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 10:20:59.705139 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:59.705113 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/234a1439-4520-4f37-9cd7-e16951c08a69-online-tls" (OuterVolumeSpecName: "online-tls") pod "234a1439-4520-4f37-9cd7-e16951c08a69" (UID: "234a1439-4520-4f37-9cd7-e16951c08a69"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:20:59.705304 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:59.705154 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/234a1439-4520-4f37-9cd7-e16951c08a69-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "234a1439-4520-4f37-9cd7-e16951c08a69" (UID: "234a1439-4520-4f37-9cd7-e16951c08a69"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:20:59.705368 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:59.705324 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/234a1439-4520-4f37-9cd7-e16951c08a69-kube-api-access-8zwf7" (OuterVolumeSpecName: "kube-api-access-8zwf7") pod "234a1439-4520-4f37-9cd7-e16951c08a69" (UID: "234a1439-4520-4f37-9cd7-e16951c08a69"). InnerVolumeSpecName "kube-api-access-8zwf7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:20:59.705408 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:59.705359 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/234a1439-4520-4f37-9cd7-e16951c08a69-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "234a1439-4520-4f37-9cd7-e16951c08a69" (UID: "234a1439-4520-4f37-9cd7-e16951c08a69"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:20:59.803347 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:59.803313 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zwf7\" (UniqueName: \"kubernetes.io/projected/234a1439-4520-4f37-9cd7-e16951c08a69-kube-api-access-8zwf7\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:20:59.803347 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:59.803344 2552 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/234a1439-4520-4f37-9cd7-e16951c08a69-online-tls\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:20:59.803347 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:59.803354 2552 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/234a1439-4520-4f37-9cd7-e16951c08a69-ui-tls\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:20:59.803560 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:59.803363 2552 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/234a1439-4520-4f37-9cd7-e16951c08a69-feast-data\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:20:59.803560 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:59.803370 2552 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/234a1439-4520-4f37-9cd7-e16951c08a69-registry-tls\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:20:59.803560 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:20:59.803377 2552 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/234a1439-4520-4f37-9cd7-e16951c08a69-offline-tls\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:21:00.099587 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.099560 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:21:00.206356 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.206317 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-registry-tls\") pod \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " Sep 07 10:21:00.206541 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.206371 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-ui-tls\") pod \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " Sep 07 10:21:00.206541 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.206406 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnf9z\" (UniqueName: \"kubernetes.io/projected/21fb0061-410a-4e3d-a45a-18de4d4a31b9-kube-api-access-rnf9z\") pod \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " Sep 07 10:21:00.206541 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.206426 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-online-tls\") pod \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " Sep 07 10:21:00.206541 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.206451 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-offline-tls\") pod \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " Sep 07 10:21:00.206541 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.206478 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/21fb0061-410a-4e3d-a45a-18de4d4a31b9-feast-data\") pod \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\" (UID: \"21fb0061-410a-4e3d-a45a-18de4d4a31b9\") " Sep 07 10:21:00.207137 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.207104 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21fb0061-410a-4e3d-a45a-18de4d4a31b9-feast-data" (OuterVolumeSpecName: "feast-data") pod "21fb0061-410a-4e3d-a45a-18de4d4a31b9" (UID: "21fb0061-410a-4e3d-a45a-18de4d4a31b9"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 10:21:00.208469 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.208445 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "21fb0061-410a-4e3d-a45a-18de4d4a31b9" (UID: "21fb0061-410a-4e3d-a45a-18de4d4a31b9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:21:00.209072 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.209047 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-online-tls" (OuterVolumeSpecName: "online-tls") pod "21fb0061-410a-4e3d-a45a-18de4d4a31b9" (UID: "21fb0061-410a-4e3d-a45a-18de4d4a31b9"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:21:00.209072 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.209054 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21fb0061-410a-4e3d-a45a-18de4d4a31b9-kube-api-access-rnf9z" (OuterVolumeSpecName: "kube-api-access-rnf9z") pod "21fb0061-410a-4e3d-a45a-18de4d4a31b9" (UID: "21fb0061-410a-4e3d-a45a-18de4d4a31b9"). InnerVolumeSpecName "kube-api-access-rnf9z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:21:00.209072 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.209064 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "21fb0061-410a-4e3d-a45a-18de4d4a31b9" (UID: "21fb0061-410a-4e3d-a45a-18de4d4a31b9"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:21:00.209220 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.209085 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "21fb0061-410a-4e3d-a45a-18de4d4a31b9" (UID: "21fb0061-410a-4e3d-a45a-18de4d4a31b9"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:21:00.307306 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.307216 2552 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/21fb0061-410a-4e3d-a45a-18de4d4a31b9-feast-data\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:21:00.307306 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.307271 2552 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-registry-tls\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:21:00.307306 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.307280 2552 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-ui-tls\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:21:00.307306 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.307288 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnf9z\" (UniqueName: \"kubernetes.io/projected/21fb0061-410a-4e3d-a45a-18de4d4a31b9-kube-api-access-rnf9z\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:21:00.307306 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.307298 2552 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-online-tls\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:21:00.307306 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.307306 2552 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/21fb0061-410a-4e3d-a45a-18de4d4a31b9-offline-tls\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:21:00.497271 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.497218 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" event={"ID":"234a1439-4520-4f37-9cd7-e16951c08a69","Type":"ContainerDied","Data":"74121dd362867c7cfae6eddf4efd243fa2de508482246aa38b89f15a1e1a4f0b"} Sep 07 10:21:00.497271 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.497270 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct" Sep 07 10:21:00.497921 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.497294 2552 scope.go:117] "RemoveContainer" containerID="1c31d652f55ed88a073cbfa5b0cdf54fdfb8fa6217c83e25d2a6a10faf10a1f1" Sep 07 10:21:00.500553 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.500527 2552 generic.go:356] "Generic (PLEG): container finished" podID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerID="3e760dd7a02f493693d0bf3b414dd770440786299f9eb6e5ef4db23f22872e95" exitCode=137 Sep 07 10:21:00.500553 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.500548 2552 generic.go:356] "Generic (PLEG): container finished" podID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerID="41db28eb567f1f2637cdfe440df0e6365b3102ec6de5f4ffedd51db514f714e2" exitCode=137 Sep 07 10:21:00.500738 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.500597 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" event={"ID":"21fb0061-410a-4e3d-a45a-18de4d4a31b9","Type":"ContainerDied","Data":"3e760dd7a02f493693d0bf3b414dd770440786299f9eb6e5ef4db23f22872e95"} Sep 07 10:21:00.500738 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.500609 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" Sep 07 10:21:00.500738 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.500635 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" event={"ID":"21fb0061-410a-4e3d-a45a-18de4d4a31b9","Type":"ContainerDied","Data":"41db28eb567f1f2637cdfe440df0e6365b3102ec6de5f4ffedd51db514f714e2"} Sep 07 10:21:00.500738 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.500654 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m" event={"ID":"21fb0061-410a-4e3d-a45a-18de4d4a31b9","Type":"ContainerDied","Data":"859b39e0899b91b9550cda29210573e1bc06d66c6317512eb8dd12f5e90a15ad"} Sep 07 10:21:00.507064 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.507044 2552 scope.go:117] "RemoveContainer" containerID="7fda11a90fb7d22067c78e15ab4763109821bd23fd33080734c4c15b4af1f71f" Sep 07 10:21:00.515551 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.515526 2552 scope.go:117] "RemoveContainer" containerID="34a1a741ae2e832363fd0f34b5d99255645ad9d5239f364a24879c401ee101c2" Sep 07 10:21:00.520099 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.520075 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct"] Sep 07 10:21:00.523473 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.523446 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6b8ddf69f6-hvwct"] Sep 07 10:21:00.523920 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.523907 2552 scope.go:117] "RemoveContainer" containerID="f57c7574fc732727ba200cb49a5b6e941b3336f58cf5e335c2ffc1b6e63bf166" Sep 07 10:21:00.530968 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.530937 2552 scope.go:117] "RemoveContainer" containerID="2e0514135eed1e72f36f3edee38e001943960be1072f9f8808c59612b6767b76" Sep 07 10:21:00.533657 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.533638 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m"] Sep 07 10:21:00.537371 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.537351 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7d48777b49-2sb8m"] Sep 07 10:21:00.538605 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.538590 2552 scope.go:117] "RemoveContainer" containerID="636aa17b687fcdb6ca91d401172d97b8e62f82544c3ca5487c2ac387ded3a162" Sep 07 10:21:00.545214 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.545194 2552 scope.go:117] "RemoveContainer" containerID="3e760dd7a02f493693d0bf3b414dd770440786299f9eb6e5ef4db23f22872e95" Sep 07 10:21:00.552174 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.552156 2552 scope.go:117] "RemoveContainer" containerID="3bee0674ba73a3abbc4feb0aa8dc87d3cb5f9680681627629cbd4e7114289f2a" Sep 07 10:21:00.559168 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.559148 2552 scope.go:117] "RemoveContainer" containerID="41db28eb567f1f2637cdfe440df0e6365b3102ec6de5f4ffedd51db514f714e2" Sep 07 10:21:00.566142 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.566123 2552 scope.go:117] "RemoveContainer" containerID="c4a69362746a16aa91d0142c3dfd76c98c00b1a7bf5c7c38376b70b4ba4e896c" Sep 07 10:21:00.572532 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.572515 2552 scope.go:117] "RemoveContainer" containerID="f3d7b3374dac14af3193cce0ab19f22c9da4cc2914ef873585728bfe221d40f0" Sep 07 10:21:00.579334 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.579315 2552 scope.go:117] "RemoveContainer" containerID="636aa17b687fcdb6ca91d401172d97b8e62f82544c3ca5487c2ac387ded3a162" Sep 07 10:21:00.579601 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:21:00.579578 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"636aa17b687fcdb6ca91d401172d97b8e62f82544c3ca5487c2ac387ded3a162\": container with ID starting with 636aa17b687fcdb6ca91d401172d97b8e62f82544c3ca5487c2ac387ded3a162 not found: ID does not exist" containerID="636aa17b687fcdb6ca91d401172d97b8e62f82544c3ca5487c2ac387ded3a162" Sep 07 10:21:00.579676 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.579615 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"636aa17b687fcdb6ca91d401172d97b8e62f82544c3ca5487c2ac387ded3a162"} err="failed to get container status \"636aa17b687fcdb6ca91d401172d97b8e62f82544c3ca5487c2ac387ded3a162\": rpc error: code = NotFound desc = could not find container \"636aa17b687fcdb6ca91d401172d97b8e62f82544c3ca5487c2ac387ded3a162\": container with ID starting with 636aa17b687fcdb6ca91d401172d97b8e62f82544c3ca5487c2ac387ded3a162 not found: ID does not exist" Sep 07 10:21:00.579676 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.579642 2552 scope.go:117] "RemoveContainer" containerID="3e760dd7a02f493693d0bf3b414dd770440786299f9eb6e5ef4db23f22872e95" Sep 07 10:21:00.579902 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:21:00.579883 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e760dd7a02f493693d0bf3b414dd770440786299f9eb6e5ef4db23f22872e95\": container with ID starting with 3e760dd7a02f493693d0bf3b414dd770440786299f9eb6e5ef4db23f22872e95 not found: ID does not exist" containerID="3e760dd7a02f493693d0bf3b414dd770440786299f9eb6e5ef4db23f22872e95" Sep 07 10:21:00.579940 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.579908 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e760dd7a02f493693d0bf3b414dd770440786299f9eb6e5ef4db23f22872e95"} err="failed to get container status \"3e760dd7a02f493693d0bf3b414dd770440786299f9eb6e5ef4db23f22872e95\": rpc error: code = NotFound desc = could not find container \"3e760dd7a02f493693d0bf3b414dd770440786299f9eb6e5ef4db23f22872e95\": container with ID starting with 3e760dd7a02f493693d0bf3b414dd770440786299f9eb6e5ef4db23f22872e95 not found: ID does not exist" Sep 07 10:21:00.579940 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.579925 2552 scope.go:117] "RemoveContainer" containerID="3bee0674ba73a3abbc4feb0aa8dc87d3cb5f9680681627629cbd4e7114289f2a" Sep 07 10:21:00.580187 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:21:00.580169 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bee0674ba73a3abbc4feb0aa8dc87d3cb5f9680681627629cbd4e7114289f2a\": container with ID starting with 3bee0674ba73a3abbc4feb0aa8dc87d3cb5f9680681627629cbd4e7114289f2a not found: ID does not exist" containerID="3bee0674ba73a3abbc4feb0aa8dc87d3cb5f9680681627629cbd4e7114289f2a" Sep 07 10:21:00.580229 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.580194 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bee0674ba73a3abbc4feb0aa8dc87d3cb5f9680681627629cbd4e7114289f2a"} err="failed to get container status \"3bee0674ba73a3abbc4feb0aa8dc87d3cb5f9680681627629cbd4e7114289f2a\": rpc error: code = NotFound desc = could not find container \"3bee0674ba73a3abbc4feb0aa8dc87d3cb5f9680681627629cbd4e7114289f2a\": container with ID starting with 3bee0674ba73a3abbc4feb0aa8dc87d3cb5f9680681627629cbd4e7114289f2a not found: ID does not exist" Sep 07 10:21:00.580229 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.580215 2552 scope.go:117] "RemoveContainer" containerID="41db28eb567f1f2637cdfe440df0e6365b3102ec6de5f4ffedd51db514f714e2" Sep 07 10:21:00.580565 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:21:00.580548 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41db28eb567f1f2637cdfe440df0e6365b3102ec6de5f4ffedd51db514f714e2\": container with ID starting with 41db28eb567f1f2637cdfe440df0e6365b3102ec6de5f4ffedd51db514f714e2 not found: ID does not exist" containerID="41db28eb567f1f2637cdfe440df0e6365b3102ec6de5f4ffedd51db514f714e2" Sep 07 10:21:00.580605 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.580570 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41db28eb567f1f2637cdfe440df0e6365b3102ec6de5f4ffedd51db514f714e2"} err="failed to get container status \"41db28eb567f1f2637cdfe440df0e6365b3102ec6de5f4ffedd51db514f714e2\": rpc error: code = NotFound desc = could not find container \"41db28eb567f1f2637cdfe440df0e6365b3102ec6de5f4ffedd51db514f714e2\": container with ID starting with 41db28eb567f1f2637cdfe440df0e6365b3102ec6de5f4ffedd51db514f714e2 not found: ID does not exist" Sep 07 10:21:00.580605 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.580583 2552 scope.go:117] "RemoveContainer" containerID="c4a69362746a16aa91d0142c3dfd76c98c00b1a7bf5c7c38376b70b4ba4e896c" Sep 07 10:21:00.580830 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:21:00.580811 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4a69362746a16aa91d0142c3dfd76c98c00b1a7bf5c7c38376b70b4ba4e896c\": container with ID starting with c4a69362746a16aa91d0142c3dfd76c98c00b1a7bf5c7c38376b70b4ba4e896c not found: ID does not exist" containerID="c4a69362746a16aa91d0142c3dfd76c98c00b1a7bf5c7c38376b70b4ba4e896c" Sep 07 10:21:00.580896 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.580839 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4a69362746a16aa91d0142c3dfd76c98c00b1a7bf5c7c38376b70b4ba4e896c"} err="failed to get container status \"c4a69362746a16aa91d0142c3dfd76c98c00b1a7bf5c7c38376b70b4ba4e896c\": rpc error: code = NotFound desc = could not find container \"c4a69362746a16aa91d0142c3dfd76c98c00b1a7bf5c7c38376b70b4ba4e896c\": container with ID starting with c4a69362746a16aa91d0142c3dfd76c98c00b1a7bf5c7c38376b70b4ba4e896c not found: ID does not exist" Sep 07 10:21:00.580896 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.580859 2552 scope.go:117] "RemoveContainer" containerID="f3d7b3374dac14af3193cce0ab19f22c9da4cc2914ef873585728bfe221d40f0" Sep 07 10:21:00.581108 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:21:00.581089 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3d7b3374dac14af3193cce0ab19f22c9da4cc2914ef873585728bfe221d40f0\": container with ID starting with f3d7b3374dac14af3193cce0ab19f22c9da4cc2914ef873585728bfe221d40f0 not found: ID does not exist" containerID="f3d7b3374dac14af3193cce0ab19f22c9da4cc2914ef873585728bfe221d40f0" Sep 07 10:21:00.581154 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.581119 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3d7b3374dac14af3193cce0ab19f22c9da4cc2914ef873585728bfe221d40f0"} err="failed to get container status \"f3d7b3374dac14af3193cce0ab19f22c9da4cc2914ef873585728bfe221d40f0\": rpc error: code = NotFound desc = could not find container \"f3d7b3374dac14af3193cce0ab19f22c9da4cc2914ef873585728bfe221d40f0\": container with ID starting with f3d7b3374dac14af3193cce0ab19f22c9da4cc2914ef873585728bfe221d40f0 not found: ID does not exist" Sep 07 10:21:00.581154 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.581138 2552 scope.go:117] "RemoveContainer" containerID="636aa17b687fcdb6ca91d401172d97b8e62f82544c3ca5487c2ac387ded3a162" Sep 07 10:21:00.581446 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.581426 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"636aa17b687fcdb6ca91d401172d97b8e62f82544c3ca5487c2ac387ded3a162"} err="failed to get container status \"636aa17b687fcdb6ca91d401172d97b8e62f82544c3ca5487c2ac387ded3a162\": rpc error: code = NotFound desc = could not find container \"636aa17b687fcdb6ca91d401172d97b8e62f82544c3ca5487c2ac387ded3a162\": container with ID starting with 636aa17b687fcdb6ca91d401172d97b8e62f82544c3ca5487c2ac387ded3a162 not found: ID does not exist" Sep 07 10:21:00.581526 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.581446 2552 scope.go:117] "RemoveContainer" containerID="3e760dd7a02f493693d0bf3b414dd770440786299f9eb6e5ef4db23f22872e95" Sep 07 10:21:00.581689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.581664 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e760dd7a02f493693d0bf3b414dd770440786299f9eb6e5ef4db23f22872e95"} err="failed to get container status \"3e760dd7a02f493693d0bf3b414dd770440786299f9eb6e5ef4db23f22872e95\": rpc error: code = NotFound desc = could not find container \"3e760dd7a02f493693d0bf3b414dd770440786299f9eb6e5ef4db23f22872e95\": container with ID starting with 3e760dd7a02f493693d0bf3b414dd770440786299f9eb6e5ef4db23f22872e95 not found: ID does not exist" Sep 07 10:21:00.581755 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.581690 2552 scope.go:117] "RemoveContainer" containerID="3bee0674ba73a3abbc4feb0aa8dc87d3cb5f9680681627629cbd4e7114289f2a" Sep 07 10:21:00.581904 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.581887 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bee0674ba73a3abbc4feb0aa8dc87d3cb5f9680681627629cbd4e7114289f2a"} err="failed to get container status \"3bee0674ba73a3abbc4feb0aa8dc87d3cb5f9680681627629cbd4e7114289f2a\": rpc error: code = NotFound desc = could not find container \"3bee0674ba73a3abbc4feb0aa8dc87d3cb5f9680681627629cbd4e7114289f2a\": container with ID starting with 3bee0674ba73a3abbc4feb0aa8dc87d3cb5f9680681627629cbd4e7114289f2a not found: ID does not exist" Sep 07 10:21:00.581982 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.581905 2552 scope.go:117] "RemoveContainer" containerID="41db28eb567f1f2637cdfe440df0e6365b3102ec6de5f4ffedd51db514f714e2" Sep 07 10:21:00.582105 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.582087 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41db28eb567f1f2637cdfe440df0e6365b3102ec6de5f4ffedd51db514f714e2"} err="failed to get container status \"41db28eb567f1f2637cdfe440df0e6365b3102ec6de5f4ffedd51db514f714e2\": rpc error: code = NotFound desc = could not find container \"41db28eb567f1f2637cdfe440df0e6365b3102ec6de5f4ffedd51db514f714e2\": container with ID starting with 41db28eb567f1f2637cdfe440df0e6365b3102ec6de5f4ffedd51db514f714e2 not found: ID does not exist" Sep 07 10:21:00.582150 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.582106 2552 scope.go:117] "RemoveContainer" containerID="c4a69362746a16aa91d0142c3dfd76c98c00b1a7bf5c7c38376b70b4ba4e896c" Sep 07 10:21:00.582352 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.582335 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4a69362746a16aa91d0142c3dfd76c98c00b1a7bf5c7c38376b70b4ba4e896c"} err="failed to get container status \"c4a69362746a16aa91d0142c3dfd76c98c00b1a7bf5c7c38376b70b4ba4e896c\": rpc error: code = NotFound desc = could not find container \"c4a69362746a16aa91d0142c3dfd76c98c00b1a7bf5c7c38376b70b4ba4e896c\": container with ID starting with c4a69362746a16aa91d0142c3dfd76c98c00b1a7bf5c7c38376b70b4ba4e896c not found: ID does not exist" Sep 07 10:21:00.582423 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.582353 2552 scope.go:117] "RemoveContainer" containerID="f3d7b3374dac14af3193cce0ab19f22c9da4cc2914ef873585728bfe221d40f0" Sep 07 10:21:00.582582 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:00.582562 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3d7b3374dac14af3193cce0ab19f22c9da4cc2914ef873585728bfe221d40f0"} err="failed to get container status \"f3d7b3374dac14af3193cce0ab19f22c9da4cc2914ef873585728bfe221d40f0\": rpc error: code = NotFound desc = could not find container \"f3d7b3374dac14af3193cce0ab19f22c9da4cc2914ef873585728bfe221d40f0\": container with ID starting with f3d7b3374dac14af3193cce0ab19f22c9da4cc2914ef873585728bfe221d40f0 not found: ID does not exist" Sep 07 10:21:01.521286 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:01.521233 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" path="/var/lib/kubelet/pods/21fb0061-410a-4e3d-a45a-18de4d4a31b9/volumes" Sep 07 10:21:01.521863 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:01.521849 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" path="/var/lib/kubelet/pods/234a1439-4520-4f37-9cd7-e16951c08a69/volumes" Sep 07 10:21:12.185497 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185462 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-72lf4"] Sep 07 10:21:12.185846 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185696 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="offline" Sep 07 10:21:12.185846 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185710 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="offline" Sep 07 10:21:12.185846 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185721 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="feast-apply" Sep 07 10:21:12.185846 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185730 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="feast-apply" Sep 07 10:21:12.185846 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185741 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="offline" Sep 07 10:21:12.185846 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185747 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="offline" Sep 07 10:21:12.185846 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185756 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="online" Sep 07 10:21:12.185846 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185761 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="online" Sep 07 10:21:12.185846 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185767 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="feast-init" Sep 07 10:21:12.185846 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185772 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="feast-init" Sep 07 10:21:12.185846 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185778 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="ui" Sep 07 10:21:12.185846 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185784 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="ui" Sep 07 10:21:12.185846 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185788 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="ui" Sep 07 10:21:12.185846 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185793 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="ui" Sep 07 10:21:12.185846 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185801 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="online" Sep 07 10:21:12.185846 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185807 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="online" Sep 07 10:21:12.185846 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185814 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="feast-init" Sep 07 10:21:12.185846 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185818 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="feast-init" Sep 07 10:21:12.185846 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185825 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="registry" Sep 07 10:21:12.185846 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185830 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="registry" Sep 07 10:21:12.185846 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185839 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="feast-apply" Sep 07 10:21:12.185846 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185843 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="feast-apply" Sep 07 10:21:12.186522 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185885 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="online" Sep 07 10:21:12.186522 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185893 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="ui" Sep 07 10:21:12.186522 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185899 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="offline" Sep 07 10:21:12.186522 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185904 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="registry" Sep 07 10:21:12.186522 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185910 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="ui" Sep 07 10:21:12.186522 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185916 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="234a1439-4520-4f37-9cd7-e16951c08a69" containerName="online" Sep 07 10:21:12.186522 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.185921 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="21fb0061-410a-4e3d-a45a-18de4d4a31b9" containerName="offline" Sep 07 10:21:12.190817 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.190797 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-72lf4" Sep 07 10:21:12.192921 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.192898 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 07 10:21:12.193055 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.192963 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 07 10:21:12.193055 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.193034 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Sep 07 10:21:12.194294 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.194271 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-mtztw\"" Sep 07 10:21:12.195236 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.195217 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-72lf4"] Sep 07 10:21:12.219440 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.219403 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-69bbd798b6-tt4zf"] Sep 07 10:21:12.222472 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.222449 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-tt4zf" Sep 07 10:21:12.229618 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.229597 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-tt4zf"] Sep 07 10:21:12.289736 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.289706 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv2z6\" (UniqueName: \"kubernetes.io/projected/31958728-d03a-4b6f-b1ab-4709e7c01d44-kube-api-access-bv2z6\") pod \"postgres-864846d4f5-72lf4\" (UID: \"31958728-d03a-4b6f-b1ab-4709e7c01d44\") " pod="test-ns-feast/postgres-864846d4f5-72lf4" Sep 07 10:21:12.289919 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.289759 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whzbr\" (UniqueName: \"kubernetes.io/projected/2b2fe0c9-3589-41c7-944c-ff6cfb62e841-kube-api-access-whzbr\") pod \"redis-69bbd798b6-tt4zf\" (UID: \"2b2fe0c9-3589-41c7-944c-ff6cfb62e841\") " pod="test-ns-feast/redis-69bbd798b6-tt4zf" Sep 07 10:21:12.289919 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.289817 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/31958728-d03a-4b6f-b1ab-4709e7c01d44-postgresdata\") pod \"postgres-864846d4f5-72lf4\" (UID: \"31958728-d03a-4b6f-b1ab-4709e7c01d44\") " pod="test-ns-feast/postgres-864846d4f5-72lf4" Sep 07 10:21:12.391127 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.391097 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv2z6\" (UniqueName: \"kubernetes.io/projected/31958728-d03a-4b6f-b1ab-4709e7c01d44-kube-api-access-bv2z6\") pod \"postgres-864846d4f5-72lf4\" (UID: \"31958728-d03a-4b6f-b1ab-4709e7c01d44\") " pod="test-ns-feast/postgres-864846d4f5-72lf4" Sep 07 10:21:12.391307 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.391155 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whzbr\" (UniqueName: \"kubernetes.io/projected/2b2fe0c9-3589-41c7-944c-ff6cfb62e841-kube-api-access-whzbr\") pod \"redis-69bbd798b6-tt4zf\" (UID: \"2b2fe0c9-3589-41c7-944c-ff6cfb62e841\") " pod="test-ns-feast/redis-69bbd798b6-tt4zf" Sep 07 10:21:12.391307 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.391192 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/31958728-d03a-4b6f-b1ab-4709e7c01d44-postgresdata\") pod \"postgres-864846d4f5-72lf4\" (UID: \"31958728-d03a-4b6f-b1ab-4709e7c01d44\") " pod="test-ns-feast/postgres-864846d4f5-72lf4" Sep 07 10:21:12.391524 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.391507 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/31958728-d03a-4b6f-b1ab-4709e7c01d44-postgresdata\") pod \"postgres-864846d4f5-72lf4\" (UID: \"31958728-d03a-4b6f-b1ab-4709e7c01d44\") " pod="test-ns-feast/postgres-864846d4f5-72lf4" Sep 07 10:21:12.399232 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.399207 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv2z6\" (UniqueName: \"kubernetes.io/projected/31958728-d03a-4b6f-b1ab-4709e7c01d44-kube-api-access-bv2z6\") pod \"postgres-864846d4f5-72lf4\" (UID: \"31958728-d03a-4b6f-b1ab-4709e7c01d44\") " pod="test-ns-feast/postgres-864846d4f5-72lf4" Sep 07 10:21:12.399358 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.399207 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whzbr\" (UniqueName: \"kubernetes.io/projected/2b2fe0c9-3589-41c7-944c-ff6cfb62e841-kube-api-access-whzbr\") pod \"redis-69bbd798b6-tt4zf\" (UID: \"2b2fe0c9-3589-41c7-944c-ff6cfb62e841\") " pod="test-ns-feast/redis-69bbd798b6-tt4zf" Sep 07 10:21:12.501178 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.501128 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-72lf4" Sep 07 10:21:12.531231 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.531199 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-tt4zf" Sep 07 10:21:12.769660 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.769582 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-72lf4"] Sep 07 10:21:12.772297 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:21:12.772269 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31958728_d03a_4b6f_b1ab_4709e7c01d44.slice/crio-a8ab347077633c9ab760d3be49aa0726431614000aa3e19f1988e827a3c520b5 WatchSource:0}: Error finding container a8ab347077633c9ab760d3be49aa0726431614000aa3e19f1988e827a3c520b5: Status 404 returned error can't find the container with id a8ab347077633c9ab760d3be49aa0726431614000aa3e19f1988e827a3c520b5 Sep 07 10:21:12.774050 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.774036 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 07 10:21:12.802462 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:12.802430 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-tt4zf"] Sep 07 10:21:12.804924 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:21:12.804898 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b2fe0c9_3589_41c7_944c_ff6cfb62e841.slice/crio-dde954c5db1723b50a5c45351c87ff95e65bebea67864d71184e9768dfaade23 WatchSource:0}: Error finding container dde954c5db1723b50a5c45351c87ff95e65bebea67864d71184e9768dfaade23: Status 404 returned error can't find the container with id dde954c5db1723b50a5c45351c87ff95e65bebea67864d71184e9768dfaade23 Sep 07 10:21:13.552528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:13.552484 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-tt4zf" event={"ID":"2b2fe0c9-3589-41c7-944c-ff6cfb62e841","Type":"ContainerStarted","Data":"dde954c5db1723b50a5c45351c87ff95e65bebea67864d71184e9768dfaade23"} Sep 07 10:21:13.554231 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:13.554196 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-72lf4" event={"ID":"31958728-d03a-4b6f-b1ab-4709e7c01d44","Type":"ContainerStarted","Data":"a8ab347077633c9ab760d3be49aa0726431614000aa3e19f1988e827a3c520b5"} Sep 07 10:21:14.560162 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:14.560121 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-tt4zf" event={"ID":"2b2fe0c9-3589-41c7-944c-ff6cfb62e841","Type":"ContainerStarted","Data":"debfa4c876c41b93f971b12469af508697089605aa2595fbbd2d65c39ea6cd1d"} Sep 07 10:21:14.574976 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:14.574918 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/redis-69bbd798b6-tt4zf" podStartSLOduration=1.032568377 podStartE2EDuration="2.574897626s" totalImagesPullingTime="1.542329249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:21:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:21:14.574034934 +0000 UTC m=+791.542446729" watchObservedRunningTime="2026-09-07 10:21:14.574897626 +0000 UTC m=+791.543309420" Sep 07 10:21:18.576284 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:18.576232 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-72lf4" event={"ID":"31958728-d03a-4b6f-b1ab-4709e7c01d44","Type":"ContainerStarted","Data":"aa8e5d6f6b6fc91ab03be346e3fbd69e9897e4f2ecd3f18e1ff48450f1f8601b"} Sep 07 10:21:18.590270 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:18.590201 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-72lf4" podStartSLOduration=1.832909648 podStartE2EDuration="6.590186367s" totalImagesPullingTime="4.757276719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:21:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:21:18.589100658 +0000 UTC m=+795.557512453" watchObservedRunningTime="2026-09-07 10:21:18.590186367 +0000 UTC m=+795.558598159" Sep 07 10:21:22.971056 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:22.971019 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-866d7898cc-lppjc"] Sep 07 10:21:22.975068 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:22.975045 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:22.977750 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:22.977720 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\"" Sep 07 10:21:22.977873 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:22.977809 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 07 10:21:22.978229 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:22.978211 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\"" Sep 07 10:21:22.978765 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:22.978604 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-7fqlz\"" Sep 07 10:21:22.978851 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:22.978823 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\"" Sep 07 10:21:22.988978 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:22.988875 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-866d7898cc-lppjc"] Sep 07 10:21:23.075847 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:23.075810 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5770463d-8a3e-4372-bd58-650bc3907cf6-registry-tls\") pod \"feast-credit-scoring-866d7898cc-lppjc\" (UID: \"5770463d-8a3e-4372-bd58-650bc3907cf6\") " pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:23.076048 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:23.075857 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzdxb\" (UniqueName: \"kubernetes.io/projected/5770463d-8a3e-4372-bd58-650bc3907cf6-kube-api-access-nzdxb\") pod \"feast-credit-scoring-866d7898cc-lppjc\" (UID: \"5770463d-8a3e-4372-bd58-650bc3907cf6\") " pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:23.076048 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:23.075884 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5770463d-8a3e-4372-bd58-650bc3907cf6-offline-tls\") pod \"feast-credit-scoring-866d7898cc-lppjc\" (UID: \"5770463d-8a3e-4372-bd58-650bc3907cf6\") " pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:23.076048 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:23.075902 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5770463d-8a3e-4372-bd58-650bc3907cf6-online-tls\") pod \"feast-credit-scoring-866d7898cc-lppjc\" (UID: \"5770463d-8a3e-4372-bd58-650bc3907cf6\") " pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:23.076048 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:23.076019 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5770463d-8a3e-4372-bd58-650bc3907cf6-feast-data\") pod \"feast-credit-scoring-866d7898cc-lppjc\" (UID: \"5770463d-8a3e-4372-bd58-650bc3907cf6\") " pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:23.176803 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:23.176766 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5770463d-8a3e-4372-bd58-650bc3907cf6-registry-tls\") pod \"feast-credit-scoring-866d7898cc-lppjc\" (UID: \"5770463d-8a3e-4372-bd58-650bc3907cf6\") " pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:23.176803 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:23.176817 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzdxb\" (UniqueName: \"kubernetes.io/projected/5770463d-8a3e-4372-bd58-650bc3907cf6-kube-api-access-nzdxb\") pod \"feast-credit-scoring-866d7898cc-lppjc\" (UID: \"5770463d-8a3e-4372-bd58-650bc3907cf6\") " pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:23.177048 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:21:23.176939 2552 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-registry-tls: secret "feast-credit-scoring-registry-tls" not found Sep 07 10:21:23.177048 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:21:23.177031 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5770463d-8a3e-4372-bd58-650bc3907cf6-registry-tls podName:5770463d-8a3e-4372-bd58-650bc3907cf6 nodeName:}" failed. No retries permitted until 2026-09-07 10:21:23.677008105 +0000 UTC m=+800.645419879 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/5770463d-8a3e-4372-bd58-650bc3907cf6-registry-tls") pod "feast-credit-scoring-866d7898cc-lppjc" (UID: "5770463d-8a3e-4372-bd58-650bc3907cf6") : secret "feast-credit-scoring-registry-tls" not found Sep 07 10:21:23.177162 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:23.176942 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5770463d-8a3e-4372-bd58-650bc3907cf6-offline-tls\") pod \"feast-credit-scoring-866d7898cc-lppjc\" (UID: \"5770463d-8a3e-4372-bd58-650bc3907cf6\") " pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:23.177162 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:23.177114 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5770463d-8a3e-4372-bd58-650bc3907cf6-online-tls\") pod \"feast-credit-scoring-866d7898cc-lppjc\" (UID: \"5770463d-8a3e-4372-bd58-650bc3907cf6\") " pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:23.177300 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:23.177178 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5770463d-8a3e-4372-bd58-650bc3907cf6-feast-data\") pod \"feast-credit-scoring-866d7898cc-lppjc\" (UID: \"5770463d-8a3e-4372-bd58-650bc3907cf6\") " pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:23.177363 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:21:23.177299 2552 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-online-tls: secret "feast-credit-scoring-online-tls" not found Sep 07 10:21:23.177363 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:21:23.177339 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5770463d-8a3e-4372-bd58-650bc3907cf6-online-tls podName:5770463d-8a3e-4372-bd58-650bc3907cf6 nodeName:}" failed. No retries permitted until 2026-09-07 10:21:23.677326171 +0000 UTC m=+800.645737948 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/5770463d-8a3e-4372-bd58-650bc3907cf6-online-tls") pod "feast-credit-scoring-866d7898cc-lppjc" (UID: "5770463d-8a3e-4372-bd58-650bc3907cf6") : secret "feast-credit-scoring-online-tls" not found Sep 07 10:21:23.177636 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:23.177613 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5770463d-8a3e-4372-bd58-650bc3907cf6-feast-data\") pod \"feast-credit-scoring-866d7898cc-lppjc\" (UID: \"5770463d-8a3e-4372-bd58-650bc3907cf6\") " pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:23.179598 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:23.179576 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5770463d-8a3e-4372-bd58-650bc3907cf6-offline-tls\") pod \"feast-credit-scoring-866d7898cc-lppjc\" (UID: \"5770463d-8a3e-4372-bd58-650bc3907cf6\") " pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:23.189422 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:23.189402 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzdxb\" (UniqueName: \"kubernetes.io/projected/5770463d-8a3e-4372-bd58-650bc3907cf6-kube-api-access-nzdxb\") pod \"feast-credit-scoring-866d7898cc-lppjc\" (UID: \"5770463d-8a3e-4372-bd58-650bc3907cf6\") " pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:23.681934 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:23.681896 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5770463d-8a3e-4372-bd58-650bc3907cf6-registry-tls\") pod \"feast-credit-scoring-866d7898cc-lppjc\" (UID: \"5770463d-8a3e-4372-bd58-650bc3907cf6\") " pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:23.682094 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:23.681952 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5770463d-8a3e-4372-bd58-650bc3907cf6-online-tls\") pod \"feast-credit-scoring-866d7898cc-lppjc\" (UID: \"5770463d-8a3e-4372-bd58-650bc3907cf6\") " pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:23.684419 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:23.684394 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5770463d-8a3e-4372-bd58-650bc3907cf6-registry-tls\") pod \"feast-credit-scoring-866d7898cc-lppjc\" (UID: \"5770463d-8a3e-4372-bd58-650bc3907cf6\") " pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:23.684419 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:23.684410 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5770463d-8a3e-4372-bd58-650bc3907cf6-online-tls\") pod \"feast-credit-scoring-866d7898cc-lppjc\" (UID: \"5770463d-8a3e-4372-bd58-650bc3907cf6\") " pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:23.892852 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:23.892810 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:24.169651 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:24.169612 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-866d7898cc-lppjc"] Sep 07 10:21:24.173829 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:21:24.173802 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5770463d_8a3e_4372_bd58_650bc3907cf6.slice/crio-f080e63939cda5d6f67825e88566077178bbfcebe718e6ace822131b3b975a21 WatchSource:0}: Error finding container f080e63939cda5d6f67825e88566077178bbfcebe718e6ace822131b3b975a21: Status 404 returned error can't find the container with id f080e63939cda5d6f67825e88566077178bbfcebe718e6ace822131b3b975a21 Sep 07 10:21:24.598922 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:24.598881 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" event={"ID":"5770463d-8a3e-4372-bd58-650bc3907cf6","Type":"ContainerStarted","Data":"36863f05f3bf159403b7b40967fdd4258041e624d44f9a6ae49c0db4daeed7f8"} Sep 07 10:21:24.599090 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:24.598954 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" event={"ID":"5770463d-8a3e-4372-bd58-650bc3907cf6","Type":"ContainerStarted","Data":"f080e63939cda5d6f67825e88566077178bbfcebe718e6ace822131b3b975a21"} Sep 07 10:21:25.603288 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:25.603178 2552 generic.go:356] "Generic (PLEG): container finished" podID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerID="36863f05f3bf159403b7b40967fdd4258041e624d44f9a6ae49c0db4daeed7f8" exitCode=0 Sep 07 10:21:25.603288 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:25.603224 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" event={"ID":"5770463d-8a3e-4372-bd58-650bc3907cf6","Type":"ContainerDied","Data":"36863f05f3bf159403b7b40967fdd4258041e624d44f9a6ae49c0db4daeed7f8"} Sep 07 10:21:26.609036 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:26.609000 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" event={"ID":"5770463d-8a3e-4372-bd58-650bc3907cf6","Type":"ContainerStarted","Data":"5c5786144b6b851811c1a786f5e773851e5b4575007f0f7e3a50845a7d3e8cf4"} Sep 07 10:21:32.634251 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:32.634217 2552 generic.go:356] "Generic (PLEG): container finished" podID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerID="5c5786144b6b851811c1a786f5e773851e5b4575007f0f7e3a50845a7d3e8cf4" exitCode=0 Sep 07 10:21:32.634609 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:32.634290 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" event={"ID":"5770463d-8a3e-4372-bd58-650bc3907cf6","Type":"ContainerDied","Data":"5c5786144b6b851811c1a786f5e773851e5b4575007f0f7e3a50845a7d3e8cf4"} Sep 07 10:21:33.643818 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:33.643773 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" event={"ID":"5770463d-8a3e-4372-bd58-650bc3907cf6","Type":"ContainerStarted","Data":"7499f381f08dfc8a3221bf85de0f8e1cbee05ceafc6b488e7a2615bbb0e83a01"} Sep 07 10:21:33.644287 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:33.643855 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" event={"ID":"5770463d-8a3e-4372-bd58-650bc3907cf6","Type":"ContainerStarted","Data":"5cd17760aa9fffcf0d8f946d56be7bafa5c3118b1f8f10b8bb3b3e693849d9c8"} Sep 07 10:21:33.644287 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:33.643884 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" event={"ID":"5770463d-8a3e-4372-bd58-650bc3907cf6","Type":"ContainerStarted","Data":"acfaa35459369af3b7a93eeb4c0aaecb7f69fbd55218346766d8461b4bf5f09f"} Sep 07 10:21:33.666577 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:33.666517 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" podStartSLOduration=11.666494305 podStartE2EDuration="11.666494305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:21:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:21:33.664645636 +0000 UTC m=+810.633057426" watchObservedRunningTime="2026-09-07 10:21:33.666494305 +0000 UTC m=+810.634906099" Sep 07 10:21:33.893518 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:33.893468 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:33.893691 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:33.893610 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:33.893691 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:33.893639 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:35.893850 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:35.893805 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:35.894365 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:35.893891 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:35.894365 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:35.893913 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:35.895595 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:35.895560 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.20:6571: connect: connection refused" Sep 07 10:21:35.895595 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:35.895577 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Sep 07 10:21:35.895790 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:35.895587 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.20:8816: connect: connection refused" Sep 07 10:21:38.894142 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:38.894100 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.20:8816: connect: connection refused" Sep 07 10:21:38.894730 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:38.894099 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.20:6571: connect: connection refused" Sep 07 10:21:38.894860 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:38.894113 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Sep 07 10:21:41.893952 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:41.893900 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.20:6571: connect: connection refused" Sep 07 10:21:41.894859 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:41.894835 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:41.895777 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:41.895754 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:41.900816 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:41.900787 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:42.689284 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:42.689221 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:44.893887 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:44.893840 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.20:6571: connect: connection refused" Sep 07 10:21:47.893900 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:47.893867 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:21:47.894807 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:21:47.894786 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:22:52.475586 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.475547 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-866d7898cc-lppjc"] Sep 07 10:22:52.476061 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.475780 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="registry" containerID="cri-o://acfaa35459369af3b7a93eeb4c0aaecb7f69fbd55218346766d8461b4bf5f09f" gracePeriod=30 Sep 07 10:22:52.476061 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.475816 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="offline" containerID="cri-o://7499f381f08dfc8a3221bf85de0f8e1cbee05ceafc6b488e7a2615bbb0e83a01" gracePeriod=30 Sep 07 10:22:52.476061 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.475872 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="online" containerID="cri-o://5cd17760aa9fffcf0d8f946d56be7bafa5c3118b1f8f10b8bb3b3e693849d9c8" gracePeriod=30 Sep 07 10:22:52.491805 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.491772 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-72lf4"] Sep 07 10:22:52.491986 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.491965 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-72lf4" podUID="31958728-d03a-4b6f-b1ab-4709e7c01d44" containerName="postgres" containerID="cri-o://aa8e5d6f6b6fc91ab03be346e3fbd69e9897e4f2ecd3f18e1ff48450f1f8601b" gracePeriod=30 Sep 07 10:22:52.502403 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.502369 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-tt4zf"] Sep 07 10:22:52.502596 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.502556 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/redis-69bbd798b6-tt4zf" podUID="2b2fe0c9-3589-41c7-944c-ff6cfb62e841" containerName="redis" containerID="cri-o://debfa4c876c41b93f971b12469af508697089605aa2595fbbd2d65c39ea6cd1d" gracePeriod=30 Sep 07 10:22:52.686769 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.686723 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Sep 07 10:22:52.752443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.752420 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-72lf4" Sep 07 10:22:52.755147 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.755126 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-tt4zf" Sep 07 10:22:52.858937 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.858903 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/31958728-d03a-4b6f-b1ab-4709e7c01d44-postgresdata\") pod \"31958728-d03a-4b6f-b1ab-4709e7c01d44\" (UID: \"31958728-d03a-4b6f-b1ab-4709e7c01d44\") " Sep 07 10:22:52.859091 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.858972 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whzbr\" (UniqueName: \"kubernetes.io/projected/2b2fe0c9-3589-41c7-944c-ff6cfb62e841-kube-api-access-whzbr\") pod \"2b2fe0c9-3589-41c7-944c-ff6cfb62e841\" (UID: \"2b2fe0c9-3589-41c7-944c-ff6cfb62e841\") " Sep 07 10:22:52.859091 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.859022 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bv2z6\" (UniqueName: \"kubernetes.io/projected/31958728-d03a-4b6f-b1ab-4709e7c01d44-kube-api-access-bv2z6\") pod \"31958728-d03a-4b6f-b1ab-4709e7c01d44\" (UID: \"31958728-d03a-4b6f-b1ab-4709e7c01d44\") " Sep 07 10:22:52.861164 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.861134 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31958728-d03a-4b6f-b1ab-4709e7c01d44-kube-api-access-bv2z6" (OuterVolumeSpecName: "kube-api-access-bv2z6") pod "31958728-d03a-4b6f-b1ab-4709e7c01d44" (UID: "31958728-d03a-4b6f-b1ab-4709e7c01d44"). InnerVolumeSpecName "kube-api-access-bv2z6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:22:52.861293 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.861177 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b2fe0c9-3589-41c7-944c-ff6cfb62e841-kube-api-access-whzbr" (OuterVolumeSpecName: "kube-api-access-whzbr") pod "2b2fe0c9-3589-41c7-944c-ff6cfb62e841" (UID: "2b2fe0c9-3589-41c7-944c-ff6cfb62e841"). InnerVolumeSpecName "kube-api-access-whzbr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:22:52.888854 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.888816 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31958728-d03a-4b6f-b1ab-4709e7c01d44-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "31958728-d03a-4b6f-b1ab-4709e7c01d44" (UID: "31958728-d03a-4b6f-b1ab-4709e7c01d44"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 10:22:52.952899 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.952864 2552 generic.go:356] "Generic (PLEG): container finished" podID="31958728-d03a-4b6f-b1ab-4709e7c01d44" containerID="aa8e5d6f6b6fc91ab03be346e3fbd69e9897e4f2ecd3f18e1ff48450f1f8601b" exitCode=0 Sep 07 10:22:52.953073 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.952920 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-72lf4" Sep 07 10:22:52.953073 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.952956 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-72lf4" event={"ID":"31958728-d03a-4b6f-b1ab-4709e7c01d44","Type":"ContainerDied","Data":"aa8e5d6f6b6fc91ab03be346e3fbd69e9897e4f2ecd3f18e1ff48450f1f8601b"} Sep 07 10:22:52.953073 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.952997 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-72lf4" event={"ID":"31958728-d03a-4b6f-b1ab-4709e7c01d44","Type":"ContainerDied","Data":"a8ab347077633c9ab760d3be49aa0726431614000aa3e19f1988e827a3c520b5"} Sep 07 10:22:52.953073 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.953014 2552 scope.go:117] "RemoveContainer" containerID="aa8e5d6f6b6fc91ab03be346e3fbd69e9897e4f2ecd3f18e1ff48450f1f8601b" Sep 07 10:22:52.954315 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.954295 2552 generic.go:356] "Generic (PLEG): container finished" podID="2b2fe0c9-3589-41c7-944c-ff6cfb62e841" containerID="debfa4c876c41b93f971b12469af508697089605aa2595fbbd2d65c39ea6cd1d" exitCode=0 Sep 07 10:22:52.954419 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.954326 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-tt4zf" event={"ID":"2b2fe0c9-3589-41c7-944c-ff6cfb62e841","Type":"ContainerDied","Data":"debfa4c876c41b93f971b12469af508697089605aa2595fbbd2d65c39ea6cd1d"} Sep 07 10:22:52.954419 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.954351 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-tt4zf" Sep 07 10:22:52.954419 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.954370 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-tt4zf" event={"ID":"2b2fe0c9-3589-41c7-944c-ff6cfb62e841","Type":"ContainerDied","Data":"dde954c5db1723b50a5c45351c87ff95e65bebea67864d71184e9768dfaade23"} Sep 07 10:22:52.960335 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.960313 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whzbr\" (UniqueName: \"kubernetes.io/projected/2b2fe0c9-3589-41c7-944c-ff6cfb62e841-kube-api-access-whzbr\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:22:52.960426 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.960338 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bv2z6\" (UniqueName: \"kubernetes.io/projected/31958728-d03a-4b6f-b1ab-4709e7c01d44-kube-api-access-bv2z6\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:22:52.960426 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.960351 2552 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/31958728-d03a-4b6f-b1ab-4709e7c01d44-postgresdata\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:22:52.961522 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.961500 2552 scope.go:117] "RemoveContainer" containerID="aa8e5d6f6b6fc91ab03be346e3fbd69e9897e4f2ecd3f18e1ff48450f1f8601b" Sep 07 10:22:52.961911 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:22:52.961881 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa8e5d6f6b6fc91ab03be346e3fbd69e9897e4f2ecd3f18e1ff48450f1f8601b\": container with ID starting with aa8e5d6f6b6fc91ab03be346e3fbd69e9897e4f2ecd3f18e1ff48450f1f8601b not found: ID does not exist" containerID="aa8e5d6f6b6fc91ab03be346e3fbd69e9897e4f2ecd3f18e1ff48450f1f8601b" Sep 07 10:22:52.962019 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.961939 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa8e5d6f6b6fc91ab03be346e3fbd69e9897e4f2ecd3f18e1ff48450f1f8601b"} err="failed to get container status \"aa8e5d6f6b6fc91ab03be346e3fbd69e9897e4f2ecd3f18e1ff48450f1f8601b\": rpc error: code = NotFound desc = could not find container \"aa8e5d6f6b6fc91ab03be346e3fbd69e9897e4f2ecd3f18e1ff48450f1f8601b\": container with ID starting with aa8e5d6f6b6fc91ab03be346e3fbd69e9897e4f2ecd3f18e1ff48450f1f8601b not found: ID does not exist" Sep 07 10:22:52.962019 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.961962 2552 scope.go:117] "RemoveContainer" containerID="debfa4c876c41b93f971b12469af508697089605aa2595fbbd2d65c39ea6cd1d" Sep 07 10:22:52.969229 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.969209 2552 scope.go:117] "RemoveContainer" containerID="debfa4c876c41b93f971b12469af508697089605aa2595fbbd2d65c39ea6cd1d" Sep 07 10:22:52.969515 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:22:52.969496 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"debfa4c876c41b93f971b12469af508697089605aa2595fbbd2d65c39ea6cd1d\": container with ID starting with debfa4c876c41b93f971b12469af508697089605aa2595fbbd2d65c39ea6cd1d not found: ID does not exist" containerID="debfa4c876c41b93f971b12469af508697089605aa2595fbbd2d65c39ea6cd1d" Sep 07 10:22:52.969578 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.969523 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"debfa4c876c41b93f971b12469af508697089605aa2595fbbd2d65c39ea6cd1d"} err="failed to get container status \"debfa4c876c41b93f971b12469af508697089605aa2595fbbd2d65c39ea6cd1d\": rpc error: code = NotFound desc = could not find container \"debfa4c876c41b93f971b12469af508697089605aa2595fbbd2d65c39ea6cd1d\": container with ID starting with debfa4c876c41b93f971b12469af508697089605aa2595fbbd2d65c39ea6cd1d not found: ID does not exist" Sep 07 10:22:52.974263 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.974229 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-72lf4"] Sep 07 10:22:52.976217 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.976195 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-72lf4"] Sep 07 10:22:52.984830 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.984807 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-tt4zf"] Sep 07 10:22:52.986630 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:52.986608 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-69bbd798b6-tt4zf"] Sep 07 10:22:53.520515 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:53.520485 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b2fe0c9-3589-41c7-944c-ff6cfb62e841" path="/var/lib/kubelet/pods/2b2fe0c9-3589-41c7-944c-ff6cfb62e841/volumes" Sep 07 10:22:53.520866 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:53.520820 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31958728-d03a-4b6f-b1ab-4709e7c01d44" path="/var/lib/kubelet/pods/31958728-d03a-4b6f-b1ab-4709e7c01d44/volumes" Sep 07 10:22:53.961422 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:53.961391 2552 generic.go:356] "Generic (PLEG): container finished" podID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerID="5cd17760aa9fffcf0d8f946d56be7bafa5c3118b1f8f10b8bb3b3e693849d9c8" exitCode=0 Sep 07 10:22:53.961603 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:22:53.961466 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" event={"ID":"5770463d-8a3e-4372-bd58-650bc3907cf6","Type":"ContainerDied","Data":"5cd17760aa9fffcf0d8f946d56be7bafa5c3118b1f8f10b8bb3b3e693849d9c8"} Sep 07 10:23:02.686646 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:02.686562 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Sep 07 10:23:03.475345 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:03.475318 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/ovn-acl-logging/0.log" Sep 07 10:23:03.478163 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:03.478136 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/ovn-acl-logging/0.log" Sep 07 10:23:12.686445 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:12.686397 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Sep 07 10:23:12.686888 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:12.686543 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:23:22.686817 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:22.686772 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Sep 07 10:23:23.063398 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:23.063369 2552 generic.go:356] "Generic (PLEG): container finished" podID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerID="7499f381f08dfc8a3221bf85de0f8e1cbee05ceafc6b488e7a2615bbb0e83a01" exitCode=137 Sep 07 10:23:23.063398 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:23.063392 2552 generic.go:356] "Generic (PLEG): container finished" podID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerID="acfaa35459369af3b7a93eeb4c0aaecb7f69fbd55218346766d8461b4bf5f09f" exitCode=137 Sep 07 10:23:23.063555 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:23.063436 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" event={"ID":"5770463d-8a3e-4372-bd58-650bc3907cf6","Type":"ContainerDied","Data":"7499f381f08dfc8a3221bf85de0f8e1cbee05ceafc6b488e7a2615bbb0e83a01"} Sep 07 10:23:23.063555 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:23.063493 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" event={"ID":"5770463d-8a3e-4372-bd58-650bc3907cf6","Type":"ContainerDied","Data":"acfaa35459369af3b7a93eeb4c0aaecb7f69fbd55218346766d8461b4bf5f09f"} Sep 07 10:23:23.122732 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:23.122702 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:23:23.176292 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:23.176261 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5770463d-8a3e-4372-bd58-650bc3907cf6-online-tls\") pod \"5770463d-8a3e-4372-bd58-650bc3907cf6\" (UID: \"5770463d-8a3e-4372-bd58-650bc3907cf6\") " Sep 07 10:23:23.176450 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:23.176297 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5770463d-8a3e-4372-bd58-650bc3907cf6-registry-tls\") pod \"5770463d-8a3e-4372-bd58-650bc3907cf6\" (UID: \"5770463d-8a3e-4372-bd58-650bc3907cf6\") " Sep 07 10:23:23.176450 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:23.176319 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5770463d-8a3e-4372-bd58-650bc3907cf6-feast-data\") pod \"5770463d-8a3e-4372-bd58-650bc3907cf6\" (UID: \"5770463d-8a3e-4372-bd58-650bc3907cf6\") " Sep 07 10:23:23.176450 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:23.176336 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5770463d-8a3e-4372-bd58-650bc3907cf6-offline-tls\") pod \"5770463d-8a3e-4372-bd58-650bc3907cf6\" (UID: \"5770463d-8a3e-4372-bd58-650bc3907cf6\") " Sep 07 10:23:23.176450 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:23.176407 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzdxb\" (UniqueName: \"kubernetes.io/projected/5770463d-8a3e-4372-bd58-650bc3907cf6-kube-api-access-nzdxb\") pod \"5770463d-8a3e-4372-bd58-650bc3907cf6\" (UID: \"5770463d-8a3e-4372-bd58-650bc3907cf6\") " Sep 07 10:23:23.178544 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:23.178513 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5770463d-8a3e-4372-bd58-650bc3907cf6-online-tls" (OuterVolumeSpecName: "online-tls") pod "5770463d-8a3e-4372-bd58-650bc3907cf6" (UID: "5770463d-8a3e-4372-bd58-650bc3907cf6"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:23:23.178544 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:23.178517 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5770463d-8a3e-4372-bd58-650bc3907cf6-kube-api-access-nzdxb" (OuterVolumeSpecName: "kube-api-access-nzdxb") pod "5770463d-8a3e-4372-bd58-650bc3907cf6" (UID: "5770463d-8a3e-4372-bd58-650bc3907cf6"). InnerVolumeSpecName "kube-api-access-nzdxb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:23:23.178821 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:23.178784 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5770463d-8a3e-4372-bd58-650bc3907cf6-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "5770463d-8a3e-4372-bd58-650bc3907cf6" (UID: "5770463d-8a3e-4372-bd58-650bc3907cf6"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:23:23.178821 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:23.178795 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5770463d-8a3e-4372-bd58-650bc3907cf6-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5770463d-8a3e-4372-bd58-650bc3907cf6" (UID: "5770463d-8a3e-4372-bd58-650bc3907cf6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:23:23.187416 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:23.187393 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5770463d-8a3e-4372-bd58-650bc3907cf6-feast-data" (OuterVolumeSpecName: "feast-data") pod "5770463d-8a3e-4372-bd58-650bc3907cf6" (UID: "5770463d-8a3e-4372-bd58-650bc3907cf6"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 10:23:23.277112 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:23.277033 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzdxb\" (UniqueName: \"kubernetes.io/projected/5770463d-8a3e-4372-bd58-650bc3907cf6-kube-api-access-nzdxb\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:23:23.277112 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:23.277064 2552 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5770463d-8a3e-4372-bd58-650bc3907cf6-online-tls\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:23:23.277112 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:23.277075 2552 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/5770463d-8a3e-4372-bd58-650bc3907cf6-registry-tls\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:23:23.277112 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:23.277083 2552 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5770463d-8a3e-4372-bd58-650bc3907cf6-feast-data\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:23:23.277112 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:23.277090 2552 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5770463d-8a3e-4372-bd58-650bc3907cf6-offline-tls\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:23:24.069962 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:24.069929 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" event={"ID":"5770463d-8a3e-4372-bd58-650bc3907cf6","Type":"ContainerDied","Data":"f080e63939cda5d6f67825e88566077178bbfcebe718e6ace822131b3b975a21"} Sep 07 10:23:24.069962 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:24.069965 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-866d7898cc-lppjc" Sep 07 10:23:24.070500 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:24.069993 2552 scope.go:117] "RemoveContainer" containerID="7499f381f08dfc8a3221bf85de0f8e1cbee05ceafc6b488e7a2615bbb0e83a01" Sep 07 10:23:24.078774 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:24.078755 2552 scope.go:117] "RemoveContainer" containerID="5cd17760aa9fffcf0d8f946d56be7bafa5c3118b1f8f10b8bb3b3e693849d9c8" Sep 07 10:23:24.086764 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:24.086744 2552 scope.go:117] "RemoveContainer" containerID="acfaa35459369af3b7a93eeb4c0aaecb7f69fbd55218346766d8461b4bf5f09f" Sep 07 10:23:24.087394 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:24.087370 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-866d7898cc-lppjc"] Sep 07 10:23:24.092178 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:24.092157 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-866d7898cc-lppjc"] Sep 07 10:23:24.094597 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:24.094577 2552 scope.go:117] "RemoveContainer" containerID="5c5786144b6b851811c1a786f5e773851e5b4575007f0f7e3a50845a7d3e8cf4" Sep 07 10:23:24.101210 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:24.101193 2552 scope.go:117] "RemoveContainer" containerID="36863f05f3bf159403b7b40967fdd4258041e624d44f9a6ae49c0db4daeed7f8" Sep 07 10:23:25.520867 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:23:25.520833 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" path="/var/lib/kubelet/pods/5770463d-8a3e-4372-bd58-650bc3907cf6/volumes" Sep 07 10:25:36.356304 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.356269 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-wrsz4"] Sep 07 10:25:36.356766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.356539 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="feast-init" Sep 07 10:25:36.356766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.356550 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="feast-init" Sep 07 10:25:36.356766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.356559 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="online" Sep 07 10:25:36.356766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.356565 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="online" Sep 07 10:25:36.356766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.356571 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="offline" Sep 07 10:25:36.356766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.356576 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="offline" Sep 07 10:25:36.356766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.356582 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31958728-d03a-4b6f-b1ab-4709e7c01d44" containerName="postgres" Sep 07 10:25:36.356766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.356587 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="31958728-d03a-4b6f-b1ab-4709e7c01d44" containerName="postgres" Sep 07 10:25:36.356766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.356595 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b2fe0c9-3589-41c7-944c-ff6cfb62e841" containerName="redis" Sep 07 10:25:36.356766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.356607 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b2fe0c9-3589-41c7-944c-ff6cfb62e841" containerName="redis" Sep 07 10:25:36.356766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.356622 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="registry" Sep 07 10:25:36.356766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.356627 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="registry" Sep 07 10:25:36.356766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.356633 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="feast-apply" Sep 07 10:25:36.356766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.356638 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="feast-apply" Sep 07 10:25:36.356766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.356676 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="online" Sep 07 10:25:36.356766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.356684 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="31958728-d03a-4b6f-b1ab-4709e7c01d44" containerName="postgres" Sep 07 10:25:36.356766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.356689 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="registry" Sep 07 10:25:36.356766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.356697 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b2fe0c9-3589-41c7-944c-ff6cfb62e841" containerName="redis" Sep 07 10:25:36.356766 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.356702 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="5770463d-8a3e-4372-bd58-650bc3907cf6" containerName="offline" Sep 07 10:25:36.359233 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.359215 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-wrsz4" Sep 07 10:25:36.361802 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.361741 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Sep 07 10:25:36.361802 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.361759 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Sep 07 10:25:36.361802 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.361770 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Sep 07 10:25:36.362417 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.362399 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-lqvlz\"" Sep 07 10:25:36.365745 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.365722 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-wrsz4"] Sep 07 10:25:36.389408 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.389374 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/39b24276-1e2b-49ae-9bf8-8e00e975dc38-postgresdata\") pod \"postgres-864846d4f5-wrsz4\" (UID: \"39b24276-1e2b-49ae-9bf8-8e00e975dc38\") " pod="test-ns-feast-rest/postgres-864846d4f5-wrsz4" Sep 07 10:25:36.389557 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.389429 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghvfn\" (UniqueName: \"kubernetes.io/projected/39b24276-1e2b-49ae-9bf8-8e00e975dc38-kube-api-access-ghvfn\") pod \"postgres-864846d4f5-wrsz4\" (UID: \"39b24276-1e2b-49ae-9bf8-8e00e975dc38\") " pod="test-ns-feast-rest/postgres-864846d4f5-wrsz4" Sep 07 10:25:36.490519 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.490479 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghvfn\" (UniqueName: \"kubernetes.io/projected/39b24276-1e2b-49ae-9bf8-8e00e975dc38-kube-api-access-ghvfn\") pod \"postgres-864846d4f5-wrsz4\" (UID: \"39b24276-1e2b-49ae-9bf8-8e00e975dc38\") " pod="test-ns-feast-rest/postgres-864846d4f5-wrsz4" Sep 07 10:25:36.490688 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.490536 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/39b24276-1e2b-49ae-9bf8-8e00e975dc38-postgresdata\") pod \"postgres-864846d4f5-wrsz4\" (UID: \"39b24276-1e2b-49ae-9bf8-8e00e975dc38\") " pod="test-ns-feast-rest/postgres-864846d4f5-wrsz4" Sep 07 10:25:36.490847 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.490833 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/39b24276-1e2b-49ae-9bf8-8e00e975dc38-postgresdata\") pod \"postgres-864846d4f5-wrsz4\" (UID: \"39b24276-1e2b-49ae-9bf8-8e00e975dc38\") " pod="test-ns-feast-rest/postgres-864846d4f5-wrsz4" Sep 07 10:25:36.498531 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.498502 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghvfn\" (UniqueName: \"kubernetes.io/projected/39b24276-1e2b-49ae-9bf8-8e00e975dc38-kube-api-access-ghvfn\") pod \"postgres-864846d4f5-wrsz4\" (UID: \"39b24276-1e2b-49ae-9bf8-8e00e975dc38\") " pod="test-ns-feast-rest/postgres-864846d4f5-wrsz4" Sep 07 10:25:36.669358 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.669275 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-wrsz4" Sep 07 10:25:36.934933 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:36.934856 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-wrsz4"] Sep 07 10:25:36.937895 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:25:36.937861 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39b24276_1e2b_49ae_9bf8_8e00e975dc38.slice/crio-e3a2502c39c5c8b12838752049ab349a179073ff08affcfad346b35d9b90ec6d WatchSource:0}: Error finding container e3a2502c39c5c8b12838752049ab349a179073ff08affcfad346b35d9b90ec6d: Status 404 returned error can't find the container with id e3a2502c39c5c8b12838752049ab349a179073ff08affcfad346b35d9b90ec6d Sep 07 10:25:37.513398 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:37.513361 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-wrsz4" event={"ID":"39b24276-1e2b-49ae-9bf8-8e00e975dc38","Type":"ContainerStarted","Data":"f56b9d9205950777b2b29823e06db8fa3dee2f175e51f0e6cd2453c1bf5b53a9"} Sep 07 10:25:37.513772 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:37.513411 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-wrsz4" event={"ID":"39b24276-1e2b-49ae-9bf8-8e00e975dc38","Type":"ContainerStarted","Data":"e3a2502c39c5c8b12838752049ab349a179073ff08affcfad346b35d9b90ec6d"} Sep 07 10:25:37.527500 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:37.527448 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-wrsz4" podStartSLOduration=1.5274347320000001 podStartE2EDuration="1.527434732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:25:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:25:37.526006217 +0000 UTC m=+1054.494418010" watchObservedRunningTime="2026-09-07 10:25:37.527434732 +0000 UTC m=+1054.495846524" Sep 07 10:25:41.896105 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:41.896062 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n"] Sep 07 10:25:41.901057 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:41.901027 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:25:41.903154 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:41.903128 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\"" Sep 07 10:25:41.903315 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:41.903128 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\"" Sep 07 10:25:41.903315 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:41.903184 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-lp4sn\"" Sep 07 10:25:41.903440 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:41.903427 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Sep 07 10:25:41.907687 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:41.907665 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n"] Sep 07 10:25:41.929982 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:41.929944 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/263132b6-04fd-4510-a310-7230b606d115-online-tls\") pod \"feast-credit-scoring-657657d8bf-c2p4n\" (UID: \"263132b6-04fd-4510-a310-7230b606d115\") " pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:25:41.930130 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:41.929987 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/263132b6-04fd-4510-a310-7230b606d115-registry-tls\") pod \"feast-credit-scoring-657657d8bf-c2p4n\" (UID: \"263132b6-04fd-4510-a310-7230b606d115\") " pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:25:41.930130 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:41.930085 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trthf\" (UniqueName: \"kubernetes.io/projected/263132b6-04fd-4510-a310-7230b606d115-kube-api-access-trthf\") pod \"feast-credit-scoring-657657d8bf-c2p4n\" (UID: \"263132b6-04fd-4510-a310-7230b606d115\") " pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:25:41.930130 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:41.930124 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/263132b6-04fd-4510-a310-7230b606d115-feast-data\") pod \"feast-credit-scoring-657657d8bf-c2p4n\" (UID: \"263132b6-04fd-4510-a310-7230b606d115\") " pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:25:42.030964 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:42.030919 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trthf\" (UniqueName: \"kubernetes.io/projected/263132b6-04fd-4510-a310-7230b606d115-kube-api-access-trthf\") pod \"feast-credit-scoring-657657d8bf-c2p4n\" (UID: \"263132b6-04fd-4510-a310-7230b606d115\") " pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:25:42.031166 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:42.030971 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/263132b6-04fd-4510-a310-7230b606d115-feast-data\") pod \"feast-credit-scoring-657657d8bf-c2p4n\" (UID: \"263132b6-04fd-4510-a310-7230b606d115\") " pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:25:42.031166 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:42.031015 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/263132b6-04fd-4510-a310-7230b606d115-online-tls\") pod \"feast-credit-scoring-657657d8bf-c2p4n\" (UID: \"263132b6-04fd-4510-a310-7230b606d115\") " pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:25:42.031166 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:42.031039 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/263132b6-04fd-4510-a310-7230b606d115-registry-tls\") pod \"feast-credit-scoring-657657d8bf-c2p4n\" (UID: \"263132b6-04fd-4510-a310-7230b606d115\") " pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:25:42.031374 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:42.031322 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/263132b6-04fd-4510-a310-7230b606d115-feast-data\") pod \"feast-credit-scoring-657657d8bf-c2p4n\" (UID: \"263132b6-04fd-4510-a310-7230b606d115\") " pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:25:42.033517 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:42.033494 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/263132b6-04fd-4510-a310-7230b606d115-online-tls\") pod \"feast-credit-scoring-657657d8bf-c2p4n\" (UID: \"263132b6-04fd-4510-a310-7230b606d115\") " pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:25:42.033517 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:42.033504 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/263132b6-04fd-4510-a310-7230b606d115-registry-tls\") pod \"feast-credit-scoring-657657d8bf-c2p4n\" (UID: \"263132b6-04fd-4510-a310-7230b606d115\") " pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:25:42.037734 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:42.037708 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trthf\" (UniqueName: \"kubernetes.io/projected/263132b6-04fd-4510-a310-7230b606d115-kube-api-access-trthf\") pod \"feast-credit-scoring-657657d8bf-c2p4n\" (UID: \"263132b6-04fd-4510-a310-7230b606d115\") " pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:25:42.211446 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:42.211364 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:25:42.484507 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:42.484483 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n"] Sep 07 10:25:42.486953 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:25:42.486924 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod263132b6_04fd_4510_a310_7230b606d115.slice/crio-aab1367fbc935b6ff455acaf1e520a42725e7b1aef88f03ea98fc5fbd76aff1a WatchSource:0}: Error finding container aab1367fbc935b6ff455acaf1e520a42725e7b1aef88f03ea98fc5fbd76aff1a: Status 404 returned error can't find the container with id aab1367fbc935b6ff455acaf1e520a42725e7b1aef88f03ea98fc5fbd76aff1a Sep 07 10:25:42.531440 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:42.531409 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" event={"ID":"263132b6-04fd-4510-a310-7230b606d115","Type":"ContainerStarted","Data":"aab1367fbc935b6ff455acaf1e520a42725e7b1aef88f03ea98fc5fbd76aff1a"} Sep 07 10:25:43.538979 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:43.538942 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" event={"ID":"263132b6-04fd-4510-a310-7230b606d115","Type":"ContainerStarted","Data":"c6ea0fe84a845a9d9b2bfe92f5362ced72d180e5f47c37891af285a0ea57f90c"} Sep 07 10:25:44.543861 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:44.543824 2552 generic.go:356] "Generic (PLEG): container finished" podID="263132b6-04fd-4510-a310-7230b606d115" containerID="c6ea0fe84a845a9d9b2bfe92f5362ced72d180e5f47c37891af285a0ea57f90c" exitCode=0 Sep 07 10:25:44.544265 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:44.543913 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" event={"ID":"263132b6-04fd-4510-a310-7230b606d115","Type":"ContainerDied","Data":"c6ea0fe84a845a9d9b2bfe92f5362ced72d180e5f47c37891af285a0ea57f90c"} Sep 07 10:25:45.548839 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:45.548800 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" event={"ID":"263132b6-04fd-4510-a310-7230b606d115","Type":"ContainerStarted","Data":"648b40b8f2544b8da96ecdeaff9aa08fec30d122fd0a3c25e9548a0de40a4fa0"} Sep 07 10:25:52.574801 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:52.574768 2552 generic.go:356] "Generic (PLEG): container finished" podID="263132b6-04fd-4510-a310-7230b606d115" containerID="648b40b8f2544b8da96ecdeaff9aa08fec30d122fd0a3c25e9548a0de40a4fa0" exitCode=0 Sep 07 10:25:52.575156 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:52.574855 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" event={"ID":"263132b6-04fd-4510-a310-7230b606d115","Type":"ContainerDied","Data":"648b40b8f2544b8da96ecdeaff9aa08fec30d122fd0a3c25e9548a0de40a4fa0"} Sep 07 10:25:53.582117 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:53.582071 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" event={"ID":"263132b6-04fd-4510-a310-7230b606d115","Type":"ContainerStarted","Data":"f4204908843e14600b4ac3e6d2212f6924de116c4528b29e2d5482f893685ec4"} Sep 07 10:25:53.582539 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:53.582152 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" event={"ID":"263132b6-04fd-4510-a310-7230b606d115","Type":"ContainerStarted","Data":"cd64f4e12353a845db54312bbe635015bf80779d7bfade232b536c2447e4d2d7"} Sep 07 10:25:53.601560 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:53.601495 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" podStartSLOduration=12.601476582 podStartE2EDuration="12.601476582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:25:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:25:53.601315533 +0000 UTC m=+1070.569727318" watchObservedRunningTime="2026-09-07 10:25:53.601476582 +0000 UTC m=+1070.569888376" Sep 07 10:25:54.211568 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:54.211523 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:25:54.211754 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:54.211605 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:25:54.213216 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:54.213180 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" podUID="263132b6-04fd-4510-a310-7230b606d115" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.22:6571: connect: connection refused" Sep 07 10:25:54.213314 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:54.213206 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" podUID="263132b6-04fd-4510-a310-7230b606d115" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused" Sep 07 10:25:57.212120 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:57.212032 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" podUID="263132b6-04fd-4510-a310-7230b606d115" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.22:6571: connect: connection refused" Sep 07 10:25:57.212512 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:25:57.212169 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" podUID="263132b6-04fd-4510-a310-7230b606d115" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused" Sep 07 10:26:00.212388 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:00.212336 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" podUID="263132b6-04fd-4510-a310-7230b606d115" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.22:6571: connect: connection refused" Sep 07 10:26:00.218412 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:00.218379 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:26:00.218971 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:00.218949 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:26:00.221877 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:00.221854 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:26:02.212273 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:02.212218 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:26:03.212540 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:03.212484 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" podUID="263132b6-04fd-4510-a310-7230b606d115" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.22:6571: connect: connection refused" Sep 07 10:26:06.213218 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:06.213185 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:26:06.214001 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:06.213980 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:26:07.697235 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:07.697198 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp"] Sep 07 10:26:07.700345 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:07.700320 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:26:07.702524 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:07.702500 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\"" Sep 07 10:26:07.702666 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:07.702551 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\"" Sep 07 10:26:07.702730 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:07.702712 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-fxpvn\"" Sep 07 10:26:07.702785 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:07.702731 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Sep 07 10:26:07.710860 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:07.710838 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp"] Sep 07 10:26:07.831570 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:07.831528 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d91baf8c-c10d-43da-83ca-632eb9df4756-online-tls\") pod \"feast-driver-ranking-7b46cc778b-lklwp\" (UID: \"d91baf8c-c10d-43da-83ca-632eb9df4756\") " pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:26:07.831791 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:07.831579 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntdws\" (UniqueName: \"kubernetes.io/projected/d91baf8c-c10d-43da-83ca-632eb9df4756-kube-api-access-ntdws\") pod \"feast-driver-ranking-7b46cc778b-lklwp\" (UID: \"d91baf8c-c10d-43da-83ca-632eb9df4756\") " pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:26:07.831791 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:07.831629 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d91baf8c-c10d-43da-83ca-632eb9df4756-registry-tls\") pod \"feast-driver-ranking-7b46cc778b-lklwp\" (UID: \"d91baf8c-c10d-43da-83ca-632eb9df4756\") " pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:26:07.831791 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:07.831684 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d91baf8c-c10d-43da-83ca-632eb9df4756-feast-data\") pod \"feast-driver-ranking-7b46cc778b-lklwp\" (UID: \"d91baf8c-c10d-43da-83ca-632eb9df4756\") " pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:26:07.932263 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:07.932203 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d91baf8c-c10d-43da-83ca-632eb9df4756-feast-data\") pod \"feast-driver-ranking-7b46cc778b-lklwp\" (UID: \"d91baf8c-c10d-43da-83ca-632eb9df4756\") " pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:26:07.932419 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:07.932274 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d91baf8c-c10d-43da-83ca-632eb9df4756-online-tls\") pod \"feast-driver-ranking-7b46cc778b-lklwp\" (UID: \"d91baf8c-c10d-43da-83ca-632eb9df4756\") " pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:26:07.932419 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:07.932314 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntdws\" (UniqueName: \"kubernetes.io/projected/d91baf8c-c10d-43da-83ca-632eb9df4756-kube-api-access-ntdws\") pod \"feast-driver-ranking-7b46cc778b-lklwp\" (UID: \"d91baf8c-c10d-43da-83ca-632eb9df4756\") " pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:26:07.932419 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:07.932338 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d91baf8c-c10d-43da-83ca-632eb9df4756-registry-tls\") pod \"feast-driver-ranking-7b46cc778b-lklwp\" (UID: \"d91baf8c-c10d-43da-83ca-632eb9df4756\") " pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:26:07.932419 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:26:07.932375 2552 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-online-tls: secret "feast-driver-ranking-online-tls" not found Sep 07 10:26:07.932572 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:26:07.932450 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d91baf8c-c10d-43da-83ca-632eb9df4756-online-tls podName:d91baf8c-c10d-43da-83ca-632eb9df4756 nodeName:}" failed. No retries permitted until 2026-09-07 10:26:08.432427401 +0000 UTC m=+1085.400839185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/d91baf8c-c10d-43da-83ca-632eb9df4756-online-tls") pod "feast-driver-ranking-7b46cc778b-lklwp" (UID: "d91baf8c-c10d-43da-83ca-632eb9df4756") : secret "feast-driver-ranking-online-tls" not found Sep 07 10:26:07.932664 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:07.932646 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d91baf8c-c10d-43da-83ca-632eb9df4756-feast-data\") pod \"feast-driver-ranking-7b46cc778b-lklwp\" (UID: \"d91baf8c-c10d-43da-83ca-632eb9df4756\") " pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:26:07.934826 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:07.934797 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d91baf8c-c10d-43da-83ca-632eb9df4756-registry-tls\") pod \"feast-driver-ranking-7b46cc778b-lklwp\" (UID: \"d91baf8c-c10d-43da-83ca-632eb9df4756\") " pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:26:07.940501 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:07.940473 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntdws\" (UniqueName: \"kubernetes.io/projected/d91baf8c-c10d-43da-83ca-632eb9df4756-kube-api-access-ntdws\") pod \"feast-driver-ranking-7b46cc778b-lklwp\" (UID: \"d91baf8c-c10d-43da-83ca-632eb9df4756\") " pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:26:08.436939 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:08.436896 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d91baf8c-c10d-43da-83ca-632eb9df4756-online-tls\") pod \"feast-driver-ranking-7b46cc778b-lklwp\" (UID: \"d91baf8c-c10d-43da-83ca-632eb9df4756\") " pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:26:08.439202 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:08.439180 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d91baf8c-c10d-43da-83ca-632eb9df4756-online-tls\") pod \"feast-driver-ranking-7b46cc778b-lklwp\" (UID: \"d91baf8c-c10d-43da-83ca-632eb9df4756\") " pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:26:08.609896 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:08.609839 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:26:08.880492 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:08.880465 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp"] Sep 07 10:26:08.882765 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:26:08.882737 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd91baf8c_c10d_43da_83ca_632eb9df4756.slice/crio-8b4452559246e4a9126a160185015f639d93b9c6831f3629a567e2b0ece4e563 WatchSource:0}: Error finding container 8b4452559246e4a9126a160185015f639d93b9c6831f3629a567e2b0ece4e563: Status 404 returned error can't find the container with id 8b4452559246e4a9126a160185015f639d93b9c6831f3629a567e2b0ece4e563 Sep 07 10:26:09.663670 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:09.663633 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" event={"ID":"d91baf8c-c10d-43da-83ca-632eb9df4756","Type":"ContainerStarted","Data":"11cc024fd9292c03faac2e58205274e9899833bc46064c6d1ae182c468e13646"} Sep 07 10:26:09.663843 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:09.663711 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" event={"ID":"d91baf8c-c10d-43da-83ca-632eb9df4756","Type":"ContainerStarted","Data":"8b4452559246e4a9126a160185015f639d93b9c6831f3629a567e2b0ece4e563"} Sep 07 10:26:14.684578 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:14.684545 2552 generic.go:356] "Generic (PLEG): container finished" podID="d91baf8c-c10d-43da-83ca-632eb9df4756" containerID="11cc024fd9292c03faac2e58205274e9899833bc46064c6d1ae182c468e13646" exitCode=0 Sep 07 10:26:14.684962 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:14.684583 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" event={"ID":"d91baf8c-c10d-43da-83ca-632eb9df4756","Type":"ContainerDied","Data":"11cc024fd9292c03faac2e58205274e9899833bc46064c6d1ae182c468e13646"} Sep 07 10:26:15.690082 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:15.690047 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" event={"ID":"d91baf8c-c10d-43da-83ca-632eb9df4756","Type":"ContainerStarted","Data":"dbf1a6a44cf7db2a86f8f99e96e49a7ce259ba2160e03f7685cd66a1709fd6a8"} Sep 07 10:26:22.718409 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:22.718375 2552 generic.go:356] "Generic (PLEG): container finished" podID="d91baf8c-c10d-43da-83ca-632eb9df4756" containerID="dbf1a6a44cf7db2a86f8f99e96e49a7ce259ba2160e03f7685cd66a1709fd6a8" exitCode=0 Sep 07 10:26:22.718788 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:22.718444 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" event={"ID":"d91baf8c-c10d-43da-83ca-632eb9df4756","Type":"ContainerDied","Data":"dbf1a6a44cf7db2a86f8f99e96e49a7ce259ba2160e03f7685cd66a1709fd6a8"} Sep 07 10:26:23.726104 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:23.726055 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" event={"ID":"d91baf8c-c10d-43da-83ca-632eb9df4756","Type":"ContainerStarted","Data":"c2830f073eb2ae8e978edf75124eefb65bc8f5986e9c653c4b2f7e1080e4fa30"} Sep 07 10:26:23.726615 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:23.726140 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" event={"ID":"d91baf8c-c10d-43da-83ca-632eb9df4756","Type":"ContainerStarted","Data":"36724c6daee6572591a3359e489efb4f27c7e5cdcadecc288352d43c776bff1c"} Sep 07 10:26:23.746109 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:23.746038 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" podStartSLOduration=16.74601431 podStartE2EDuration="16.74601431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:26:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:26:23.745032944 +0000 UTC m=+1100.713444734" watchObservedRunningTime="2026-09-07 10:26:23.74601431 +0000 UTC m=+1100.714426104" Sep 07 10:26:26.610946 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:26.610901 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:26:26.611466 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:26.610983 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:26:26.612647 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:26.612607 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" podUID="d91baf8c-c10d-43da-83ca-632eb9df4756" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.23:6571: connect: connection refused" Sep 07 10:26:26.612803 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:26.612648 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" podUID="d91baf8c-c10d-43da-83ca-632eb9df4756" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Sep 07 10:26:28.610094 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:28.610036 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:26:28.610617 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:28.610125 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:26:29.610411 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:29.610361 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" podUID="d91baf8c-c10d-43da-83ca-632eb9df4756" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.23:6571: connect: connection refused" Sep 07 10:26:29.616919 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:29.616887 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:26:29.756648 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:29.756614 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:26:32.610264 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:32.610198 2552 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" podUID="d91baf8c-c10d-43da-83ca-632eb9df4756" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.23:6571: connect: connection refused" Sep 07 10:26:35.611320 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:35.611283 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:26:35.612154 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:26:35.612125 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:27:56.556014 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:56.555978 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-85bbffd6d8-vkwgv"] Sep 07 10:27:56.556533 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:56.556182 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-85bbffd6d8-vkwgv" podUID="3f5a496d-f7aa-4a13-be93-e662b0242a4e" containerName="manager" containerID="cri-o://15015645feb7dea70aa03ef881207175227b899e7313b5541b4038df0043d746" gracePeriod=10 Sep 07 10:27:56.807312 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:56.807254 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-85bbffd6d8-vkwgv" Sep 07 10:27:56.810689 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:56.810662 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n"] Sep 07 10:27:56.811056 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:56.810896 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" podUID="263132b6-04fd-4510-a310-7230b606d115" containerName="registry" containerID="cri-o://cd64f4e12353a845db54312bbe635015bf80779d7bfade232b536c2447e4d2d7" gracePeriod=30 Sep 07 10:27:56.811056 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:56.810989 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" podUID="263132b6-04fd-4510-a310-7230b606d115" containerName="online" containerID="cri-o://f4204908843e14600b4ac3e6d2212f6924de116c4528b29e2d5482f893685ec4" gracePeriod=30 Sep 07 10:27:56.824115 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:56.824084 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp"] Sep 07 10:27:56.824400 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:56.824375 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" podUID="d91baf8c-c10d-43da-83ca-632eb9df4756" containerName="registry" containerID="cri-o://36724c6daee6572591a3359e489efb4f27c7e5cdcadecc288352d43c776bff1c" gracePeriod=30 Sep 07 10:27:56.824499 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:56.824387 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" podUID="d91baf8c-c10d-43da-83ca-632eb9df4756" containerName="online" containerID="cri-o://c2830f073eb2ae8e978edf75124eefb65bc8f5986e9c653c4b2f7e1080e4fa30" gracePeriod=30 Sep 07 10:27:56.874052 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:56.874014 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvj74\" (UniqueName: \"kubernetes.io/projected/3f5a496d-f7aa-4a13-be93-e662b0242a4e-kube-api-access-rvj74\") pod \"3f5a496d-f7aa-4a13-be93-e662b0242a4e\" (UID: \"3f5a496d-f7aa-4a13-be93-e662b0242a4e\") " Sep 07 10:27:56.876278 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:56.876236 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f5a496d-f7aa-4a13-be93-e662b0242a4e-kube-api-access-rvj74" (OuterVolumeSpecName: "kube-api-access-rvj74") pod "3f5a496d-f7aa-4a13-be93-e662b0242a4e" (UID: "3f5a496d-f7aa-4a13-be93-e662b0242a4e"). InnerVolumeSpecName "kube-api-access-rvj74". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:27:56.974644 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:56.974606 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvj74\" (UniqueName: \"kubernetes.io/projected/3f5a496d-f7aa-4a13-be93-e662b0242a4e-kube-api-access-rvj74\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:27:57.077541 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:57.077454 2552 generic.go:356] "Generic (PLEG): container finished" podID="3f5a496d-f7aa-4a13-be93-e662b0242a4e" containerID="15015645feb7dea70aa03ef881207175227b899e7313b5541b4038df0043d746" exitCode=0 Sep 07 10:27:57.077541 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:57.077493 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-85bbffd6d8-vkwgv" Sep 07 10:27:57.077728 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:57.077544 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-85bbffd6d8-vkwgv" event={"ID":"3f5a496d-f7aa-4a13-be93-e662b0242a4e","Type":"ContainerDied","Data":"15015645feb7dea70aa03ef881207175227b899e7313b5541b4038df0043d746"} Sep 07 10:27:57.077728 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:57.077591 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-85bbffd6d8-vkwgv" event={"ID":"3f5a496d-f7aa-4a13-be93-e662b0242a4e","Type":"ContainerDied","Data":"6a4a8bf1b81eadefef3e4c49a8e4145a96a7d57913b6aa8ce090b370308ddb78"} Sep 07 10:27:57.077728 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:57.077612 2552 scope.go:117] "RemoveContainer" containerID="15015645feb7dea70aa03ef881207175227b899e7313b5541b4038df0043d746" Sep 07 10:27:57.085443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:57.085222 2552 scope.go:117] "RemoveContainer" containerID="15015645feb7dea70aa03ef881207175227b899e7313b5541b4038df0043d746" Sep 07 10:27:57.085591 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:27:57.085567 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15015645feb7dea70aa03ef881207175227b899e7313b5541b4038df0043d746\": container with ID starting with 15015645feb7dea70aa03ef881207175227b899e7313b5541b4038df0043d746 not found: ID does not exist" containerID="15015645feb7dea70aa03ef881207175227b899e7313b5541b4038df0043d746" Sep 07 10:27:57.085666 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:57.085597 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15015645feb7dea70aa03ef881207175227b899e7313b5541b4038df0043d746"} err="failed to get container status \"15015645feb7dea70aa03ef881207175227b899e7313b5541b4038df0043d746\": rpc error: code = NotFound desc = could not find container \"15015645feb7dea70aa03ef881207175227b899e7313b5541b4038df0043d746\": container with ID starting with 15015645feb7dea70aa03ef881207175227b899e7313b5541b4038df0043d746 not found: ID does not exist" Sep 07 10:27:57.096649 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:57.096628 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-85bbffd6d8-vkwgv"] Sep 07 10:27:57.099935 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:57.099911 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-85bbffd6d8-vkwgv"] Sep 07 10:27:57.522254 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:57.522196 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f5a496d-f7aa-4a13-be93-e662b0242a4e" path="/var/lib/kubelet/pods/3f5a496d-f7aa-4a13-be93-e662b0242a4e/volumes" Sep 07 10:27:58.083823 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:58.083785 2552 generic.go:356] "Generic (PLEG): container finished" podID="d91baf8c-c10d-43da-83ca-632eb9df4756" containerID="c2830f073eb2ae8e978edf75124eefb65bc8f5986e9c653c4b2f7e1080e4fa30" exitCode=0 Sep 07 10:27:58.084265 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:58.083853 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" event={"ID":"d91baf8c-c10d-43da-83ca-632eb9df4756","Type":"ContainerDied","Data":"c2830f073eb2ae8e978edf75124eefb65bc8f5986e9c653c4b2f7e1080e4fa30"} Sep 07 10:27:58.086108 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:58.086088 2552 generic.go:356] "Generic (PLEG): container finished" podID="263132b6-04fd-4510-a310-7230b606d115" containerID="f4204908843e14600b4ac3e6d2212f6924de116c4528b29e2d5482f893685ec4" exitCode=0 Sep 07 10:27:58.086205 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:58.086131 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" event={"ID":"263132b6-04fd-4510-a310-7230b606d115","Type":"ContainerDied","Data":"f4204908843e14600b4ac3e6d2212f6924de116c4528b29e2d5482f893685ec4"} Sep 07 10:27:59.350521 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:59.350485 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-wrsz4"] Sep 07 10:27:59.350924 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:59.350661 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-wrsz4" podUID="39b24276-1e2b-49ae-9bf8-8e00e975dc38" containerName="postgres" containerID="cri-o://f56b9d9205950777b2b29823e06db8fa3dee2f175e51f0e6cd2453c1bf5b53a9" gracePeriod=30 Sep 07 10:27:59.591147 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:59.591122 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-wrsz4" Sep 07 10:27:59.689792 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:59.689700 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghvfn\" (UniqueName: \"kubernetes.io/projected/39b24276-1e2b-49ae-9bf8-8e00e975dc38-kube-api-access-ghvfn\") pod \"39b24276-1e2b-49ae-9bf8-8e00e975dc38\" (UID: \"39b24276-1e2b-49ae-9bf8-8e00e975dc38\") " Sep 07 10:27:59.689792 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:59.689743 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/39b24276-1e2b-49ae-9bf8-8e00e975dc38-postgresdata\") pod \"39b24276-1e2b-49ae-9bf8-8e00e975dc38\" (UID: \"39b24276-1e2b-49ae-9bf8-8e00e975dc38\") " Sep 07 10:27:59.691862 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:59.691834 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39b24276-1e2b-49ae-9bf8-8e00e975dc38-kube-api-access-ghvfn" (OuterVolumeSpecName: "kube-api-access-ghvfn") pod "39b24276-1e2b-49ae-9bf8-8e00e975dc38" (UID: "39b24276-1e2b-49ae-9bf8-8e00e975dc38"). InnerVolumeSpecName "kube-api-access-ghvfn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:27:59.719640 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:59.719555 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39b24276-1e2b-49ae-9bf8-8e00e975dc38-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "39b24276-1e2b-49ae-9bf8-8e00e975dc38" (UID: "39b24276-1e2b-49ae-9bf8-8e00e975dc38"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 10:27:59.754056 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:59.754018 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" podUID="d91baf8c-c10d-43da-83ca-632eb9df4756" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Sep 07 10:27:59.790866 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:59.790829 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghvfn\" (UniqueName: \"kubernetes.io/projected/39b24276-1e2b-49ae-9bf8-8e00e975dc38-kube-api-access-ghvfn\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:27:59.790866 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:27:59.790863 2552 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/39b24276-1e2b-49ae-9bf8-8e00e975dc38-postgresdata\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:28:00.094773 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:00.094735 2552 generic.go:356] "Generic (PLEG): container finished" podID="39b24276-1e2b-49ae-9bf8-8e00e975dc38" containerID="f56b9d9205950777b2b29823e06db8fa3dee2f175e51f0e6cd2453c1bf5b53a9" exitCode=0 Sep 07 10:28:00.094969 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:00.094784 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-wrsz4" Sep 07 10:28:00.094969 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:00.094787 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-wrsz4" event={"ID":"39b24276-1e2b-49ae-9bf8-8e00e975dc38","Type":"ContainerDied","Data":"f56b9d9205950777b2b29823e06db8fa3dee2f175e51f0e6cd2453c1bf5b53a9"} Sep 07 10:28:00.094969 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:00.094836 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-wrsz4" event={"ID":"39b24276-1e2b-49ae-9bf8-8e00e975dc38","Type":"ContainerDied","Data":"e3a2502c39c5c8b12838752049ab349a179073ff08affcfad346b35d9b90ec6d"} Sep 07 10:28:00.094969 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:00.094861 2552 scope.go:117] "RemoveContainer" containerID="f56b9d9205950777b2b29823e06db8fa3dee2f175e51f0e6cd2453c1bf5b53a9" Sep 07 10:28:00.103571 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:00.103554 2552 scope.go:117] "RemoveContainer" containerID="f56b9d9205950777b2b29823e06db8fa3dee2f175e51f0e6cd2453c1bf5b53a9" Sep 07 10:28:00.103842 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:28:00.103823 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f56b9d9205950777b2b29823e06db8fa3dee2f175e51f0e6cd2453c1bf5b53a9\": container with ID starting with f56b9d9205950777b2b29823e06db8fa3dee2f175e51f0e6cd2453c1bf5b53a9 not found: ID does not exist" containerID="f56b9d9205950777b2b29823e06db8fa3dee2f175e51f0e6cd2453c1bf5b53a9" Sep 07 10:28:00.103910 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:00.103850 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f56b9d9205950777b2b29823e06db8fa3dee2f175e51f0e6cd2453c1bf5b53a9"} err="failed to get container status \"f56b9d9205950777b2b29823e06db8fa3dee2f175e51f0e6cd2453c1bf5b53a9\": rpc error: code = NotFound desc = could not find container \"f56b9d9205950777b2b29823e06db8fa3dee2f175e51f0e6cd2453c1bf5b53a9\": container with ID starting with f56b9d9205950777b2b29823e06db8fa3dee2f175e51f0e6cd2453c1bf5b53a9 not found: ID does not exist" Sep 07 10:28:00.113877 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:00.113848 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-wrsz4"] Sep 07 10:28:00.116104 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:00.116081 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-wrsz4"] Sep 07 10:28:00.219871 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:00.219826 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" podUID="263132b6-04fd-4510-a310-7230b606d115" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused" Sep 07 10:28:01.521024 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:01.520990 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39b24276-1e2b-49ae-9bf8-8e00e975dc38" path="/var/lib/kubelet/pods/39b24276-1e2b-49ae-9bf8-8e00e975dc38/volumes" Sep 07 10:28:03.503722 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:03.503691 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/ovn-acl-logging/0.log" Sep 07 10:28:03.504195 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:03.504098 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/ovn-acl-logging/0.log" Sep 07 10:28:09.753259 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:09.753192 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" podUID="d91baf8c-c10d-43da-83ca-632eb9df4756" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Sep 07 10:28:10.219280 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:10.219227 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" podUID="263132b6-04fd-4510-a310-7230b606d115" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused" Sep 07 10:28:19.753788 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:19.753747 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" podUID="d91baf8c-c10d-43da-83ca-632eb9df4756" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Sep 07 10:28:19.754164 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:19.753888 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:28:20.220080 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:20.220047 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" podUID="263132b6-04fd-4510-a310-7230b606d115" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused" Sep 07 10:28:20.220270 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:20.220158 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:28:27.192349 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.192266 2552 generic.go:356] "Generic (PLEG): container finished" podID="d91baf8c-c10d-43da-83ca-632eb9df4756" containerID="36724c6daee6572591a3359e489efb4f27c7e5cdcadecc288352d43c776bff1c" exitCode=137 Sep 07 10:28:27.192349 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.192333 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" event={"ID":"d91baf8c-c10d-43da-83ca-632eb9df4756","Type":"ContainerDied","Data":"36724c6daee6572591a3359e489efb4f27c7e5cdcadecc288352d43c776bff1c"} Sep 07 10:28:27.194698 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.194676 2552 generic.go:356] "Generic (PLEG): container finished" podID="263132b6-04fd-4510-a310-7230b606d115" containerID="cd64f4e12353a845db54312bbe635015bf80779d7bfade232b536c2447e4d2d7" exitCode=137 Sep 07 10:28:27.194819 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.194753 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" event={"ID":"263132b6-04fd-4510-a310-7230b606d115","Type":"ContainerDied","Data":"cd64f4e12353a845db54312bbe635015bf80779d7bfade232b536c2447e4d2d7"} Sep 07 10:28:27.481866 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.481841 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:28:27.484223 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.484199 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:28:27.598579 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.598547 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d91baf8c-c10d-43da-83ca-632eb9df4756-online-tls\") pod \"d91baf8c-c10d-43da-83ca-632eb9df4756\" (UID: \"d91baf8c-c10d-43da-83ca-632eb9df4756\") " Sep 07 10:28:27.598579 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.598582 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d91baf8c-c10d-43da-83ca-632eb9df4756-registry-tls\") pod \"d91baf8c-c10d-43da-83ca-632eb9df4756\" (UID: \"d91baf8c-c10d-43da-83ca-632eb9df4756\") " Sep 07 10:28:27.598841 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.598606 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/263132b6-04fd-4510-a310-7230b606d115-online-tls\") pod \"263132b6-04fd-4510-a310-7230b606d115\" (UID: \"263132b6-04fd-4510-a310-7230b606d115\") " Sep 07 10:28:27.598841 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.598639 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ntdws\" (UniqueName: \"kubernetes.io/projected/d91baf8c-c10d-43da-83ca-632eb9df4756-kube-api-access-ntdws\") pod \"d91baf8c-c10d-43da-83ca-632eb9df4756\" (UID: \"d91baf8c-c10d-43da-83ca-632eb9df4756\") " Sep 07 10:28:27.598841 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.598671 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/263132b6-04fd-4510-a310-7230b606d115-feast-data\") pod \"263132b6-04fd-4510-a310-7230b606d115\" (UID: \"263132b6-04fd-4510-a310-7230b606d115\") " Sep 07 10:28:27.598841 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.598700 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trthf\" (UniqueName: \"kubernetes.io/projected/263132b6-04fd-4510-a310-7230b606d115-kube-api-access-trthf\") pod \"263132b6-04fd-4510-a310-7230b606d115\" (UID: \"263132b6-04fd-4510-a310-7230b606d115\") " Sep 07 10:28:27.598841 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.598757 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d91baf8c-c10d-43da-83ca-632eb9df4756-feast-data\") pod \"d91baf8c-c10d-43da-83ca-632eb9df4756\" (UID: \"d91baf8c-c10d-43da-83ca-632eb9df4756\") " Sep 07 10:28:27.598841 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.598812 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/263132b6-04fd-4510-a310-7230b606d115-registry-tls\") pod \"263132b6-04fd-4510-a310-7230b606d115\" (UID: \"263132b6-04fd-4510-a310-7230b606d115\") " Sep 07 10:28:27.599540 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.599503 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d91baf8c-c10d-43da-83ca-632eb9df4756-feast-data" (OuterVolumeSpecName: "feast-data") pod "d91baf8c-c10d-43da-83ca-632eb9df4756" (UID: "d91baf8c-c10d-43da-83ca-632eb9df4756"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 10:28:27.600873 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.600847 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/263132b6-04fd-4510-a310-7230b606d115-online-tls" (OuterVolumeSpecName: "online-tls") pod "263132b6-04fd-4510-a310-7230b606d115" (UID: "263132b6-04fd-4510-a310-7230b606d115"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:28:27.601267 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.601221 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d91baf8c-c10d-43da-83ca-632eb9df4756-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d91baf8c-c10d-43da-83ca-632eb9df4756" (UID: "d91baf8c-c10d-43da-83ca-632eb9df4756"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:28:27.601341 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.601309 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/263132b6-04fd-4510-a310-7230b606d115-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "263132b6-04fd-4510-a310-7230b606d115" (UID: "263132b6-04fd-4510-a310-7230b606d115"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:28:27.601341 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.601314 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d91baf8c-c10d-43da-83ca-632eb9df4756-online-tls" (OuterVolumeSpecName: "online-tls") pod "d91baf8c-c10d-43da-83ca-632eb9df4756" (UID: "d91baf8c-c10d-43da-83ca-632eb9df4756"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 10:28:27.601453 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.601335 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/263132b6-04fd-4510-a310-7230b606d115-kube-api-access-trthf" (OuterVolumeSpecName: "kube-api-access-trthf") pod "263132b6-04fd-4510-a310-7230b606d115" (UID: "263132b6-04fd-4510-a310-7230b606d115"). InnerVolumeSpecName "kube-api-access-trthf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:28:27.601509 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.601494 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d91baf8c-c10d-43da-83ca-632eb9df4756-kube-api-access-ntdws" (OuterVolumeSpecName: "kube-api-access-ntdws") pod "d91baf8c-c10d-43da-83ca-632eb9df4756" (UID: "d91baf8c-c10d-43da-83ca-632eb9df4756"). InnerVolumeSpecName "kube-api-access-ntdws". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:28:27.608181 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.608156 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/263132b6-04fd-4510-a310-7230b606d115-feast-data" (OuterVolumeSpecName: "feast-data") pod "263132b6-04fd-4510-a310-7230b606d115" (UID: "263132b6-04fd-4510-a310-7230b606d115"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 10:28:27.700056 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.700015 2552 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d91baf8c-c10d-43da-83ca-632eb9df4756-online-tls\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:28:27.700056 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.700048 2552 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d91baf8c-c10d-43da-83ca-632eb9df4756-registry-tls\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:28:27.700056 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.700056 2552 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/263132b6-04fd-4510-a310-7230b606d115-online-tls\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:28:27.700056 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.700067 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntdws\" (UniqueName: \"kubernetes.io/projected/d91baf8c-c10d-43da-83ca-632eb9df4756-kube-api-access-ntdws\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:28:27.700341 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.700077 2552 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/263132b6-04fd-4510-a310-7230b606d115-feast-data\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:28:27.700341 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.700085 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trthf\" (UniqueName: \"kubernetes.io/projected/263132b6-04fd-4510-a310-7230b606d115-kube-api-access-trthf\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:28:27.700341 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.700093 2552 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d91baf8c-c10d-43da-83ca-632eb9df4756-feast-data\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:28:27.700341 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:27.700101 2552 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/263132b6-04fd-4510-a310-7230b606d115-registry-tls\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:28:28.201529 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:28.201442 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" Sep 07 10:28:28.201529 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:28.201431 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp" event={"ID":"d91baf8c-c10d-43da-83ca-632eb9df4756","Type":"ContainerDied","Data":"8b4452559246e4a9126a160185015f639d93b9c6831f3629a567e2b0ece4e563"} Sep 07 10:28:28.202036 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:28.201577 2552 scope.go:117] "RemoveContainer" containerID="c2830f073eb2ae8e978edf75124eefb65bc8f5986e9c653c4b2f7e1080e4fa30" Sep 07 10:28:28.204135 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:28.204111 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" event={"ID":"263132b6-04fd-4510-a310-7230b606d115","Type":"ContainerDied","Data":"aab1367fbc935b6ff455acaf1e520a42725e7b1aef88f03ea98fc5fbd76aff1a"} Sep 07 10:28:28.204267 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:28.204120 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n" Sep 07 10:28:28.210599 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:28.210566 2552 scope.go:117] "RemoveContainer" containerID="36724c6daee6572591a3359e489efb4f27c7e5cdcadecc288352d43c776bff1c" Sep 07 10:28:28.218813 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:28.218792 2552 scope.go:117] "RemoveContainer" containerID="dbf1a6a44cf7db2a86f8f99e96e49a7ce259ba2160e03f7685cd66a1709fd6a8" Sep 07 10:28:28.225314 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:28.225280 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp"] Sep 07 10:28:28.227313 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:28.227286 2552 scope.go:117] "RemoveContainer" containerID="11cc024fd9292c03faac2e58205274e9899833bc46064c6d1ae182c468e13646" Sep 07 10:28:28.228387 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:28.228361 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7b46cc778b-lklwp"] Sep 07 10:28:28.235251 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:28.235221 2552 scope.go:117] "RemoveContainer" containerID="f4204908843e14600b4ac3e6d2212f6924de116c4528b29e2d5482f893685ec4" Sep 07 10:28:28.238573 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:28.238549 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n"] Sep 07 10:28:28.241810 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:28.241782 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-657657d8bf-c2p4n"] Sep 07 10:28:28.244177 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:28.244161 2552 scope.go:117] "RemoveContainer" containerID="cd64f4e12353a845db54312bbe635015bf80779d7bfade232b536c2447e4d2d7" Sep 07 10:28:28.251794 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:28.251775 2552 scope.go:117] "RemoveContainer" containerID="648b40b8f2544b8da96ecdeaff9aa08fec30d122fd0a3c25e9548a0de40a4fa0" Sep 07 10:28:28.258832 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:28.258813 2552 scope.go:117] "RemoveContainer" containerID="c6ea0fe84a845a9d9b2bfe92f5362ced72d180e5f47c37891af285a0ea57f90c" Sep 07 10:28:29.523996 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:29.523963 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="263132b6-04fd-4510-a310-7230b606d115" path="/var/lib/kubelet/pods/263132b6-04fd-4510-a310-7230b606d115/volumes" Sep 07 10:28:29.524473 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:28:29.524458 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d91baf8c-c10d-43da-83ca-632eb9df4756" path="/var/lib/kubelet/pods/d91baf8c-c10d-43da-83ca-632eb9df4756/volumes" Sep 07 10:33:03.524201 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:33:03.524170 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/ovn-acl-logging/0.log" Sep 07 10:33:03.525089 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:33:03.525068 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/ovn-acl-logging/0.log" Sep 07 10:38:03.546832 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:38:03.546800 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/ovn-acl-logging/0.log" Sep 07 10:38:03.548533 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:38:03.548507 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/ovn-acl-logging/0.log" Sep 07 10:39:51.842344 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842300 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-d6q2s/must-gather-r2kzc"] Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842612 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39b24276-1e2b-49ae-9bf8-8e00e975dc38" containerName="postgres" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842631 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="39b24276-1e2b-49ae-9bf8-8e00e975dc38" containerName="postgres" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842646 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d91baf8c-c10d-43da-83ca-632eb9df4756" containerName="online" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842655 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="d91baf8c-c10d-43da-83ca-632eb9df4756" containerName="online" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842670 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="263132b6-04fd-4510-a310-7230b606d115" containerName="feast-init" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842679 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="263132b6-04fd-4510-a310-7230b606d115" containerName="feast-init" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842690 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d91baf8c-c10d-43da-83ca-632eb9df4756" containerName="feast-apply" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842698 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="d91baf8c-c10d-43da-83ca-632eb9df4756" containerName="feast-apply" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842712 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f5a496d-f7aa-4a13-be93-e662b0242a4e" containerName="manager" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842720 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f5a496d-f7aa-4a13-be93-e662b0242a4e" containerName="manager" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842729 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d91baf8c-c10d-43da-83ca-632eb9df4756" containerName="registry" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842736 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="d91baf8c-c10d-43da-83ca-632eb9df4756" containerName="registry" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842747 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="263132b6-04fd-4510-a310-7230b606d115" containerName="feast-apply" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842755 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="263132b6-04fd-4510-a310-7230b606d115" containerName="feast-apply" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842764 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="263132b6-04fd-4510-a310-7230b606d115" containerName="registry" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842770 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="263132b6-04fd-4510-a310-7230b606d115" containerName="registry" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842781 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="263132b6-04fd-4510-a310-7230b606d115" containerName="online" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842790 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="263132b6-04fd-4510-a310-7230b606d115" containerName="online" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842799 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d91baf8c-c10d-43da-83ca-632eb9df4756" containerName="feast-init" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842805 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="d91baf8c-c10d-43da-83ca-632eb9df4756" containerName="feast-init" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842850 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="39b24276-1e2b-49ae-9bf8-8e00e975dc38" containerName="postgres" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842858 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f5a496d-f7aa-4a13-be93-e662b0242a4e" containerName="manager" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842863 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="d91baf8c-c10d-43da-83ca-632eb9df4756" containerName="online" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842870 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="263132b6-04fd-4510-a310-7230b606d115" containerName="online" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842875 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="d91baf8c-c10d-43da-83ca-632eb9df4756" containerName="registry" Sep 07 10:39:51.844528 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.842880 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="263132b6-04fd-4510-a310-7230b606d115" containerName="registry" Sep 07 10:39:51.845373 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.845344 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d6q2s/must-gather-r2kzc" Sep 07 10:39:51.847858 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.847835 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d6q2s\"/\"openshift-service-ca.crt\"" Sep 07 10:39:51.847985 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.847880 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-d6q2s\"/\"default-dockercfg-z2q54\"" Sep 07 10:39:51.848545 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.848528 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d6q2s\"/\"kube-root-ca.crt\"" Sep 07 10:39:51.853288 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.853263 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d6q2s/must-gather-r2kzc"] Sep 07 10:39:51.948417 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.948375 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/19d6845b-97ba-4223-ae57-7fbbb28d91dd-must-gather-output\") pod \"must-gather-r2kzc\" (UID: \"19d6845b-97ba-4223-ae57-7fbbb28d91dd\") " pod="openshift-must-gather-d6q2s/must-gather-r2kzc" Sep 07 10:39:51.948417 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:51.948420 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmpc7\" (UniqueName: \"kubernetes.io/projected/19d6845b-97ba-4223-ae57-7fbbb28d91dd-kube-api-access-jmpc7\") pod \"must-gather-r2kzc\" (UID: \"19d6845b-97ba-4223-ae57-7fbbb28d91dd\") " pod="openshift-must-gather-d6q2s/must-gather-r2kzc" Sep 07 10:39:52.049723 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:52.049678 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmpc7\" (UniqueName: \"kubernetes.io/projected/19d6845b-97ba-4223-ae57-7fbbb28d91dd-kube-api-access-jmpc7\") pod \"must-gather-r2kzc\" (UID: \"19d6845b-97ba-4223-ae57-7fbbb28d91dd\") " pod="openshift-must-gather-d6q2s/must-gather-r2kzc" Sep 07 10:39:52.049917 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:52.049755 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/19d6845b-97ba-4223-ae57-7fbbb28d91dd-must-gather-output\") pod \"must-gather-r2kzc\" (UID: \"19d6845b-97ba-4223-ae57-7fbbb28d91dd\") " pod="openshift-must-gather-d6q2s/must-gather-r2kzc" Sep 07 10:39:52.050093 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:52.050071 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/19d6845b-97ba-4223-ae57-7fbbb28d91dd-must-gather-output\") pod \"must-gather-r2kzc\" (UID: \"19d6845b-97ba-4223-ae57-7fbbb28d91dd\") " pod="openshift-must-gather-d6q2s/must-gather-r2kzc" Sep 07 10:39:52.059354 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:52.059327 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmpc7\" (UniqueName: \"kubernetes.io/projected/19d6845b-97ba-4223-ae57-7fbbb28d91dd-kube-api-access-jmpc7\") pod \"must-gather-r2kzc\" (UID: \"19d6845b-97ba-4223-ae57-7fbbb28d91dd\") " pod="openshift-must-gather-d6q2s/must-gather-r2kzc" Sep 07 10:39:52.155627 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:52.155533 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d6q2s/must-gather-r2kzc" Sep 07 10:39:52.421292 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:52.421198 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d6q2s/must-gather-r2kzc"] Sep 07 10:39:52.425915 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:39:52.425885 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19d6845b_97ba_4223_ae57_7fbbb28d91dd.slice/crio-452be112b379f00b6d40924a579275073731d3346bcc4d0b97fe7a6909efbbcf WatchSource:0}: Error finding container 452be112b379f00b6d40924a579275073731d3346bcc4d0b97fe7a6909efbbcf: Status 404 returned error can't find the container with id 452be112b379f00b6d40924a579275073731d3346bcc4d0b97fe7a6909efbbcf Sep 07 10:39:52.427663 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:52.427646 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 07 10:39:52.438035 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:52.438009 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d6q2s/must-gather-r2kzc" event={"ID":"19d6845b-97ba-4223-ae57-7fbbb28d91dd","Type":"ContainerStarted","Data":"452be112b379f00b6d40924a579275073731d3346bcc4d0b97fe7a6909efbbcf"} Sep 07 10:39:59.466977 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:59.466935 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d6q2s/must-gather-r2kzc" event={"ID":"19d6845b-97ba-4223-ae57-7fbbb28d91dd","Type":"ContainerStarted","Data":"7113aaef6861bffc0a7aac393b4e012a28f0ce5948c8cc40933a249adf677994"} Sep 07 10:39:59.467403 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:59.466987 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d6q2s/must-gather-r2kzc" event={"ID":"19d6845b-97ba-4223-ae57-7fbbb28d91dd","Type":"ContainerStarted","Data":"026ffa4cc33c97e75a643e3f6b7c901ce7a356b5c004a3bcbbeed7f0b0e688ca"} Sep 07 10:39:59.483705 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:39:59.483655 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-d6q2s/must-gather-r2kzc" podStartSLOduration=2.523047133 podStartE2EDuration="8.483638365s" totalImagesPullingTime="5.960591232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:39:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:39:59.482562796 +0000 UTC m=+1916.450974602" watchObservedRunningTime="2026-09-07 10:39:59.483638365 +0000 UTC m=+1916.452050158" Sep 07 10:40:08.502075 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:08.502041 2552 generic.go:356] "Generic (PLEG): container finished" podID="19d6845b-97ba-4223-ae57-7fbbb28d91dd" containerID="026ffa4cc33c97e75a643e3f6b7c901ce7a356b5c004a3bcbbeed7f0b0e688ca" exitCode=0 Sep 07 10:40:08.502498 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:08.502101 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d6q2s/must-gather-r2kzc" event={"ID":"19d6845b-97ba-4223-ae57-7fbbb28d91dd","Type":"ContainerDied","Data":"026ffa4cc33c97e75a643e3f6b7c901ce7a356b5c004a3bcbbeed7f0b0e688ca"} Sep 07 10:40:08.502498 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:08.502459 2552 scope.go:117] "RemoveContainer" containerID="026ffa4cc33c97e75a643e3f6b7c901ce7a356b5c004a3bcbbeed7f0b0e688ca" Sep 07 10:40:09.074296 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:09.074264 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-d6q2s_must-gather-r2kzc_19d6845b-97ba-4223-ae57-7fbbb28d91dd/gather/0.log" Sep 07 10:40:12.390919 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:12.390868 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-llgqf_5c35d143-5a49-4636-b86b-c1dd3f6df6c3/global-pull-secret-syncer/0.log" Sep 07 10:40:12.506651 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:12.506618 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4qt9r_0e3d4718-d092-48e6-86f6-f09e1793eac1/konnectivity-agent/0.log" Sep 07 10:40:12.590304 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:12.590270 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-225.ec2.internal_35886f0cd374d86381ddcbd8cc29c623/haproxy/0.log" Sep 07 10:40:14.373577 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:14.373541 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-d6q2s/must-gather-r2kzc"] Sep 07 10:40:14.374034 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:14.373723 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-d6q2s/must-gather-r2kzc" podUID="19d6845b-97ba-4223-ae57-7fbbb28d91dd" containerName="copy" containerID="cri-o://7113aaef6861bffc0a7aac393b4e012a28f0ce5948c8cc40933a249adf677994" gracePeriod=2 Sep 07 10:40:14.377444 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:14.376986 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-d6q2s/must-gather-r2kzc"] Sep 07 10:40:14.526291 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:14.526262 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-d6q2s_must-gather-r2kzc_19d6845b-97ba-4223-ae57-7fbbb28d91dd/copy/0.log" Sep 07 10:40:14.526708 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:14.526682 2552 generic.go:356] "Generic (PLEG): container finished" podID="19d6845b-97ba-4223-ae57-7fbbb28d91dd" containerID="7113aaef6861bffc0a7aac393b4e012a28f0ce5948c8cc40933a249adf677994" exitCode=143 Sep 07 10:40:14.608281 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:14.608255 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-d6q2s_must-gather-r2kzc_19d6845b-97ba-4223-ae57-7fbbb28d91dd/copy/0.log" Sep 07 10:40:14.610009 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:14.609992 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d6q2s/must-gather-r2kzc" Sep 07 10:40:14.611772 ip-10-0-140-225 kubenswrapper[2552]: E0907 10:40:14.611747 2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-r2kzc\" is forbidden: User \"system:node:ip-10-0-140-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-d6q2s\": no relationship found between node 'ip-10-0-140-225.ec2.internal' and this object" podUID="19d6845b-97ba-4223-ae57-7fbbb28d91dd" pod="openshift-must-gather-d6q2s/must-gather-r2kzc" Sep 07 10:40:14.719009 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:14.718975 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmpc7\" (UniqueName: \"kubernetes.io/projected/19d6845b-97ba-4223-ae57-7fbbb28d91dd-kube-api-access-jmpc7\") pod \"19d6845b-97ba-4223-ae57-7fbbb28d91dd\" (UID: \"19d6845b-97ba-4223-ae57-7fbbb28d91dd\") " Sep 07 10:40:14.719159 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:14.719068 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/19d6845b-97ba-4223-ae57-7fbbb28d91dd-must-gather-output\") pod \"19d6845b-97ba-4223-ae57-7fbbb28d91dd\" (UID: \"19d6845b-97ba-4223-ae57-7fbbb28d91dd\") " Sep 07 10:40:14.719464 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:14.719441 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19d6845b-97ba-4223-ae57-7fbbb28d91dd-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "19d6845b-97ba-4223-ae57-7fbbb28d91dd" (UID: "19d6845b-97ba-4223-ae57-7fbbb28d91dd"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 10:40:14.721262 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:14.721226 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19d6845b-97ba-4223-ae57-7fbbb28d91dd-kube-api-access-jmpc7" (OuterVolumeSpecName: "kube-api-access-jmpc7") pod "19d6845b-97ba-4223-ae57-7fbbb28d91dd" (UID: "19d6845b-97ba-4223-ae57-7fbbb28d91dd"). InnerVolumeSpecName "kube-api-access-jmpc7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 10:40:14.820563 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:14.820518 2552 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/19d6845b-97ba-4223-ae57-7fbbb28d91dd-must-gather-output\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:40:14.820563 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:14.820556 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmpc7\" (UniqueName: \"kubernetes.io/projected/19d6845b-97ba-4223-ae57-7fbbb28d91dd-kube-api-access-jmpc7\") on node \"ip-10-0-140-225.ec2.internal\" DevicePath \"\"" Sep 07 10:40:15.520311 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:15.520276 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19d6845b-97ba-4223-ae57-7fbbb28d91dd" path="/var/lib/kubelet/pods/19d6845b-97ba-4223-ae57-7fbbb28d91dd/volumes" Sep 07 10:40:15.531389 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:15.531363 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-d6q2s_must-gather-r2kzc_19d6845b-97ba-4223-ae57-7fbbb28d91dd/copy/0.log" Sep 07 10:40:15.531775 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:15.531760 2552 scope.go:117] "RemoveContainer" containerID="7113aaef6861bffc0a7aac393b4e012a28f0ce5948c8cc40933a249adf677994" Sep 07 10:40:15.531832 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:15.531788 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d6q2s/must-gather-r2kzc" Sep 07 10:40:15.539003 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:15.538983 2552 scope.go:117] "RemoveContainer" containerID="026ffa4cc33c97e75a643e3f6b7c901ce7a356b5c004a3bcbbeed7f0b0e688ca" Sep 07 10:40:16.742119 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:16.742088 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hwpns_b5f2976e-e1b6-4659-80a2-555511cc90e8/node-exporter/0.log" Sep 07 10:40:16.763429 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:16.763389 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hwpns_b5f2976e-e1b6-4659-80a2-555511cc90e8/kube-rbac-proxy/0.log" Sep 07 10:40:16.786622 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:16.786594 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hwpns_b5f2976e-e1b6-4659-80a2-555511cc90e8/init-textfile/0.log" Sep 07 10:40:19.937796 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:19.937757 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb"] Sep 07 10:40:19.938265 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:19.937987 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19d6845b-97ba-4223-ae57-7fbbb28d91dd" containerName="gather" Sep 07 10:40:19.938265 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:19.937998 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="19d6845b-97ba-4223-ae57-7fbbb28d91dd" containerName="gather" Sep 07 10:40:19.938265 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:19.938006 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19d6845b-97ba-4223-ae57-7fbbb28d91dd" containerName="copy" Sep 07 10:40:19.938265 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:19.938011 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="19d6845b-97ba-4223-ae57-7fbbb28d91dd" containerName="copy" Sep 07 10:40:19.938265 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:19.938061 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="19d6845b-97ba-4223-ae57-7fbbb28d91dd" containerName="gather" Sep 07 10:40:19.938265 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:19.938069 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="19d6845b-97ba-4223-ae57-7fbbb28d91dd" containerName="copy" Sep 07 10:40:19.941284 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:19.941266 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb" Sep 07 10:40:19.943373 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:19.943345 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4plbr\"/\"openshift-service-ca.crt\"" Sep 07 10:40:19.944040 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:19.944022 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4plbr\"/\"kube-root-ca.crt\"" Sep 07 10:40:19.944103 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:19.944044 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4plbr\"/\"default-dockercfg-5l59n\"" Sep 07 10:40:19.948024 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:19.947991 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb"] Sep 07 10:40:19.957491 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:19.957460 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1daa1d1e-9eb3-43c2-ba45-39a778dbe9df-podres\") pod \"perf-node-gather-daemonset-j4qjb\" (UID: \"1daa1d1e-9eb3-43c2-ba45-39a778dbe9df\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb" Sep 07 10:40:19.957657 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:19.957500 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1daa1d1e-9eb3-43c2-ba45-39a778dbe9df-sys\") pod \"perf-node-gather-daemonset-j4qjb\" (UID: \"1daa1d1e-9eb3-43c2-ba45-39a778dbe9df\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb" Sep 07 10:40:19.957657 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:19.957550 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1daa1d1e-9eb3-43c2-ba45-39a778dbe9df-proc\") pod \"perf-node-gather-daemonset-j4qjb\" (UID: \"1daa1d1e-9eb3-43c2-ba45-39a778dbe9df\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb" Sep 07 10:40:19.957657 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:19.957595 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1daa1d1e-9eb3-43c2-ba45-39a778dbe9df-lib-modules\") pod \"perf-node-gather-daemonset-j4qjb\" (UID: \"1daa1d1e-9eb3-43c2-ba45-39a778dbe9df\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb" Sep 07 10:40:19.957809 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:19.957657 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wc7m\" (UniqueName: \"kubernetes.io/projected/1daa1d1e-9eb3-43c2-ba45-39a778dbe9df-kube-api-access-9wc7m\") pod \"perf-node-gather-daemonset-j4qjb\" (UID: \"1daa1d1e-9eb3-43c2-ba45-39a778dbe9df\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb" Sep 07 10:40:20.058440 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:20.058395 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1daa1d1e-9eb3-43c2-ba45-39a778dbe9df-podres\") pod \"perf-node-gather-daemonset-j4qjb\" (UID: \"1daa1d1e-9eb3-43c2-ba45-39a778dbe9df\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb" Sep 07 10:40:20.058440 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:20.058439 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1daa1d1e-9eb3-43c2-ba45-39a778dbe9df-sys\") pod \"perf-node-gather-daemonset-j4qjb\" (UID: \"1daa1d1e-9eb3-43c2-ba45-39a778dbe9df\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb" Sep 07 10:40:20.058671 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:20.058462 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1daa1d1e-9eb3-43c2-ba45-39a778dbe9df-proc\") pod \"perf-node-gather-daemonset-j4qjb\" (UID: \"1daa1d1e-9eb3-43c2-ba45-39a778dbe9df\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb" Sep 07 10:40:20.058671 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:20.058494 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1daa1d1e-9eb3-43c2-ba45-39a778dbe9df-lib-modules\") pod \"perf-node-gather-daemonset-j4qjb\" (UID: \"1daa1d1e-9eb3-43c2-ba45-39a778dbe9df\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb" Sep 07 10:40:20.058671 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:20.058511 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wc7m\" (UniqueName: \"kubernetes.io/projected/1daa1d1e-9eb3-43c2-ba45-39a778dbe9df-kube-api-access-9wc7m\") pod \"perf-node-gather-daemonset-j4qjb\" (UID: \"1daa1d1e-9eb3-43c2-ba45-39a778dbe9df\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb" Sep 07 10:40:20.058671 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:20.058549 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1daa1d1e-9eb3-43c2-ba45-39a778dbe9df-sys\") pod \"perf-node-gather-daemonset-j4qjb\" (UID: \"1daa1d1e-9eb3-43c2-ba45-39a778dbe9df\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb" Sep 07 10:40:20.058671 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:20.058588 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1daa1d1e-9eb3-43c2-ba45-39a778dbe9df-proc\") pod \"perf-node-gather-daemonset-j4qjb\" (UID: \"1daa1d1e-9eb3-43c2-ba45-39a778dbe9df\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb" Sep 07 10:40:20.058671 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:20.058590 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1daa1d1e-9eb3-43c2-ba45-39a778dbe9df-podres\") pod \"perf-node-gather-daemonset-j4qjb\" (UID: \"1daa1d1e-9eb3-43c2-ba45-39a778dbe9df\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb" Sep 07 10:40:20.058671 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:20.058662 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1daa1d1e-9eb3-43c2-ba45-39a778dbe9df-lib-modules\") pod \"perf-node-gather-daemonset-j4qjb\" (UID: \"1daa1d1e-9eb3-43c2-ba45-39a778dbe9df\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb" Sep 07 10:40:20.066474 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:20.066445 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wc7m\" (UniqueName: \"kubernetes.io/projected/1daa1d1e-9eb3-43c2-ba45-39a778dbe9df-kube-api-access-9wc7m\") pod \"perf-node-gather-daemonset-j4qjb\" (UID: \"1daa1d1e-9eb3-43c2-ba45-39a778dbe9df\") " pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb" Sep 07 10:40:20.251742 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:20.251697 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb" Sep 07 10:40:20.516570 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:20.516430 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb"] Sep 07 10:40:20.519174 ip-10-0-140-225 kubenswrapper[2552]: W0907 10:40:20.519141 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1daa1d1e_9eb3_43c2_ba45_39a778dbe9df.slice/crio-fc905055a210862f8b5fdde1e5c5908a86d6bd04ad69200c0240d3ebf25f43f4 WatchSource:0}: Error finding container fc905055a210862f8b5fdde1e5c5908a86d6bd04ad69200c0240d3ebf25f43f4: Status 404 returned error can't find the container with id fc905055a210862f8b5fdde1e5c5908a86d6bd04ad69200c0240d3ebf25f43f4 Sep 07 10:40:20.550205 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:20.550166 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb" event={"ID":"1daa1d1e-9eb3-43c2-ba45-39a778dbe9df","Type":"ContainerStarted","Data":"fc905055a210862f8b5fdde1e5c5908a86d6bd04ad69200c0240d3ebf25f43f4"} Sep 07 10:40:21.145260 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:21.145210 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-p5xgx_57c1502d-f287-4126-8d7a-ab800b49d84e/dns/0.log" Sep 07 10:40:21.163737 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:21.163711 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-p5xgx_57c1502d-f287-4126-8d7a-ab800b49d84e/kube-rbac-proxy/0.log" Sep 07 10:40:21.184758 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:21.184726 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kqh2w_f9e1b00d-e190-452a-8322-333f9397c2fa/dns-node-resolver/0.log" Sep 07 10:40:21.555190 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:21.555150 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb" event={"ID":"1daa1d1e-9eb3-43c2-ba45-39a778dbe9df","Type":"ContainerStarted","Data":"8382692f421163e87715e81e52f83e8f87eb364cec5f55062a22e4ce8e03986a"} Sep 07 10:40:21.555405 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:21.555291 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb" Sep 07 10:40:21.574407 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:21.571762 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb" podStartSLOduration=2.571740645 podStartE2EDuration="2.571740645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:40:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:40:21.570031577 +0000 UTC m=+1938.538443374" watchObservedRunningTime="2026-09-07 10:40:21.571740645 +0000 UTC m=+1938.540152440" Sep 07 10:40:21.671650 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:21.671611 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v9stm_aff3118f-d191-4990-b60b-958c0c686aec/node-ca/0.log" Sep 07 10:40:22.615958 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:22.615923 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cdkr7_24404e76-4654-44b0-ab31-06822ef18722/serve-healthcheck-canary/0.log" Sep 07 10:40:22.989443 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:22.989415 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9klwx_da9be304-bff3-4215-89ae-04bee504e10e/kube-rbac-proxy/0.log" Sep 07 10:40:23.008488 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:23.008455 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9klwx_da9be304-bff3-4215-89ae-04bee504e10e/exporter/0.log" Sep 07 10:40:23.027013 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:23.026979 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9klwx_da9be304-bff3-4215-89ae-04bee504e10e/extractor/0.log" Sep 07 10:40:27.128579 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:27.128528 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-l9jh8_24fbe6c1-0754-4fce-a795-334557b3cc16/migrator/0.log" Sep 07 10:40:27.146075 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:27.146044 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-l9jh8_24fbe6c1-0754-4fce-a795-334557b3cc16/graceful-termination/0.log" Sep 07 10:40:27.568222 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:27.568191 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4plbr/perf-node-gather-daemonset-j4qjb" Sep 07 10:40:28.439952 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:28.439922 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmbs2_a04783c7-15ce-4d0c-82a3-6b946c5ecda0/kube-multus-additional-cni-plugins/0.log" Sep 07 10:40:28.458944 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:28.458919 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmbs2_a04783c7-15ce-4d0c-82a3-6b946c5ecda0/egress-router-binary-copy/0.log" Sep 07 10:40:28.479176 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:28.479150 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmbs2_a04783c7-15ce-4d0c-82a3-6b946c5ecda0/cni-plugins/0.log" Sep 07 10:40:28.501751 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:28.501725 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmbs2_a04783c7-15ce-4d0c-82a3-6b946c5ecda0/bond-cni-plugin/0.log" Sep 07 10:40:28.522660 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:28.522634 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmbs2_a04783c7-15ce-4d0c-82a3-6b946c5ecda0/routeoverride-cni/0.log" Sep 07 10:40:28.541830 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:28.541802 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmbs2_a04783c7-15ce-4d0c-82a3-6b946c5ecda0/whereabouts-cni-bincopy/0.log" Sep 07 10:40:28.560931 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:28.560847 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmbs2_a04783c7-15ce-4d0c-82a3-6b946c5ecda0/whereabouts-cni/0.log" Sep 07 10:40:28.777161 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:28.777126 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vs9qt_2293164a-1a0c-49ea-95b1-d5d0b223d585/kube-multus/0.log" Sep 07 10:40:28.843230 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:28.843148 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-f5sf9_6cae549b-87f4-4031-b984-83a7d64f9dd3/network-metrics-daemon/0.log" Sep 07 10:40:28.867178 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:28.867147 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-f5sf9_6cae549b-87f4-4031-b984-83a7d64f9dd3/kube-rbac-proxy/0.log" Sep 07 10:40:30.084123 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:30.084095 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/ovn-controller/0.log" Sep 07 10:40:30.101485 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:30.101463 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/ovn-acl-logging/1.log" Sep 07 10:40:30.101485 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:30.101472 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/ovn-acl-logging/0.log" Sep 07 10:40:30.118929 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:30.118904 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/kube-rbac-proxy-node/0.log" Sep 07 10:40:30.140294 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:30.140259 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/kube-rbac-proxy-ovn-metrics/0.log" Sep 07 10:40:30.158374 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:30.158328 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/northd/0.log" Sep 07 10:40:30.177085 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:30.177056 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/nbdb/0.log" Sep 07 10:40:30.195969 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:30.195940 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/sbdb/0.log" Sep 07 10:40:30.232041 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:30.232004 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lcshb_fe03fc91-4aa7-46e8-a9a2-e78173df653b/ovnkube-controller/0.log" Sep 07 10:40:31.270854 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:31.270826 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-w9rwx_1359e8cd-f362-45c5-8071-c59d23eab440/network-check-target-container/0.log" Sep 07 10:40:32.090845 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:32.090818 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-z27cp_36ee18c1-8608-4bf7-84d8-f94c414d02bc/iptables-alerter/0.log" Sep 07 10:40:32.680253 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:32.680213 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-9rj9f_c06f0d60-b864-4616-82e6-3bf85f17249b/tuned/0.log" Sep 07 10:40:35.270678 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:35.270645 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca-operator_service-ca-operator-859f94c4b-5xkkz_29bcf16b-9878-4b5b-859e-d2cf6be8ff9d/service-ca-operator/1.log" Sep 07 10:40:35.271436 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:35.271418 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca-operator_service-ca-operator-859f94c4b-5xkkz_29bcf16b-9878-4b5b-859e-d2cf6be8ff9d/service-ca-operator/0.log" Sep 07 10:40:35.569633 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:35.569600 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca_service-ca-75d99c484-86dvg_76df16bc-648e-49d3-8496-23293f551fe2/service-ca-controller/0.log" Sep 07 10:40:36.050862 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:36.050829 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-m4tlq_b2836d86-221b-4f6e-9d5e-5f0edf93e61b/csi-driver/0.log" Sep 07 10:40:36.068373 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:36.068348 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-m4tlq_b2836d86-221b-4f6e-9d5e-5f0edf93e61b/csi-node-driver-registrar/0.log" Sep 07 10:40:36.086482 ip-10-0-140-225 kubenswrapper[2552]: I0907 10:40:36.086455 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-m4tlq_b2836d86-221b-4f6e-9d5e-5f0edf93e61b/csi-liveness-probe/0.log"