Sep 11 08:44:33.252783 ip-10-0-141-169 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 11 08:44:33.252793 ip-10-0-141-169 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 11 08:44:33.252801 ip-10-0-141-169 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 11 08:44:33.253014 ip-10-0-141-169 systemd[1]: Failed to start Kubernetes Kubelet. Sep 11 08:44:42.449855 ip-10-0-141-169 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Sep 11 08:44:42.449867 ip-10-0-141-169 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot f6661be899b648eea5d7cc5dbe332dac -- Sep 11 08:46:32.982578 ip-10-0-141-169 systemd[1]: Starting Kubernetes Kubelet... Sep 11 08:46:33.375219 ip-10-0-141-169 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 11 08:46:33.375219 ip-10-0-141-169 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 11 08:46:33.375219 ip-10-0-141-169 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 11 08:46:33.375219 ip-10-0-141-169 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 11 08:46:33.375219 ip-10-0-141-169 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 11 08:46:33.376193 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.376127 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 11 08:46:33.380102 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380088 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 08:46:33.380102 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380102 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 08:46:33.380170 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380106 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 08:46:33.380170 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380109 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 08:46:33.380170 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380112 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 08:46:33.380170 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380115 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 08:46:33.380170 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380118 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 08:46:33.380170 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380121 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 08:46:33.380170 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380124 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 08:46:33.380170 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380127 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 08:46:33.380170 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380129 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 08:46:33.380170 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380132 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 08:46:33.380170 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380135 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 08:46:33.380170 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380138 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 08:46:33.380170 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380140 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 08:46:33.380170 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380148 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 08:46:33.380170 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380151 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 08:46:33.380170 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380153 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 08:46:33.380170 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380156 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 08:46:33.380170 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380159 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 08:46:33.380170 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380163 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 08:46:33.380170 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380166 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 08:46:33.380668 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380169 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 08:46:33.380668 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380172 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 08:46:33.380668 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380175 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 08:46:33.380668 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380178 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 08:46:33.380668 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380181 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 08:46:33.380668 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380184 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 08:46:33.380668 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380187 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 08:46:33.380668 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380189 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 08:46:33.380668 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380192 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 08:46:33.380668 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380194 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 08:46:33.380668 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380197 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 08:46:33.380668 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380199 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 08:46:33.380668 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380203 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 08:46:33.380668 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380208 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 08:46:33.380668 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380212 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 08:46:33.380668 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380215 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 08:46:33.380668 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380218 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 08:46:33.380668 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380221 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 08:46:33.380668 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380224 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 08:46:33.381119 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380226 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 08:46:33.381119 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380229 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 08:46:33.381119 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380232 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 08:46:33.381119 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380235 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 08:46:33.381119 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380237 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 08:46:33.381119 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380239 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 08:46:33.381119 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380242 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 08:46:33.381119 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380244 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 08:46:33.381119 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380247 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 08:46:33.381119 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380249 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 08:46:33.381119 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380252 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 08:46:33.381119 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380254 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 08:46:33.381119 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380256 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 08:46:33.381119 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380259 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 08:46:33.381119 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380262 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 08:46:33.381119 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380265 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 08:46:33.381119 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380268 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 08:46:33.381119 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380271 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 08:46:33.381119 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380273 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 08:46:33.381119 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380276 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 08:46:33.381697 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380279 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 08:46:33.381697 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380281 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 08:46:33.381697 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380284 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 08:46:33.381697 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380286 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 08:46:33.381697 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380288 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 08:46:33.381697 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380291 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 08:46:33.381697 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380293 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 08:46:33.381697 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380296 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 08:46:33.381697 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380298 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 08:46:33.381697 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380313 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 08:46:33.381697 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380316 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 08:46:33.381697 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380319 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 08:46:33.381697 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380322 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 08:46:33.381697 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380325 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 08:46:33.381697 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380328 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 08:46:33.381697 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380330 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 08:46:33.381697 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380333 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 08:46:33.381697 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380335 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 08:46:33.381697 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380338 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 08:46:33.382173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380340 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 08:46:33.382173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380343 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 08:46:33.382173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380345 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 08:46:33.382173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380348 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 08:46:33.382173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380351 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 08:46:33.382173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380354 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 08:46:33.382173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380356 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 08:46:33.382173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380360 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 08:46:33.382173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380363 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 08:46:33.382173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380366 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 08:46:33.382173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380368 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 08:46:33.382173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380371 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 08:46:33.382173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380374 2562 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 08:46:33.382173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380376 2562 feature_gate.go:328] unrecognized feature gate: Example Sep 11 08:46:33.382173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380379 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 08:46:33.382173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380382 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 08:46:33.382173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380384 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 08:46:33.382173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380387 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 08:46:33.382173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.380389 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 08:46:33.382173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381074 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 08:46:33.382727 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381081 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 08:46:33.382727 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381085 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 08:46:33.382727 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381088 2562 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 08:46:33.382727 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381091 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 08:46:33.382727 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381094 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 08:46:33.382727 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381097 2562 feature_gate.go:328] unrecognized feature gate: Example Sep 11 08:46:33.382727 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381101 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 08:46:33.382727 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381103 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 08:46:33.382727 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381106 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 08:46:33.382727 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381109 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 08:46:33.382727 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381112 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 08:46:33.382727 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381114 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 08:46:33.382727 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381117 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 08:46:33.382727 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381120 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 08:46:33.382727 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381122 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 08:46:33.382727 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381125 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 08:46:33.382727 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381128 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 08:46:33.382727 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381132 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 08:46:33.382727 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381134 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 08:46:33.382727 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381137 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 08:46:33.383222 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381140 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 08:46:33.383222 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381143 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 08:46:33.383222 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381145 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 08:46:33.383222 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381148 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 08:46:33.383222 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381152 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 08:46:33.383222 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381156 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 08:46:33.383222 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381159 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 08:46:33.383222 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381162 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 08:46:33.383222 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381165 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 08:46:33.383222 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381168 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 08:46:33.383222 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381171 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 08:46:33.383222 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381174 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 08:46:33.383222 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381177 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 08:46:33.383222 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381180 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 08:46:33.383222 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381183 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 08:46:33.383222 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381185 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 08:46:33.383222 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381187 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 08:46:33.383222 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381191 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 08:46:33.383222 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381194 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 08:46:33.383703 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381196 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 08:46:33.383703 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381199 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 08:46:33.383703 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381201 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 08:46:33.383703 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381204 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 08:46:33.383703 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381207 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 08:46:33.383703 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381209 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 08:46:33.383703 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381214 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 08:46:33.383703 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381216 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 08:46:33.383703 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381219 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 08:46:33.383703 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381222 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 08:46:33.383703 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381224 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 08:46:33.383703 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381228 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 08:46:33.383703 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381232 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 08:46:33.383703 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381235 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 08:46:33.383703 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381238 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 08:46:33.383703 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381242 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 08:46:33.383703 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381245 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 08:46:33.383703 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381248 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 08:46:33.383703 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381251 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 08:46:33.384165 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381254 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 08:46:33.384165 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381257 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 08:46:33.384165 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381259 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 08:46:33.384165 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381262 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 08:46:33.384165 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381264 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 08:46:33.384165 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381267 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 08:46:33.384165 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381269 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 08:46:33.384165 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381272 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 08:46:33.384165 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381274 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 08:46:33.384165 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381277 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 08:46:33.384165 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381279 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 08:46:33.384165 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381282 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 08:46:33.384165 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381285 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 08:46:33.384165 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381288 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 08:46:33.384165 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381290 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 08:46:33.384165 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381293 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 08:46:33.384165 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381295 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 08:46:33.384165 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381298 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 08:46:33.384165 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381315 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 08:46:33.384165 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381318 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 08:46:33.384670 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381321 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 08:46:33.384670 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381323 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 08:46:33.384670 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381326 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 08:46:33.384670 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381329 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 08:46:33.384670 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381331 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 08:46:33.384670 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381334 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 08:46:33.384670 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381337 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 08:46:33.384670 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381339 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 08:46:33.384670 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381342 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 08:46:33.384670 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381346 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 08:46:33.384670 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381348 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 08:46:33.384670 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381351 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 08:46:33.384670 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381354 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 08:46:33.384670 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381356 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 08:46:33.384670 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381359 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 08:46:33.384670 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381361 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 08:46:33.384670 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381364 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 08:46:33.384670 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381366 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 08:46:33.384670 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381369 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 08:46:33.384670 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381372 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 08:46:33.385149 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381719 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 08:46:33.385149 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381723 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 08:46:33.385149 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381726 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 08:46:33.385149 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381729 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 08:46:33.385149 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381733 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 08:46:33.385149 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381736 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 08:46:33.385149 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381739 2562 feature_gate.go:328] unrecognized feature gate: Example Sep 11 08:46:33.385149 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381742 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 08:46:33.385149 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381744 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 08:46:33.385149 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381747 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 08:46:33.385149 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381750 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 08:46:33.385149 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381752 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 08:46:33.385149 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381755 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 08:46:33.385149 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381757 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 08:46:33.385149 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381760 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 08:46:33.385149 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381762 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 08:46:33.385149 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381765 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 08:46:33.385149 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381767 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 08:46:33.385149 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381770 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 08:46:33.385149 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381772 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 08:46:33.385643 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381775 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 08:46:33.385643 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381777 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 08:46:33.385643 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381780 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 08:46:33.385643 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381784 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 08:46:33.385643 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381789 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 08:46:33.385643 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381792 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 08:46:33.385643 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381795 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 08:46:33.385643 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381798 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 08:46:33.385643 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381801 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 08:46:33.385643 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381804 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 08:46:33.385643 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381807 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 08:46:33.385643 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381810 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 08:46:33.385643 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381813 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 08:46:33.385643 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381816 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 08:46:33.385643 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381819 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 08:46:33.385643 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381822 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 08:46:33.385643 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381825 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 08:46:33.385643 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381828 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 08:46:33.385643 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381831 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 08:46:33.386136 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381834 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 08:46:33.386136 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381837 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 08:46:33.386136 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381840 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 08:46:33.386136 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381843 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 08:46:33.386136 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381845 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 08:46:33.386136 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381848 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 08:46:33.386136 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381851 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 08:46:33.386136 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381853 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 08:46:33.386136 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381855 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 08:46:33.386136 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381858 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 08:46:33.386136 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381861 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 08:46:33.386136 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381863 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 08:46:33.386136 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381866 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 08:46:33.386136 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381868 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 08:46:33.386136 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381870 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 08:46:33.386136 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381873 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 08:46:33.386136 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381877 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 08:46:33.386136 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381879 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 08:46:33.386136 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381882 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 08:46:33.386136 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381885 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 08:46:33.386649 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381887 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 08:46:33.386649 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381890 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 08:46:33.386649 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381892 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 08:46:33.386649 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381895 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 08:46:33.386649 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381897 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 08:46:33.386649 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381900 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 08:46:33.386649 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381903 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 08:46:33.386649 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381905 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 08:46:33.386649 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381907 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 08:46:33.386649 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381910 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 08:46:33.386649 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381913 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 08:46:33.386649 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381916 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 08:46:33.386649 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381918 2562 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 08:46:33.386649 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381921 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 08:46:33.386649 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381923 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 08:46:33.386649 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381926 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 08:46:33.386649 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381928 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 08:46:33.386649 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381931 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 08:46:33.386649 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381935 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 08:46:33.386649 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381937 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 08:46:33.387173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381939 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 08:46:33.387173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381942 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 08:46:33.387173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381944 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 08:46:33.387173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381947 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 08:46:33.387173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381949 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 08:46:33.387173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381952 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 08:46:33.387173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381956 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 08:46:33.387173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381959 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 08:46:33.387173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381963 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 08:46:33.387173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381966 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 08:46:33.387173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381969 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 08:46:33.387173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381972 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 08:46:33.387173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381975 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 08:46:33.387173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381977 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 08:46:33.387173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381980 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 08:46:33.387173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381982 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 08:46:33.387173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381984 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 08:46:33.387173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381987 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 08:46:33.387173 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381990 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 08:46:33.387651 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.381992 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 08:46:33.387651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382069 2562 flags.go:64] FLAG: --address="0.0.0.0" Sep 11 08:46:33.387651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382102 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 11 08:46:33.387651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382109 2562 flags.go:64] FLAG: --anonymous-auth="true" Sep 11 08:46:33.387651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382113 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 11 08:46:33.387651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382117 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 11 08:46:33.387651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382120 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 11 08:46:33.387651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382124 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 11 08:46:33.387651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382128 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 11 08:46:33.387651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382131 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 11 08:46:33.387651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382134 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 11 08:46:33.387651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382138 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 11 08:46:33.387651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382141 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 11 08:46:33.387651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382144 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 11 08:46:33.387651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382147 2562 flags.go:64] FLAG: --cgroup-root="" Sep 11 08:46:33.387651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382150 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 11 08:46:33.387651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382152 2562 flags.go:64] FLAG: --client-ca-file="" Sep 11 08:46:33.387651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382155 2562 flags.go:64] FLAG: --cloud-provider="external" Sep 11 08:46:33.387651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382158 2562 flags.go:64] FLAG: --cluster-dns="[]" Sep 11 08:46:33.387651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382162 2562 flags.go:64] FLAG: --cluster-domain="" Sep 11 08:46:33.387651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382165 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 11 08:46:33.387651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382169 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 11 08:46:33.387651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382173 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 11 08:46:33.387651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382176 2562 flags.go:64] FLAG: --container-log-max-files="5" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382180 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382183 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382186 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382189 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382192 2562 flags.go:64] FLAG: --contention-profiling="false" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382195 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382198 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382201 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382204 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382210 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382213 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382216 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382219 2562 flags.go:64] FLAG: --enable-load-reader="false" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382222 2562 flags.go:64] FLAG: --enable-server="true" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382225 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382238 2562 flags.go:64] FLAG: --event-burst="100" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382242 2562 flags.go:64] FLAG: --event-qps="50" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382244 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382247 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382250 2562 flags.go:64] FLAG: --eviction-hard="" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382261 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382264 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382267 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382270 2562 flags.go:64] FLAG: --eviction-soft="" Sep 11 08:46:33.388237 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382273 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 11 08:46:33.388887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382276 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 11 08:46:33.388887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382279 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 11 08:46:33.388887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382282 2562 flags.go:64] FLAG: --experimental-mounter-path="" Sep 11 08:46:33.388887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382284 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 11 08:46:33.388887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382288 2562 flags.go:64] FLAG: --fail-swap-on="true" Sep 11 08:46:33.388887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382291 2562 flags.go:64] FLAG: --feature-gates="" Sep 11 08:46:33.388887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382294 2562 flags.go:64] FLAG: --file-check-frequency="20s" Sep 11 08:46:33.388887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382297 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 11 08:46:33.388887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382315 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 11 08:46:33.388887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382319 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 11 08:46:33.388887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382324 2562 flags.go:64] FLAG: --healthz-port="10248" Sep 11 08:46:33.388887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382329 2562 flags.go:64] FLAG: --help="false" Sep 11 08:46:33.388887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382334 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-141-169.ec2.internal" Sep 11 08:46:33.388887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382338 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 11 08:46:33.388887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382341 2562 flags.go:64] FLAG: --http-check-frequency="20s" Sep 11 08:46:33.388887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382344 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 11 08:46:33.388887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382349 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 11 08:46:33.388887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382352 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 11 08:46:33.388887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382355 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 11 08:46:33.388887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382358 2562 flags.go:64] FLAG: --image-service-endpoint="" Sep 11 08:46:33.388887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382360 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 11 08:46:33.388887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382363 2562 flags.go:64] FLAG: --kube-api-burst="100" Sep 11 08:46:33.388887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382367 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382370 2562 flags.go:64] FLAG: --kube-api-qps="50" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382373 2562 flags.go:64] FLAG: --kube-reserved="" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382376 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382379 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382382 2562 flags.go:64] FLAG: --kubelet-cgroups="" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382385 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382387 2562 flags.go:64] FLAG: --lock-file="" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382390 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382393 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382397 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382405 2562 flags.go:64] FLAG: --log-json-split-stream="false" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382421 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382425 2562 flags.go:64] FLAG: --log-text-split-stream="false" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382428 2562 flags.go:64] FLAG: --logging-format="text" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382431 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382434 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382437 2562 flags.go:64] FLAG: --manifest-url="" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382440 2562 flags.go:64] FLAG: --manifest-url-header="" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382444 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382447 2562 flags.go:64] FLAG: --max-open-files="1000000" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382451 2562 flags.go:64] FLAG: --max-pods="110" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382454 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382457 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382460 2562 flags.go:64] FLAG: --memory-manager-policy="None" Sep 11 08:46:33.389492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382463 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382467 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382470 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382473 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382480 2562 flags.go:64] FLAG: --node-status-max-images="50" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382484 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382489 2562 flags.go:64] FLAG: --oom-score-adj="-999" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382494 2562 flags.go:64] FLAG: --pod-cidr="" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382500 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382506 2562 flags.go:64] FLAG: --pod-manifest-path="" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382509 2562 flags.go:64] FLAG: --pod-max-pids="-1" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382512 2562 flags.go:64] FLAG: --pods-per-core="0" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382516 2562 flags.go:64] FLAG: --port="10250" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382519 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382522 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fd064c12c8abf80a" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382525 2562 flags.go:64] FLAG: --qos-reserved="" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382528 2562 flags.go:64] FLAG: --read-only-port="10255" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382531 2562 flags.go:64] FLAG: --register-node="true" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382533 2562 flags.go:64] FLAG: --register-with-taints="" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382537 2562 flags.go:64] FLAG: --registry-burst="10" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382540 2562 flags.go:64] FLAG: --registry-qps="5" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382543 2562 flags.go:64] FLAG: --reserved-cpus="" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382546 2562 flags.go:64] FLAG: --reserved-memory="" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382549 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382553 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 11 08:46:33.390113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382555 2562 flags.go:64] FLAG: --rotate-certificates="false" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382558 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382561 2562 flags.go:64] FLAG: --runonce="false" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382565 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382570 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382574 2562 flags.go:64] FLAG: --seccomp-default="false" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382579 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382584 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382587 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382591 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382594 2562 flags.go:64] FLAG: --storage-driver-password="root" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382597 2562 flags.go:64] FLAG: --storage-driver-secure="false" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382600 2562 flags.go:64] FLAG: --storage-driver-table="stats" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382603 2562 flags.go:64] FLAG: --storage-driver-user="root" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382605 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382608 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382612 2562 flags.go:64] FLAG: --system-cgroups="" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382615 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382620 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382623 2562 flags.go:64] FLAG: --tls-cert-file="" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382626 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382629 2562 flags.go:64] FLAG: --tls-min-version="" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382632 2562 flags.go:64] FLAG: --tls-private-key-file="" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382635 2562 flags.go:64] FLAG: --topology-manager-policy="none" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382638 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 11 08:46:33.390761 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382640 2562 flags.go:64] FLAG: --topology-manager-scope="container" Sep 11 08:46:33.391390 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382643 2562 flags.go:64] FLAG: --v="2" Sep 11 08:46:33.391390 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382648 2562 flags.go:64] FLAG: --version="false" Sep 11 08:46:33.391390 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382654 2562 flags.go:64] FLAG: --vmodule="" Sep 11 08:46:33.391390 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382660 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 11 08:46:33.391390 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.382665 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 11 08:46:33.391390 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382780 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 08:46:33.391390 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382785 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 08:46:33.391390 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382788 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 08:46:33.391390 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382790 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 08:46:33.391390 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382793 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 08:46:33.391390 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382797 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 08:46:33.391390 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382799 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 08:46:33.391390 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382802 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 08:46:33.391390 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382806 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 08:46:33.391390 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382810 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 08:46:33.391390 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382816 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 08:46:33.391390 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382820 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 08:46:33.391390 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382824 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 08:46:33.391390 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382827 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 08:46:33.391390 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382830 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 08:46:33.391390 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382833 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 08:46:33.391902 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382835 2562 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 08:46:33.391902 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382838 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 08:46:33.391902 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382841 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 08:46:33.391902 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382844 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 08:46:33.391902 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382846 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 08:46:33.391902 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382849 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 08:46:33.391902 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382852 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 08:46:33.391902 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382856 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 08:46:33.391902 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382859 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 08:46:33.391902 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382863 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 08:46:33.391902 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382865 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 08:46:33.391902 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382869 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 08:46:33.391902 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382872 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 08:46:33.391902 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382874 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 08:46:33.391902 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382877 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 08:46:33.391902 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382880 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 08:46:33.391902 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382882 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 08:46:33.391902 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382885 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 08:46:33.391902 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382890 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 08:46:33.392396 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382894 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 08:46:33.392396 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382898 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 08:46:33.392396 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382902 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 08:46:33.392396 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382906 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 08:46:33.392396 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382909 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 08:46:33.392396 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382912 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 08:46:33.392396 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382914 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 08:46:33.392396 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382918 2562 feature_gate.go:328] unrecognized feature gate: Example Sep 11 08:46:33.392396 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382920 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 08:46:33.392396 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382923 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 08:46:33.392396 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382926 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 08:46:33.392396 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382930 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 08:46:33.392396 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382934 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 08:46:33.392396 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382936 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 08:46:33.392396 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382939 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 08:46:33.392396 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382942 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 08:46:33.392396 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382945 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 08:46:33.392396 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382948 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 08:46:33.392396 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382950 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 08:46:33.392396 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382953 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 08:46:33.392890 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382955 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 08:46:33.392890 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382959 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 08:46:33.392890 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382963 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 08:46:33.392890 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382965 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 08:46:33.392890 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382971 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 08:46:33.392890 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382975 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 08:46:33.392890 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382980 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 08:46:33.392890 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382984 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 08:46:33.392890 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382987 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 08:46:33.392890 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382990 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 08:46:33.392890 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382993 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 08:46:33.392890 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382995 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 08:46:33.392890 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.382998 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 08:46:33.392890 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383000 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 08:46:33.392890 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383003 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 08:46:33.392890 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383006 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 08:46:33.392890 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383008 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 08:46:33.392890 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383011 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 08:46:33.392890 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383013 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 08:46:33.393365 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383017 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 08:46:33.393365 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383020 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 08:46:33.393365 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383022 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 08:46:33.393365 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383025 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 08:46:33.393365 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383028 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 08:46:33.393365 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383030 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 08:46:33.393365 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383033 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 08:46:33.393365 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383035 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 08:46:33.393365 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383038 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 08:46:33.393365 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383041 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 08:46:33.393365 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383044 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 08:46:33.393365 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383047 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 08:46:33.393365 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383051 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 08:46:33.393365 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383055 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 08:46:33.393365 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383060 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 08:46:33.393365 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383064 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 08:46:33.393365 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383067 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 08:46:33.393365 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383072 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 08:46:33.393365 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383074 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 08:46:33.393365 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383076 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 08:46:33.393857 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383079 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 08:46:33.393857 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383081 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 08:46:33.393857 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383084 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 08:46:33.393857 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383086 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 08:46:33.393857 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.383089 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 08:46:33.393857 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.383094 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 11 08:46:33.393857 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.390427 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 11 08:46:33.393857 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.390441 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 11 08:46:33.393857 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.391236 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 11 08:46:33.393857 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.391253 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 11 08:46:33.393857 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391337 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 08:46:33.393857 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391342 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 08:46:33.393857 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391346 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 08:46:33.393857 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391349 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 08:46:33.393857 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391352 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 08:46:33.394266 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391355 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 08:46:33.394266 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391358 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 08:46:33.394266 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391361 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 08:46:33.394266 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391364 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 08:46:33.394266 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391367 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 08:46:33.394266 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391369 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 08:46:33.394266 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391372 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 08:46:33.394266 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391392 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 08:46:33.394266 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391395 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 08:46:33.394266 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391398 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 08:46:33.394266 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391400 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 08:46:33.394266 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391403 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 08:46:33.394266 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391406 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 08:46:33.394266 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391409 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 08:46:33.394266 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391411 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 08:46:33.394266 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391414 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 08:46:33.394266 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391418 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 08:46:33.394266 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391420 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 08:46:33.394266 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391423 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 08:46:33.394266 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391425 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 08:46:33.394768 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391428 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 08:46:33.394768 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391431 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 08:46:33.394768 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391433 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 08:46:33.394768 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391436 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 08:46:33.394768 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391438 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 08:46:33.394768 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391441 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 08:46:33.394768 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391444 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 08:46:33.394768 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391448 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 08:46:33.394768 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391451 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 08:46:33.394768 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391454 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 08:46:33.394768 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391471 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 08:46:33.394768 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391475 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 08:46:33.394768 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391478 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 08:46:33.394768 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391481 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 08:46:33.394768 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391483 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 08:46:33.394768 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391486 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 08:46:33.394768 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391488 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 08:46:33.394768 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391491 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 08:46:33.394768 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391494 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 08:46:33.395248 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391496 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 08:46:33.395248 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391506 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 08:46:33.395248 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391508 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 08:46:33.395248 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391511 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 08:46:33.395248 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391514 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 08:46:33.395248 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391516 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 08:46:33.395248 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391519 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 08:46:33.395248 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391521 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 08:46:33.395248 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391524 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 08:46:33.395248 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391527 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 08:46:33.395248 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391529 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 08:46:33.395248 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391532 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 08:46:33.395248 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391534 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 08:46:33.395248 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391537 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 08:46:33.395248 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391540 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 08:46:33.395248 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391542 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 08:46:33.395248 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391545 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 08:46:33.395248 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391547 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 08:46:33.395248 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391549 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 08:46:33.395775 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391552 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 08:46:33.395775 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391554 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 08:46:33.395775 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391557 2562 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 08:46:33.395775 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391559 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 08:46:33.395775 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391562 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 08:46:33.395775 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391564 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 08:46:33.395775 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391567 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 08:46:33.395775 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391570 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 08:46:33.395775 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391572 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 08:46:33.395775 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391574 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 08:46:33.395775 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391577 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 08:46:33.395775 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391579 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 08:46:33.395775 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391582 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 08:46:33.395775 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391584 2562 feature_gate.go:328] unrecognized feature gate: Example Sep 11 08:46:33.395775 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391587 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 08:46:33.395775 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391595 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 08:46:33.395775 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391598 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 08:46:33.395775 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391601 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 08:46:33.395775 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391603 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 08:46:33.395775 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391606 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 08:46:33.397057 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391608 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 08:46:33.397057 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391611 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 08:46:33.397057 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391613 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 08:46:33.397057 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391616 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 08:46:33.397057 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391618 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 08:46:33.397057 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391621 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 08:46:33.397057 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391623 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 08:46:33.397057 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391626 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 08:46:33.397057 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391630 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 08:46:33.397057 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391634 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 08:46:33.397057 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391638 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 08:46:33.397057 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391641 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 08:46:33.397057 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391644 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 08:46:33.397057 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391647 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 08:46:33.397057 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391650 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 08:46:33.397057 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391653 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 08:46:33.397485 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.391658 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 11 08:46:33.397485 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391821 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 08:46:33.397485 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391827 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 08:46:33.397485 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391832 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 08:46:33.397485 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391836 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 08:46:33.397485 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391841 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 08:46:33.397485 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391846 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 08:46:33.397485 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391850 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 08:46:33.397485 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391853 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 08:46:33.397485 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391856 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 08:46:33.397485 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391860 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 08:46:33.397485 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391863 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 08:46:33.397485 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391872 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 08:46:33.397485 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391875 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 08:46:33.397485 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391878 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 08:46:33.397864 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391881 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 08:46:33.397864 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391883 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 08:46:33.397864 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391886 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 08:46:33.397864 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391889 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 08:46:33.397864 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391891 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 08:46:33.397864 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391894 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 08:46:33.397864 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391896 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 08:46:33.397864 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391899 2562 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 08:46:33.397864 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391901 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 08:46:33.397864 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391904 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 08:46:33.397864 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391907 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 08:46:33.397864 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391909 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 08:46:33.397864 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391912 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 08:46:33.397864 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391915 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 08:46:33.397864 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391917 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 08:46:33.397864 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391920 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 08:46:33.397864 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391922 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 08:46:33.397864 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391925 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 08:46:33.397864 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391927 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 08:46:33.398556 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391931 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 08:46:33.398556 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391935 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 08:46:33.398556 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391938 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 08:46:33.398556 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391941 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 08:46:33.398556 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391943 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 08:46:33.398556 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391946 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 08:46:33.398556 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391949 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 08:46:33.398556 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391952 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 08:46:33.398556 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391956 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 08:46:33.398556 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391958 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 08:46:33.398556 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391961 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 08:46:33.398556 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391970 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 08:46:33.398556 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391973 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 08:46:33.398556 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391976 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 08:46:33.398556 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391978 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 08:46:33.398556 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391981 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 08:46:33.398556 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391983 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 08:46:33.398556 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391986 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 08:46:33.398556 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391988 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 08:46:33.398556 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391991 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 08:46:33.399073 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391993 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 08:46:33.399073 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391995 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 08:46:33.399073 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.391998 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 08:46:33.399073 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392000 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 08:46:33.399073 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392003 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 08:46:33.399073 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392005 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 08:46:33.399073 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392008 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 08:46:33.399073 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392010 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 08:46:33.399073 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392012 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 08:46:33.399073 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392015 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 08:46:33.399073 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392017 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 08:46:33.399073 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392020 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 08:46:33.399073 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392022 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 08:46:33.399073 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392025 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 08:46:33.399073 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392027 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 08:46:33.399073 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392031 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 08:46:33.399073 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392035 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 08:46:33.399073 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392037 2562 feature_gate.go:328] unrecognized feature gate: Example Sep 11 08:46:33.399073 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392040 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 08:46:33.399571 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392043 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 08:46:33.399571 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392045 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 08:46:33.399571 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392048 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 08:46:33.399571 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392050 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 08:46:33.399571 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392053 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 08:46:33.399571 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392062 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 08:46:33.399571 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392065 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 08:46:33.399571 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392067 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 08:46:33.399571 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392070 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 08:46:33.399571 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392072 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 08:46:33.399571 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392075 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 08:46:33.399571 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392077 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 08:46:33.399571 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392080 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 08:46:33.399571 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392082 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 08:46:33.399571 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392085 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 08:46:33.399571 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392088 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 08:46:33.399571 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392091 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 08:46:33.399571 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392094 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 08:46:33.399571 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392096 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 08:46:33.399571 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392099 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 08:46:33.400085 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392101 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 08:46:33.400085 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392104 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 08:46:33.400085 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392107 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 08:46:33.400085 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392109 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 08:46:33.400085 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392111 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 08:46:33.400085 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392114 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 08:46:33.400085 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:33.392117 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 08:46:33.400085 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.392121 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 11 08:46:33.400085 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.392219 2562 server.go:962] "Client rotation is on, will bootstrap in background" Sep 11 08:46:33.400085 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.395185 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 11 08:46:33.400085 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.395959 2562 server.go:1019] "Starting client certificate rotation" Sep 11 08:46:33.400085 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.396087 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 11 08:46:33.400085 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.396119 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 11 08:46:33.403200 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.403182 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 11 08:46:33.406909 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.406889 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 11 08:46:33.417242 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.417226 2562 log.go:25] "Validated CRI v1 runtime API" Sep 11 08:46:33.422228 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.422214 2562 log.go:25] "Validated CRI v1 image API" Sep 11 08:46:33.423107 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.423091 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 11 08:46:33.430388 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.430363 2562 fs.go:135] Filesystem UUIDs: map[29b3d7e1-d4dc-49c0-abb8-c3a55ec9c33b:/dev/nvme0n1p3 505b09a9-0a43-429e-a790-75ca1f54be1e:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Sep 11 08:46:33.430467 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.430383 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 11 08:46:33.435946 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.435927 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 11 08:46:33.436061 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.435541 2562 manager.go:216] Machine: {Timestamp:2026-09-11 08:46:33.434436221 +0000 UTC m=+0.334334510 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098388 MemoryCapacity:33164472320 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29ac27f05abccbe9ca4dfd7878c052 SystemUUID:ec29ac27-f05a-bccb-e9ca-4dfd7878c052 BootID:f6661be8-99b6-48ee-a5d7-cc5dbe332dac Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:73:c4:88:b9:6b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:73:c4:88:b9:6b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:6d:42:e3:56:66 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164472320 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 11 08:46:33.436061 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.436059 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 11 08:46:33.436259 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.436198 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 11 08:46:33.438989 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.438968 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 11 08:46:33.439120 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.438991 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-169.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 11 08:46:33.439874 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.439864 2562 topology_manager.go:138] "Creating topology manager with none policy" Sep 11 08:46:33.439913 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.439877 2562 container_manager_linux.go:309] "Creating device plugin manager" Sep 11 08:46:33.439913 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.439885 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 11 08:46:33.440613 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.440603 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 11 08:46:33.441723 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.441714 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 11 08:46:33.445695 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.445684 2562 state_mem.go:36] "Initialized new in-memory state store" Sep 11 08:46:33.445818 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.445808 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 11 08:46:33.445849 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.445836 2562 kubelet.go:486] "Attempting to sync node with API server" Sep 11 08:46:33.445884 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.445859 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 11 08:46:33.445936 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.445928 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 11 08:46:33.445964 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.445957 2562 kubelet.go:398] "Adding apiserver pod source" Sep 11 08:46:33.445992 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.445965 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 11 08:46:33.446444 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.446430 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 11 08:46:33.446503 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.446468 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 11 08:46:33.449445 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.449423 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 11 08:46:33.451867 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.451850 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 11 08:46:33.451925 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.451883 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 11 08:46:33.454474 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.454450 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 11 08:46:33.454474 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.454476 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 11 08:46:33.454594 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.454486 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 11 08:46:33.454594 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.454495 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 11 08:46:33.454594 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.454514 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 11 08:46:33.454594 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.454522 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 11 08:46:33.454594 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.454541 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 11 08:46:33.454594 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.454551 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 11 08:46:33.454849 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.454627 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 11 08:46:33.454849 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.454638 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 11 08:46:33.454849 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.454653 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 11 08:46:33.454849 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.454664 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 11 08:46:33.455509 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.455495 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 11 08:46:33.455639 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.455629 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 11 08:46:33.458587 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.458568 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-169.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 11 08:46:33.459191 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:33.459168 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-169.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 11 08:46:33.459231 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:33.459168 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 11 08:46:33.459277 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.459262 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5l4ct" Sep 11 08:46:33.459415 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.459402 2562 server.go:1268] "Started kubelet" Sep 11 08:46:33.459476 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.459456 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 11 08:46:33.459548 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.459502 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 11 08:46:33.459589 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.459571 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 11 08:46:33.460264 ip-10-0-141-169 systemd[1]: Started Kubernetes Kubelet. Sep 11 08:46:33.461326 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.461210 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 11 08:46:33.462332 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.462317 2562 server.go:310] "Adding debug handlers to kubelet server" Sep 11 08:46:33.463734 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:33.463717 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 11 08:46:33.465817 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:33.464972 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-169.ec2.internal.18d437e175f9605d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-169.ec2.internal,UID:ip-10-0-141-169.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-169.ec2.internal,},FirstTimestamp:2026-09-11 08:46:33.459376221 +0000 UTC m=+0.359274510,LastTimestamp:2026-09-11 08:46:33.459376221 +0000 UTC m=+0.359274510,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-169.ec2.internal,}" Sep 11 08:46:33.466725 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.466709 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 11 08:46:33.467164 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.467147 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 11 08:46:33.467251 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.467226 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5l4ct" Sep 11 08:46:33.467320 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:33.467269 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-169.ec2.internal\" not found" Sep 11 08:46:33.467856 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.467834 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 11 08:46:33.467939 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.467840 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 11 08:46:33.467939 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.467878 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 11 08:46:33.468042 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.468020 2562 factory.go:55] Registering systemd factory Sep 11 08:46:33.468042 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.468039 2562 factory.go:223] Registration of the systemd container factory successfully Sep 11 08:46:33.468139 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.468091 2562 reconstruct.go:97] "Volume reconstruction finished" Sep 11 08:46:33.468139 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.468102 2562 reconciler.go:29] "Reconciler: start to sync state" Sep 11 08:46:33.468243 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.468230 2562 factory.go:153] Registering CRI-O factory Sep 11 08:46:33.468278 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.468245 2562 factory.go:223] Registration of the crio container factory successfully Sep 11 08:46:33.468322 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.468290 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 11 08:46:33.468375 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.468327 2562 factory.go:103] Registering Raw factory Sep 11 08:46:33.468375 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.468348 2562 manager.go:1196] Started watching for new ooms in manager Sep 11 08:46:33.468779 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.468768 2562 manager.go:349] Starting recovery of all containers Sep 11 08:46:33.476992 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.476961 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 08:46:33.480520 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:33.480476 2562 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-169.ec2.internal\" not found" node="ip-10-0-141-169.ec2.internal" Sep 11 08:46:33.482092 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.482077 2562 manager.go:354] Recovery completed Sep 11 08:46:33.487178 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.487162 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 08:46:33.489454 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.489440 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-169.ec2.internal" event="NodeHasSufficientMemory" Sep 11 08:46:33.489537 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.489466 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-169.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 08:46:33.489537 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.489477 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-169.ec2.internal" event="NodeHasSufficientPID" Sep 11 08:46:33.490155 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.490141 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 11 08:46:33.490155 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.490155 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 11 08:46:33.490258 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.490173 2562 state_mem.go:36] "Initialized new in-memory state store" Sep 11 08:46:33.490258 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.490145 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 08:46:33.491213 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.491192 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-169.ec2.internal" event="NodeHasSufficientMemory" Sep 11 08:46:33.491275 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.491221 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-169.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 08:46:33.491275 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.491234 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-169.ec2.internal" event="NodeHasSufficientPID" Sep 11 08:46:33.491275 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.491258 2562 policy_none.go:49] "None policy: Start" Sep 11 08:46:33.491275 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.491270 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 11 08:46:33.491420 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.491282 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 11 08:46:33.492367 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.492356 2562 policy_none.go:47] "Start" Sep 11 08:46:33.565023 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.565001 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 11 08:46:33.567125 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.566084 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 11 08:46:33.567125 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.566135 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 11 08:46:33.567125 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.566160 2562 kubelet.go:2448] "Starting kubelet main sync loop" Sep 11 08:46:33.567125 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:33.566231 2562 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Sep 11 08:46:33.567425 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:33.567405 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-169.ec2.internal\" not found" Sep 11 08:46:33.568697 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.568680 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 08:46:33.658830 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.658814 2562 manager.go:337] "Starting Device Plugin manager" Sep 11 08:46:33.658918 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:33.658848 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 11 08:46:33.658918 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.658857 2562 server.go:87] "Starting device plugin registration server" Sep 11 08:46:33.659124 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.659105 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 11 08:46:33.659213 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.659118 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 11 08:46:33.659213 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.659195 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 11 08:46:33.659372 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.659357 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 11 08:46:33.659372 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.659370 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 11 08:46:33.659925 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:33.659907 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 11 08:46:33.659981 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:33.659952 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-169.ec2.internal\" not found" Sep 11 08:46:33.666589 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.666573 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-169.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-169.ec2.internal"] Sep 11 08:46:33.666677 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.666633 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 08:46:33.668080 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.668066 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-169.ec2.internal" event="NodeHasSufficientMemory" Sep 11 08:46:33.668163 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.668096 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-169.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 08:46:33.668163 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.668111 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-169.ec2.internal" event="NodeHasSufficientPID" Sep 11 08:46:33.669187 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.669170 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/40c4a495594d5730f78ee6e94869d17f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-169.ec2.internal\" (UID: \"40c4a495594d5730f78ee6e94869d17f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-169.ec2.internal" Sep 11 08:46:33.669250 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.669193 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40c4a495594d5730f78ee6e94869d17f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-169.ec2.internal\" (UID: \"40c4a495594d5730f78ee6e94869d17f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-169.ec2.internal" Sep 11 08:46:33.670265 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.670251 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 08:46:33.670387 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.670369 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-169.ec2.internal" Sep 11 08:46:33.670437 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.670395 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 08:46:33.670992 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.670974 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-169.ec2.internal" event="NodeHasSufficientMemory" Sep 11 08:46:33.671059 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.671001 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-169.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 08:46:33.671059 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.670981 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-169.ec2.internal" event="NodeHasSufficientMemory" Sep 11 08:46:33.671059 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.671032 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-169.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 08:46:33.671059 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.671043 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-169.ec2.internal" event="NodeHasSufficientPID" Sep 11 08:46:33.671059 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.671012 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-169.ec2.internal" event="NodeHasSufficientPID" Sep 11 08:46:33.673178 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.673163 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-169.ec2.internal" Sep 11 08:46:33.673253 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.673190 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 08:46:33.673955 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.673941 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-169.ec2.internal" event="NodeHasSufficientMemory" Sep 11 08:46:33.674030 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.673970 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-169.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 08:46:33.674030 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.673985 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-169.ec2.internal" event="NodeHasSufficientPID" Sep 11 08:46:33.695886 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:33.695864 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-169.ec2.internal\" not found" node="ip-10-0-141-169.ec2.internal" Sep 11 08:46:33.700125 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:33.700110 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-169.ec2.internal\" not found" node="ip-10-0-141-169.ec2.internal" Sep 11 08:46:33.759750 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.759727 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 08:46:33.760475 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.760462 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-169.ec2.internal" event="NodeHasSufficientMemory" Sep 11 08:46:33.760543 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.760485 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-169.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 08:46:33.760543 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.760497 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-169.ec2.internal" event="NodeHasSufficientPID" Sep 11 08:46:33.760543 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.760528 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-169.ec2.internal" Sep 11 08:46:33.769764 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.769749 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-169.ec2.internal" Sep 11 08:46:33.769809 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:33.769766 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-169.ec2.internal\": node \"ip-10-0-141-169.ec2.internal\" not found" Sep 11 08:46:33.769901 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.769883 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/40c4a495594d5730f78ee6e94869d17f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-169.ec2.internal\" (UID: \"40c4a495594d5730f78ee6e94869d17f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-169.ec2.internal" Sep 11 08:46:33.769936 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.769910 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40c4a495594d5730f78ee6e94869d17f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-169.ec2.internal\" (UID: \"40c4a495594d5730f78ee6e94869d17f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-169.ec2.internal" Sep 11 08:46:33.769936 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.769928 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/458a6d26c1086ff6914f03deab3d15f2-config\") pod \"kube-apiserver-proxy-ip-10-0-141-169.ec2.internal\" (UID: \"458a6d26c1086ff6914f03deab3d15f2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-169.ec2.internal" Sep 11 08:46:33.770012 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.769977 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/40c4a495594d5730f78ee6e94869d17f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-169.ec2.internal\" (UID: \"40c4a495594d5730f78ee6e94869d17f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-169.ec2.internal" Sep 11 08:46:33.770012 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.769987 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40c4a495594d5730f78ee6e94869d17f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-169.ec2.internal\" (UID: \"40c4a495594d5730f78ee6e94869d17f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-169.ec2.internal" Sep 11 08:46:33.785348 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:33.785331 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-169.ec2.internal\" not found" Sep 11 08:46:33.870451 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.870420 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/458a6d26c1086ff6914f03deab3d15f2-config\") pod \"kube-apiserver-proxy-ip-10-0-141-169.ec2.internal\" (UID: \"458a6d26c1086ff6914f03deab3d15f2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-169.ec2.internal" Sep 11 08:46:33.870536 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.870465 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/458a6d26c1086ff6914f03deab3d15f2-config\") pod \"kube-apiserver-proxy-ip-10-0-141-169.ec2.internal\" (UID: \"458a6d26c1086ff6914f03deab3d15f2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-169.ec2.internal" Sep 11 08:46:33.885550 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:33.885512 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-169.ec2.internal\" not found" Sep 11 08:46:33.986224 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:33.986196 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-169.ec2.internal\" not found" Sep 11 08:46:33.998294 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:33.998272 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-169.ec2.internal" Sep 11 08:46:34.004802 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.004785 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-169.ec2.internal" Sep 11 08:46:34.087029 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:34.086881 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-169.ec2.internal\" not found" Sep 11 08:46:34.187389 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:34.187341 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-169.ec2.internal\" not found" Sep 11 08:46:34.287830 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:34.287811 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-169.ec2.internal\" not found" Sep 11 08:46:34.348565 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.348544 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 08:46:34.368050 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.368031 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-169.ec2.internal" Sep 11 08:46:34.379948 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.379928 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 11 08:46:34.381721 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.381707 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-169.ec2.internal" Sep 11 08:46:34.391085 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.391068 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 11 08:46:34.395529 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.395505 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 11 08:46:34.395661 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.395641 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 11 08:46:34.395718 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.395665 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 11 08:46:34.395718 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.395670 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 11 08:46:34.446991 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.446934 2562 apiserver.go:52] "Watching apiserver" Sep 11 08:46:34.464488 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.464467 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 11 08:46:34.464847 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.464823 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-tgx4q","kube-system/global-pull-secret-syncer-tldcl","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr","openshift-cluster-node-tuning-operator/tuned-nxhhd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-169.ec2.internal","openshift-multus/network-metrics-daemon-97r66","openshift-ovn-kubernetes/ovnkube-node-n6jlz","kube-system/konnectivity-agent-4smrv","kube-system/kube-apiserver-proxy-ip-10-0-141-169.ec2.internal","openshift-dns/node-resolver-cbhgc","openshift-image-registry/node-ca-9gfk6","openshift-multus/multus-additional-cni-plugins-wpkr7","openshift-multus/multus-x9gk7","openshift-network-diagnostics/network-check-target-q7bqv"] Sep 11 08:46:34.467267 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.467239 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 11 08:46:34.467475 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.467457 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.470091 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.470036 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 11 08:46:34.470091 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.470056 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 11 08:46:34.470091 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.470068 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 11 08:46:34.470091 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.470079 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 11 08:46:34.470352 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.470141 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-96w8j\"" Sep 11 08:46:34.470352 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.470062 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 11 08:46:34.470352 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.470171 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 11 08:46:34.470451 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.470402 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9gfk6" Sep 11 08:46:34.471839 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.471794 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 08:41:34 +0000 UTC" deadline="2028-03-20 21:37:28.6299848 +0000 UTC" Sep 11 08:46:34.471907 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.471839 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13356h50m54.158149661s" Sep 11 08:46:34.472075 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.472051 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 11 08:46:34.472302 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.472288 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 11 08:46:34.472387 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.472301 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5j6q6\"" Sep 11 08:46:34.472387 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.472321 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 11 08:46:34.473335 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.473317 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" Sep 11 08:46:34.473716 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.473697 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-systemd-units\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.473783 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.473724 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-host-run-netns\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.473783 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.473747 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-run-systemd\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.473783 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.473766 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-var-lib-openvswitch\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.473878 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.473790 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-run-openvswitch\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.473878 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.473814 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-node-log\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.473878 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.473829 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch8zm\" (UniqueName: \"kubernetes.io/projected/1e30076a-a074-40b2-8c43-2b2757da92d2-kube-api-access-ch8zm\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.473878 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.473844 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/395401b2-9748-42b5-b600-4fa1f7369b1e-serviceca\") pod \"node-ca-9gfk6\" (UID: \"395401b2-9748-42b5-b600-4fa1f7369b1e\") " pod="openshift-image-registry/node-ca-9gfk6" Sep 11 08:46:34.473878 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.473862 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-host-slash\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.474036 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.473895 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.474036 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.473911 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1e30076a-a074-40b2-8c43-2b2757da92d2-ovnkube-config\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.474036 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.473926 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6c69\" (UniqueName: \"kubernetes.io/projected/395401b2-9748-42b5-b600-4fa1f7369b1e-kube-api-access-t6c69\") pod \"node-ca-9gfk6\" (UID: \"395401b2-9748-42b5-b600-4fa1f7369b1e\") " pod="openshift-image-registry/node-ca-9gfk6" Sep 11 08:46:34.474036 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.473971 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-host-kubelet\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.474036 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.474002 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-host-cni-netd\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.474225 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.474043 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1e30076a-a074-40b2-8c43-2b2757da92d2-env-overrides\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.474225 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.474069 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1e30076a-a074-40b2-8c43-2b2757da92d2-ovn-node-metrics-cert\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.474225 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.474087 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1e30076a-a074-40b2-8c43-2b2757da92d2-ovnkube-script-lib\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.474225 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.474122 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/395401b2-9748-42b5-b600-4fa1f7369b1e-host\") pod \"node-ca-9gfk6\" (UID: \"395401b2-9748-42b5-b600-4fa1f7369b1e\") " pod="openshift-image-registry/node-ca-9gfk6" Sep 11 08:46:34.474225 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.474145 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-etc-openvswitch\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.474225 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.474163 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-run-ovn\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.474225 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.474178 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-log-socket\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.474225 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.474193 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-host-run-ovn-kubernetes\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.474225 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.474210 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-host-cni-bin\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.475217 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.475201 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 11 08:46:34.475275 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.475242 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-77ttq\"" Sep 11 08:46:34.475375 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.475361 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 11 08:46:34.475565 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.475547 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 11 08:46:34.476228 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.476213 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.477943 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.477925 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 11 08:46:34.478051 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.478038 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 11 08:46:34.478121 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.478087 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6hp2t\"" Sep 11 08:46:34.479078 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.479060 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:46:34.479167 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:34.479148 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7bqv" podUID="f24c0bea-46aa-432e-a51f-f25451143d07" Sep 11 08:46:34.480432 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.480415 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 11 08:46:34.481934 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.481917 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:46:34.482085 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:34.482001 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-97r66" podUID="faa7ad4f-7140-43cd-a195-d659bd3bee15" Sep 11 08:46:34.484478 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.484453 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4smrv" Sep 11 08:46:34.486303 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.486284 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-shj8d\"" Sep 11 08:46:34.486434 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.486348 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 11 08:46:34.486434 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.486417 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 11 08:46:34.487903 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.487882 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.490923 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.490907 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:34.491014 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:34.490991 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tldcl" podUID="e53c87e2-2721-4540-87c6-214593d333a4" Sep 11 08:46:34.492166 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.491999 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 11 08:46:34.492264 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.492179 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 11 08:46:34.492264 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.492237 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 11 08:46:34.492540 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.492365 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 11 08:46:34.492906 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.492888 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-x579w\"" Sep 11 08:46:34.495104 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.495083 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cbhgc" Sep 11 08:46:34.497483 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.497461 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6dk6p\"" Sep 11 08:46:34.497783 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.497763 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 11 08:46:34.497863 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.497826 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 11 08:46:34.498670 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.498655 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tgx4q" Sep 11 08:46:34.500702 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.500683 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-99s78\"" Sep 11 08:46:34.500702 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.500699 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 11 08:46:34.500849 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.500720 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 11 08:46:34.500889 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.500855 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 11 08:46:34.501331 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.501317 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wpkr7" Sep 11 08:46:34.503170 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.503148 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 11 08:46:34.503269 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.503209 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 11 08:46:34.503356 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.503338 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-j56cv\"" Sep 11 08:46:34.504219 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.504200 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hdw68" Sep 11 08:46:34.514658 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.514637 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hdw68" Sep 11 08:46:34.568727 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.568706 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 11 08:46:34.574642 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.574618 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/395401b2-9748-42b5-b600-4fa1f7369b1e-serviceca\") pod \"node-ca-9gfk6\" (UID: \"395401b2-9748-42b5-b600-4fa1f7369b1e\") " pod="openshift-image-registry/node-ca-9gfk6" Sep 11 08:46:34.574746 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.574660 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/731bdf3b-9131-4284-a232-93b9b73f3a89-multus-daemon-config\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.574811 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.574786 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c62c56b1-a68f-4a3f-a41b-23a2a08e148d-cnibin\") pod \"multus-additional-cni-plugins-wpkr7\" (UID: \"c62c56b1-a68f-4a3f-a41b-23a2a08e148d\") " pod="openshift-multus/multus-additional-cni-plugins-wpkr7" Sep 11 08:46:34.574867 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.574816 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.574867 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.574855 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1e30076a-a074-40b2-8c43-2b2757da92d2-ovnkube-config\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.574972 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.574887 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/45e03031-b380-4f4d-8259-09eaccf50bf1-registration-dir\") pod \"aws-ebs-csi-driver-node-k6cxr\" (UID: \"45e03031-b380-4f4d-8259-09eaccf50bf1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" Sep 11 08:46:34.574972 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.574907 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-host-kubelet\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.574972 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.574917 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.574972 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.574930 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/395401b2-9748-42b5-b600-4fa1f7369b1e-host\") pod \"node-ca-9gfk6\" (UID: \"395401b2-9748-42b5-b600-4fa1f7369b1e\") " pod="openshift-image-registry/node-ca-9gfk6" Sep 11 08:46:34.575166 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.574984 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-host-kubelet\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.575166 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.574992 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-etc-sysctl-d\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.575166 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575042 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/395401b2-9748-42b5-b600-4fa1f7369b1e-host\") pod \"node-ca-9gfk6\" (UID: \"395401b2-9748-42b5-b600-4fa1f7369b1e\") " pod="openshift-image-registry/node-ca-9gfk6" Sep 11 08:46:34.575166 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575087 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-host-var-lib-cni-bin\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.575166 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575118 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e53c87e2-2721-4540-87c6-214593d333a4-original-pull-secret\") pod \"global-pull-secret-syncer-tldcl\" (UID: \"e53c87e2-2721-4540-87c6-214593d333a4\") " pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:34.575166 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575160 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-run-ovn\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.575432 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575183 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-run\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.575432 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575206 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smc6d\" (UniqueName: \"kubernetes.io/projected/8901cf6c-2530-49cc-971f-431e24a0b88d-kube-api-access-smc6d\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.575432 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575212 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/395401b2-9748-42b5-b600-4fa1f7369b1e-serviceca\") pod \"node-ca-9gfk6\" (UID: \"395401b2-9748-42b5-b600-4fa1f7369b1e\") " pod="openshift-image-registry/node-ca-9gfk6" Sep 11 08:46:34.575432 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575209 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-run-ovn\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.575432 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575236 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-host-run-k8s-cni-cncf-io\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.575432 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575291 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-host-run-multus-certs\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.575432 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575339 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c62c56b1-a68f-4a3f-a41b-23a2a08e148d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wpkr7\" (UID: \"c62c56b1-a68f-4a3f-a41b-23a2a08e148d\") " pod="openshift-multus/multus-additional-cni-plugins-wpkr7" Sep 11 08:46:34.575432 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575368 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c62c56b1-a68f-4a3f-a41b-23a2a08e148d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wpkr7\" (UID: \"c62c56b1-a68f-4a3f-a41b-23a2a08e148d\") " pod="openshift-multus/multus-additional-cni-plugins-wpkr7" Sep 11 08:46:34.575432 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575415 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-var-lib-openvswitch\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.575792 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575454 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-node-log\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.575792 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575490 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-var-lib-openvswitch\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.575792 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575537 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-node-log\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.575792 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575546 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1e30076a-a074-40b2-8c43-2b2757da92d2-ovnkube-config\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.575792 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575564 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-host-slash\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.575792 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575521 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-host-slash\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.575792 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575629 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/45e03031-b380-4f4d-8259-09eaccf50bf1-socket-dir\") pod \"aws-ebs-csi-driver-node-k6cxr\" (UID: \"45e03031-b380-4f4d-8259-09eaccf50bf1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" Sep 11 08:46:34.575792 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575684 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-etc-kubernetes\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.575792 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575700 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-etc-systemd\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.575792 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575716 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-host-cni-netd\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.575792 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575730 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-etc-modprobe-d\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.575792 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575751 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-lib-modules\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.575792 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575772 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faa7ad4f-7140-43cd-a195-d659bd3bee15-metrics-certs\") pod \"network-metrics-daemon-97r66\" (UID: \"faa7ad4f-7140-43cd-a195-d659bd3bee15\") " pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:46:34.575792 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575787 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-host-cni-netd\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.576271 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575814 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf-agent-certs\") pod \"konnectivity-agent-4smrv\" (UID: \"20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf\") " pod="kube-system/konnectivity-agent-4smrv" Sep 11 08:46:34.576271 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575831 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4cd3f396-137e-4748-83b5-582ddc810965-hosts-file\") pod \"node-resolver-cbhgc\" (UID: \"4cd3f396-137e-4748-83b5-582ddc810965\") " pod="openshift-dns/node-resolver-cbhgc" Sep 11 08:46:34.576271 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575846 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2ck2\" (UniqueName: \"kubernetes.io/projected/4cd3f396-137e-4748-83b5-582ddc810965-kube-api-access-v2ck2\") pod \"node-resolver-cbhgc\" (UID: \"4cd3f396-137e-4748-83b5-582ddc810965\") " pod="openshift-dns/node-resolver-cbhgc" Sep 11 08:46:34.576271 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575871 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e53c87e2-2721-4540-87c6-214593d333a4-kubelet-config\") pod \"global-pull-secret-syncer-tldcl\" (UID: \"e53c87e2-2721-4540-87c6-214593d333a4\") " pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:34.576271 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575892 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-host-cni-bin\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.576271 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575908 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-var-lib-kubelet\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.576271 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575937 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf-konnectivity-ca\") pod \"konnectivity-agent-4smrv\" (UID: \"20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf\") " pod="kube-system/konnectivity-agent-4smrv" Sep 11 08:46:34.576271 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575963 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-host-cni-bin\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.576271 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.575980 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlkzg\" (UniqueName: \"kubernetes.io/projected/f24c0bea-46aa-432e-a51f-f25451143d07-kube-api-access-zlkzg\") pod \"network-check-target-q7bqv\" (UID: \"f24c0bea-46aa-432e-a51f-f25451143d07\") " pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:46:34.576271 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576000 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-host-var-lib-cni-multus\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.576271 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576023 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/caf9e2dc-28c5-4aa3-bfd0-b09bbfd49b0b-host-slash\") pod \"iptables-alerter-tgx4q\" (UID: \"caf9e2dc-28c5-4aa3-bfd0-b09bbfd49b0b\") " pod="openshift-network-operator/iptables-alerter-tgx4q" Sep 11 08:46:34.576271 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576064 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-systemd-units\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.576271 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576080 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-host-run-netns\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.576271 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576094 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-run-systemd\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.576271 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576112 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-run-openvswitch\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.576271 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576140 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-host-run-netns\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.576271 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576135 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6c69\" (UniqueName: \"kubernetes.io/projected/395401b2-9748-42b5-b600-4fa1f7369b1e-kube-api-access-t6c69\") pod \"node-ca-9gfk6\" (UID: \"395401b2-9748-42b5-b600-4fa1f7369b1e\") " pod="openshift-image-registry/node-ca-9gfk6" Sep 11 08:46:34.577139 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576157 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-systemd-units\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.577139 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576184 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-run-systemd\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.577139 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576182 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-run-openvswitch\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.577139 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576212 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/45e03031-b380-4f4d-8259-09eaccf50bf1-etc-selinux\") pod \"aws-ebs-csi-driver-node-k6cxr\" (UID: \"45e03031-b380-4f4d-8259-09eaccf50bf1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" Sep 11 08:46:34.577139 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576239 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gthqd\" (UniqueName: \"kubernetes.io/projected/caf9e2dc-28c5-4aa3-bfd0-b09bbfd49b0b-kube-api-access-gthqd\") pod \"iptables-alerter-tgx4q\" (UID: \"caf9e2dc-28c5-4aa3-bfd0-b09bbfd49b0b\") " pod="openshift-network-operator/iptables-alerter-tgx4q" Sep 11 08:46:34.577139 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576258 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c62c56b1-a68f-4a3f-a41b-23a2a08e148d-system-cni-dir\") pod \"multus-additional-cni-plugins-wpkr7\" (UID: \"c62c56b1-a68f-4a3f-a41b-23a2a08e148d\") " pod="openshift-multus/multus-additional-cni-plugins-wpkr7" Sep 11 08:46:34.577139 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576301 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-etc-sysconfig\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.577139 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576344 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-sys\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.577139 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576377 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8901cf6c-2530-49cc-971f-431e24a0b88d-tmp\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.577139 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576418 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rdkj\" (UniqueName: \"kubernetes.io/projected/faa7ad4f-7140-43cd-a195-d659bd3bee15-kube-api-access-4rdkj\") pod \"network-metrics-daemon-97r66\" (UID: \"faa7ad4f-7140-43cd-a195-d659bd3bee15\") " pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:46:34.577139 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576457 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-system-cni-dir\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.577139 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576471 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-multus-cni-dir\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.577139 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576489 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8ckl\" (UniqueName: \"kubernetes.io/projected/731bdf3b-9131-4284-a232-93b9b73f3a89-kube-api-access-d8ckl\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.577139 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576508 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1e30076a-a074-40b2-8c43-2b2757da92d2-env-overrides\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.577139 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576543 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1e30076a-a074-40b2-8c43-2b2757da92d2-ovn-node-metrics-cert\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.577139 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576623 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/45e03031-b380-4f4d-8259-09eaccf50bf1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k6cxr\" (UID: \"45e03031-b380-4f4d-8259-09eaccf50bf1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" Sep 11 08:46:34.577139 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576647 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-host\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.577784 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576665 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c62c56b1-a68f-4a3f-a41b-23a2a08e148d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wpkr7\" (UID: \"c62c56b1-a68f-4a3f-a41b-23a2a08e148d\") " pod="openshift-multus/multus-additional-cni-plugins-wpkr7" Sep 11 08:46:34.577784 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576765 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jxhc\" (UniqueName: \"kubernetes.io/projected/c62c56b1-a68f-4a3f-a41b-23a2a08e148d-kube-api-access-6jxhc\") pod \"multus-additional-cni-plugins-wpkr7\" (UID: \"c62c56b1-a68f-4a3f-a41b-23a2a08e148d\") " pod="openshift-multus/multus-additional-cni-plugins-wpkr7" Sep 11 08:46:34.577784 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576802 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-etc-openvswitch\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.577784 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576892 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/731bdf3b-9131-4284-a232-93b9b73f3a89-cni-binary-copy\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.577784 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576883 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-etc-openvswitch\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.577784 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576934 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e53c87e2-2721-4540-87c6-214593d333a4-dbus\") pod \"global-pull-secret-syncer-tldcl\" (UID: \"e53c87e2-2721-4540-87c6-214593d333a4\") " pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:34.577784 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576961 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-etc-kubernetes\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.577784 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576976 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/45e03031-b380-4f4d-8259-09eaccf50bf1-device-dir\") pod \"aws-ebs-csi-driver-node-k6cxr\" (UID: \"45e03031-b380-4f4d-8259-09eaccf50bf1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" Sep 11 08:46:34.577784 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.576995 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/45e03031-b380-4f4d-8259-09eaccf50bf1-sys-fs\") pod \"aws-ebs-csi-driver-node-k6cxr\" (UID: \"45e03031-b380-4f4d-8259-09eaccf50bf1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" Sep 11 08:46:34.577784 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.577030 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cd3f396-137e-4748-83b5-582ddc810965-tmp-dir\") pod \"node-resolver-cbhgc\" (UID: \"4cd3f396-137e-4748-83b5-582ddc810965\") " pod="openshift-dns/node-resolver-cbhgc" Sep 11 08:46:34.577784 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.577045 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-host-var-lib-kubelet\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.577784 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.577045 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1e30076a-a074-40b2-8c43-2b2757da92d2-env-overrides\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.577784 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.577061 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c62c56b1-a68f-4a3f-a41b-23a2a08e148d-cni-binary-copy\") pod \"multus-additional-cni-plugins-wpkr7\" (UID: \"c62c56b1-a68f-4a3f-a41b-23a2a08e148d\") " pod="openshift-multus/multus-additional-cni-plugins-wpkr7" Sep 11 08:46:34.577784 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.577096 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1e30076a-a074-40b2-8c43-2b2757da92d2-ovnkube-script-lib\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.577784 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.577073 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 11 08:46:34.577784 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.577125 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8901cf6c-2530-49cc-971f-431e24a0b88d-etc-tuned\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.577784 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.577146 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-multus-conf-dir\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.578398 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.577170 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/caf9e2dc-28c5-4aa3-bfd0-b09bbfd49b0b-iptables-alerter-script\") pod \"iptables-alerter-tgx4q\" (UID: \"caf9e2dc-28c5-4aa3-bfd0-b09bbfd49b0b\") " pod="openshift-network-operator/iptables-alerter-tgx4q" Sep 11 08:46:34.578398 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.577194 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-log-socket\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.578398 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.577273 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-host-run-ovn-kubernetes\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.578398 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.577281 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-log-socket\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.578398 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.577353 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e30076a-a074-40b2-8c43-2b2757da92d2-host-run-ovn-kubernetes\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.578398 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.577348 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2mpq\" (UniqueName: \"kubernetes.io/projected/45e03031-b380-4f4d-8259-09eaccf50bf1-kube-api-access-w2mpq\") pod \"aws-ebs-csi-driver-node-k6cxr\" (UID: \"45e03031-b380-4f4d-8259-09eaccf50bf1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" Sep 11 08:46:34.578398 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.577397 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-etc-sysctl-conf\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.578398 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.577419 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-cnibin\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.578398 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.577442 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-multus-socket-dir-parent\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.578398 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.577462 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-host-run-netns\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.578398 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.577477 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-hostroot\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.578398 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.577494 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch8zm\" (UniqueName: \"kubernetes.io/projected/1e30076a-a074-40b2-8c43-2b2757da92d2-kube-api-access-ch8zm\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.578398 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.577734 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1e30076a-a074-40b2-8c43-2b2757da92d2-ovnkube-script-lib\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.580367 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.580348 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1e30076a-a074-40b2-8c43-2b2757da92d2-ovn-node-metrics-cert\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.587627 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.587609 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6c69\" (UniqueName: \"kubernetes.io/projected/395401b2-9748-42b5-b600-4fa1f7369b1e-kube-api-access-t6c69\") pod \"node-ca-9gfk6\" (UID: \"395401b2-9748-42b5-b600-4fa1f7369b1e\") " pod="openshift-image-registry/node-ca-9gfk6" Sep 11 08:46:34.588084 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.588063 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch8zm\" (UniqueName: \"kubernetes.io/projected/1e30076a-a074-40b2-8c43-2b2757da92d2-kube-api-access-ch8zm\") pod \"ovnkube-node-n6jlz\" (UID: \"1e30076a-a074-40b2-8c43-2b2757da92d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.661420 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:34.661365 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40c4a495594d5730f78ee6e94869d17f.slice/crio-5a6803a755b02acd3815acea3ad6668103e91b76d74afe5b61b9f16270ec9a28 WatchSource:0}: Error finding container 5a6803a755b02acd3815acea3ad6668103e91b76d74afe5b61b9f16270ec9a28: Status 404 returned error can't find the container with id 5a6803a755b02acd3815acea3ad6668103e91b76d74afe5b61b9f16270ec9a28 Sep 11 08:46:34.661726 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:34.661706 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod458a6d26c1086ff6914f03deab3d15f2.slice/crio-3238f2bf35b3e2f00f27942faf7e1644be1c1f4e8be33514569970cb04dbfce8 WatchSource:0}: Error finding container 3238f2bf35b3e2f00f27942faf7e1644be1c1f4e8be33514569970cb04dbfce8: Status 404 returned error can't find the container with id 3238f2bf35b3e2f00f27942faf7e1644be1c1f4e8be33514569970cb04dbfce8 Sep 11 08:46:34.665821 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.665789 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 11 08:46:34.678014 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.677993 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2mpq\" (UniqueName: \"kubernetes.io/projected/45e03031-b380-4f4d-8259-09eaccf50bf1-kube-api-access-w2mpq\") pod \"aws-ebs-csi-driver-node-k6cxr\" (UID: \"45e03031-b380-4f4d-8259-09eaccf50bf1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" Sep 11 08:46:34.678107 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678021 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-etc-sysctl-conf\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.678107 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678037 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-cnibin\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.678107 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678053 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-multus-socket-dir-parent\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.678107 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678068 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-host-run-netns\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.678107 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678081 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-hostroot\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.678334 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678134 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/731bdf3b-9131-4284-a232-93b9b73f3a89-multus-daemon-config\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.678334 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678150 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-multus-socket-dir-parent\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.678334 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678151 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-host-run-netns\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.678334 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678160 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c62c56b1-a68f-4a3f-a41b-23a2a08e148d-cnibin\") pod \"multus-additional-cni-plugins-wpkr7\" (UID: \"c62c56b1-a68f-4a3f-a41b-23a2a08e148d\") " pod="openshift-multus/multus-additional-cni-plugins-wpkr7" Sep 11 08:46:34.678334 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678185 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-cnibin\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.678334 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678197 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c62c56b1-a68f-4a3f-a41b-23a2a08e148d-cnibin\") pod \"multus-additional-cni-plugins-wpkr7\" (UID: \"c62c56b1-a68f-4a3f-a41b-23a2a08e148d\") " pod="openshift-multus/multus-additional-cni-plugins-wpkr7" Sep 11 08:46:34.678334 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678184 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-etc-sysctl-conf\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.678334 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678197 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-hostroot\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.678334 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678213 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/45e03031-b380-4f4d-8259-09eaccf50bf1-registration-dir\") pod \"aws-ebs-csi-driver-node-k6cxr\" (UID: \"45e03031-b380-4f4d-8259-09eaccf50bf1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" Sep 11 08:46:34.678334 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678260 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/45e03031-b380-4f4d-8259-09eaccf50bf1-registration-dir\") pod \"aws-ebs-csi-driver-node-k6cxr\" (UID: \"45e03031-b380-4f4d-8259-09eaccf50bf1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" Sep 11 08:46:34.678334 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678267 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-etc-sysctl-d\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.678334 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678298 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-host-var-lib-cni-bin\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.678334 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678334 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e53c87e2-2721-4540-87c6-214593d333a4-original-pull-secret\") pod \"global-pull-secret-syncer-tldcl\" (UID: \"e53c87e2-2721-4540-87c6-214593d333a4\") " pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:34.678989 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678350 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-run\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.678989 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678370 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smc6d\" (UniqueName: \"kubernetes.io/projected/8901cf6c-2530-49cc-971f-431e24a0b88d-kube-api-access-smc6d\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.678989 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678389 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-etc-sysctl-d\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.678989 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678396 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-host-run-k8s-cni-cncf-io\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.678989 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678368 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-host-var-lib-cni-bin\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.678989 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678426 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-host-run-multus-certs\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.678989 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:34.678447 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 08:46:34.678989 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678453 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-host-run-k8s-cni-cncf-io\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.678989 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678463 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c62c56b1-a68f-4a3f-a41b-23a2a08e148d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wpkr7\" (UID: \"c62c56b1-a68f-4a3f-a41b-23a2a08e148d\") " pod="openshift-multus/multus-additional-cni-plugins-wpkr7" Sep 11 08:46:34.678989 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678469 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-host-run-multus-certs\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.678989 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678425 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-run\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.678989 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678505 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c62c56b1-a68f-4a3f-a41b-23a2a08e148d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wpkr7\" (UID: \"c62c56b1-a68f-4a3f-a41b-23a2a08e148d\") " pod="openshift-multus/multus-additional-cni-plugins-wpkr7" Sep 11 08:46:34.678989 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:34.678538 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e53c87e2-2721-4540-87c6-214593d333a4-original-pull-secret podName:e53c87e2-2721-4540-87c6-214593d333a4 nodeName:}" failed. No retries permitted until 2026-09-11 08:46:35.178499535 +0000 UTC m=+2.078397826 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e53c87e2-2721-4540-87c6-214593d333a4-original-pull-secret") pod "global-pull-secret-syncer-tldcl" (UID: "e53c87e2-2721-4540-87c6-214593d333a4") : object "kube-system"/"original-pull-secret" not registered Sep 11 08:46:34.678989 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678575 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/45e03031-b380-4f4d-8259-09eaccf50bf1-socket-dir\") pod \"aws-ebs-csi-driver-node-k6cxr\" (UID: \"45e03031-b380-4f4d-8259-09eaccf50bf1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" Sep 11 08:46:34.678989 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678603 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-etc-kubernetes\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.678989 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678617 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c62c56b1-a68f-4a3f-a41b-23a2a08e148d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wpkr7\" (UID: \"c62c56b1-a68f-4a3f-a41b-23a2a08e148d\") " pod="openshift-multus/multus-additional-cni-plugins-wpkr7" Sep 11 08:46:34.678989 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678629 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-etc-systemd\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.679967 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678683 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-etc-systemd\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.679967 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678681 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-etc-modprobe-d\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.679967 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678725 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-lib-modules\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.679967 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678729 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-etc-kubernetes\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.679967 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678739 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/45e03031-b380-4f4d-8259-09eaccf50bf1-socket-dir\") pod \"aws-ebs-csi-driver-node-k6cxr\" (UID: \"45e03031-b380-4f4d-8259-09eaccf50bf1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" Sep 11 08:46:34.679967 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678755 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faa7ad4f-7140-43cd-a195-d659bd3bee15-metrics-certs\") pod \"network-metrics-daemon-97r66\" (UID: \"faa7ad4f-7140-43cd-a195-d659bd3bee15\") " pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:46:34.679967 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678764 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-etc-modprobe-d\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.679967 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678780 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf-agent-certs\") pod \"konnectivity-agent-4smrv\" (UID: \"20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf\") " pod="kube-system/konnectivity-agent-4smrv" Sep 11 08:46:34.679967 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678796 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/731bdf3b-9131-4284-a232-93b9b73f3a89-multus-daemon-config\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.679967 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678804 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4cd3f396-137e-4748-83b5-582ddc810965-hosts-file\") pod \"node-resolver-cbhgc\" (UID: \"4cd3f396-137e-4748-83b5-582ddc810965\") " pod="openshift-dns/node-resolver-cbhgc" Sep 11 08:46:34.679967 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678815 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-lib-modules\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.679967 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678828 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2ck2\" (UniqueName: \"kubernetes.io/projected/4cd3f396-137e-4748-83b5-582ddc810965-kube-api-access-v2ck2\") pod \"node-resolver-cbhgc\" (UID: \"4cd3f396-137e-4748-83b5-582ddc810965\") " pod="openshift-dns/node-resolver-cbhgc" Sep 11 08:46:34.679967 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:34.678844 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 08:46:34.679967 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678853 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e53c87e2-2721-4540-87c6-214593d333a4-kubelet-config\") pod \"global-pull-secret-syncer-tldcl\" (UID: \"e53c87e2-2721-4540-87c6-214593d333a4\") " pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:34.679967 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678863 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4cd3f396-137e-4748-83b5-582ddc810965-hosts-file\") pod \"node-resolver-cbhgc\" (UID: \"4cd3f396-137e-4748-83b5-582ddc810965\") " pod="openshift-dns/node-resolver-cbhgc" Sep 11 08:46:34.679967 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678877 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-var-lib-kubelet\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.679967 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:34.678891 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faa7ad4f-7140-43cd-a195-d659bd3bee15-metrics-certs podName:faa7ad4f-7140-43cd-a195-d659bd3bee15 nodeName:}" failed. No retries permitted until 2026-09-11 08:46:35.178875247 +0000 UTC m=+2.078773530 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/faa7ad4f-7140-43cd-a195-d659bd3bee15-metrics-certs") pod "network-metrics-daemon-97r66" (UID: "faa7ad4f-7140-43cd-a195-d659bd3bee15") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 08:46:34.680944 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678922 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-var-lib-kubelet\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.680944 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678929 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf-konnectivity-ca\") pod \"konnectivity-agent-4smrv\" (UID: \"20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf\") " pod="kube-system/konnectivity-agent-4smrv" Sep 11 08:46:34.680944 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678948 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e53c87e2-2721-4540-87c6-214593d333a4-kubelet-config\") pod \"global-pull-secret-syncer-tldcl\" (UID: \"e53c87e2-2721-4540-87c6-214593d333a4\") " pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:34.680944 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678959 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlkzg\" (UniqueName: \"kubernetes.io/projected/f24c0bea-46aa-432e-a51f-f25451143d07-kube-api-access-zlkzg\") pod \"network-check-target-q7bqv\" (UID: \"f24c0bea-46aa-432e-a51f-f25451143d07\") " pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:46:34.680944 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.678984 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-host-var-lib-cni-multus\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.680944 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679010 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/caf9e2dc-28c5-4aa3-bfd0-b09bbfd49b0b-host-slash\") pod \"iptables-alerter-tgx4q\" (UID: \"caf9e2dc-28c5-4aa3-bfd0-b09bbfd49b0b\") " pod="openshift-network-operator/iptables-alerter-tgx4q" Sep 11 08:46:34.680944 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679038 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/45e03031-b380-4f4d-8259-09eaccf50bf1-etc-selinux\") pod \"aws-ebs-csi-driver-node-k6cxr\" (UID: \"45e03031-b380-4f4d-8259-09eaccf50bf1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" Sep 11 08:46:34.680944 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679066 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gthqd\" (UniqueName: \"kubernetes.io/projected/caf9e2dc-28c5-4aa3-bfd0-b09bbfd49b0b-kube-api-access-gthqd\") pod \"iptables-alerter-tgx4q\" (UID: \"caf9e2dc-28c5-4aa3-bfd0-b09bbfd49b0b\") " pod="openshift-network-operator/iptables-alerter-tgx4q" Sep 11 08:46:34.680944 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679082 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-host-var-lib-cni-multus\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.680944 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679091 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c62c56b1-a68f-4a3f-a41b-23a2a08e148d-system-cni-dir\") pod \"multus-additional-cni-plugins-wpkr7\" (UID: \"c62c56b1-a68f-4a3f-a41b-23a2a08e148d\") " pod="openshift-multus/multus-additional-cni-plugins-wpkr7" Sep 11 08:46:34.680944 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679095 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c62c56b1-a68f-4a3f-a41b-23a2a08e148d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wpkr7\" (UID: \"c62c56b1-a68f-4a3f-a41b-23a2a08e148d\") " pod="openshift-multus/multus-additional-cni-plugins-wpkr7" Sep 11 08:46:34.680944 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679117 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-etc-sysconfig\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.680944 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679138 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/caf9e2dc-28c5-4aa3-bfd0-b09bbfd49b0b-host-slash\") pod \"iptables-alerter-tgx4q\" (UID: \"caf9e2dc-28c5-4aa3-bfd0-b09bbfd49b0b\") " pod="openshift-network-operator/iptables-alerter-tgx4q" Sep 11 08:46:34.680944 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679154 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-sys\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.680944 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679168 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-etc-sysconfig\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.680944 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679180 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8901cf6c-2530-49cc-971f-431e24a0b88d-tmp\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.680944 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679190 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/45e03031-b380-4f4d-8259-09eaccf50bf1-etc-selinux\") pod \"aws-ebs-csi-driver-node-k6cxr\" (UID: \"45e03031-b380-4f4d-8259-09eaccf50bf1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" Sep 11 08:46:34.681995 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679179 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c62c56b1-a68f-4a3f-a41b-23a2a08e148d-system-cni-dir\") pod \"multus-additional-cni-plugins-wpkr7\" (UID: \"c62c56b1-a68f-4a3f-a41b-23a2a08e148d\") " pod="openshift-multus/multus-additional-cni-plugins-wpkr7" Sep 11 08:46:34.681995 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679215 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rdkj\" (UniqueName: \"kubernetes.io/projected/faa7ad4f-7140-43cd-a195-d659bd3bee15-kube-api-access-4rdkj\") pod \"network-metrics-daemon-97r66\" (UID: \"faa7ad4f-7140-43cd-a195-d659bd3bee15\") " pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:46:34.681995 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679217 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-sys\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.681995 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679246 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-system-cni-dir\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.681995 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679273 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-multus-cni-dir\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.681995 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679310 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8ckl\" (UniqueName: \"kubernetes.io/projected/731bdf3b-9131-4284-a232-93b9b73f3a89-kube-api-access-d8ckl\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.681995 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679342 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/45e03031-b380-4f4d-8259-09eaccf50bf1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k6cxr\" (UID: \"45e03031-b380-4f4d-8259-09eaccf50bf1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" Sep 11 08:46:34.681995 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679368 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-host\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.681995 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679417 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c62c56b1-a68f-4a3f-a41b-23a2a08e148d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wpkr7\" (UID: \"c62c56b1-a68f-4a3f-a41b-23a2a08e148d\") " pod="openshift-multus/multus-additional-cni-plugins-wpkr7" Sep 11 08:46:34.681995 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679377 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-system-cni-dir\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.681995 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679478 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf-konnectivity-ca\") pod \"konnectivity-agent-4smrv\" (UID: \"20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf\") " pod="kube-system/konnectivity-agent-4smrv" Sep 11 08:46:34.681995 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679474 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jxhc\" (UniqueName: \"kubernetes.io/projected/c62c56b1-a68f-4a3f-a41b-23a2a08e148d-kube-api-access-6jxhc\") pod \"multus-additional-cni-plugins-wpkr7\" (UID: \"c62c56b1-a68f-4a3f-a41b-23a2a08e148d\") " pod="openshift-multus/multus-additional-cni-plugins-wpkr7" Sep 11 08:46:34.681995 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679519 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/731bdf3b-9131-4284-a232-93b9b73f3a89-cni-binary-copy\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.681995 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679533 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/45e03031-b380-4f4d-8259-09eaccf50bf1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k6cxr\" (UID: \"45e03031-b380-4f4d-8259-09eaccf50bf1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" Sep 11 08:46:34.681995 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679545 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e53c87e2-2721-4540-87c6-214593d333a4-dbus\") pod \"global-pull-secret-syncer-tldcl\" (UID: \"e53c87e2-2721-4540-87c6-214593d333a4\") " pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:34.681995 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679491 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8901cf6c-2530-49cc-971f-431e24a0b88d-host\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.681995 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679574 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-etc-kubernetes\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.681995 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679578 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-multus-cni-dir\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.682621 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679598 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/45e03031-b380-4f4d-8259-09eaccf50bf1-device-dir\") pod \"aws-ebs-csi-driver-node-k6cxr\" (UID: \"45e03031-b380-4f4d-8259-09eaccf50bf1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" Sep 11 08:46:34.682621 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679629 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/45e03031-b380-4f4d-8259-09eaccf50bf1-sys-fs\") pod \"aws-ebs-csi-driver-node-k6cxr\" (UID: \"45e03031-b380-4f4d-8259-09eaccf50bf1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" Sep 11 08:46:34.682621 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679643 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e53c87e2-2721-4540-87c6-214593d333a4-dbus\") pod \"global-pull-secret-syncer-tldcl\" (UID: \"e53c87e2-2721-4540-87c6-214593d333a4\") " pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:34.682621 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679652 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cd3f396-137e-4748-83b5-582ddc810965-tmp-dir\") pod \"node-resolver-cbhgc\" (UID: \"4cd3f396-137e-4748-83b5-582ddc810965\") " pod="openshift-dns/node-resolver-cbhgc" Sep 11 08:46:34.682621 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679679 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-host-var-lib-kubelet\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.682621 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679692 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/45e03031-b380-4f4d-8259-09eaccf50bf1-device-dir\") pod \"aws-ebs-csi-driver-node-k6cxr\" (UID: \"45e03031-b380-4f4d-8259-09eaccf50bf1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" Sep 11 08:46:34.682621 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679705 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c62c56b1-a68f-4a3f-a41b-23a2a08e148d-cni-binary-copy\") pod \"multus-additional-cni-plugins-wpkr7\" (UID: \"c62c56b1-a68f-4a3f-a41b-23a2a08e148d\") " pod="openshift-multus/multus-additional-cni-plugins-wpkr7" Sep 11 08:46:34.682621 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679733 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8901cf6c-2530-49cc-971f-431e24a0b88d-etc-tuned\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.682621 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679732 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-etc-kubernetes\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.682621 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679790 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-host-var-lib-kubelet\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.682621 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.679797 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/45e03031-b380-4f4d-8259-09eaccf50bf1-sys-fs\") pod \"aws-ebs-csi-driver-node-k6cxr\" (UID: \"45e03031-b380-4f4d-8259-09eaccf50bf1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" Sep 11 08:46:34.682621 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.680032 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c62c56b1-a68f-4a3f-a41b-23a2a08e148d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wpkr7\" (UID: \"c62c56b1-a68f-4a3f-a41b-23a2a08e148d\") " pod="openshift-multus/multus-additional-cni-plugins-wpkr7" Sep 11 08:46:34.682621 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.680060 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cd3f396-137e-4748-83b5-582ddc810965-tmp-dir\") pod \"node-resolver-cbhgc\" (UID: \"4cd3f396-137e-4748-83b5-582ddc810965\") " pod="openshift-dns/node-resolver-cbhgc" Sep 11 08:46:34.682621 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.680120 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-multus-conf-dir\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.682621 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.680129 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/731bdf3b-9131-4284-a232-93b9b73f3a89-cni-binary-copy\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.682621 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.680150 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/caf9e2dc-28c5-4aa3-bfd0-b09bbfd49b0b-iptables-alerter-script\") pod \"iptables-alerter-tgx4q\" (UID: \"caf9e2dc-28c5-4aa3-bfd0-b09bbfd49b0b\") " pod="openshift-network-operator/iptables-alerter-tgx4q" Sep 11 08:46:34.682621 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.680195 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/731bdf3b-9131-4284-a232-93b9b73f3a89-multus-conf-dir\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.683113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.680301 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c62c56b1-a68f-4a3f-a41b-23a2a08e148d-cni-binary-copy\") pod \"multus-additional-cni-plugins-wpkr7\" (UID: \"c62c56b1-a68f-4a3f-a41b-23a2a08e148d\") " pod="openshift-multus/multus-additional-cni-plugins-wpkr7" Sep 11 08:46:34.683113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.680733 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/caf9e2dc-28c5-4aa3-bfd0-b09bbfd49b0b-iptables-alerter-script\") pod \"iptables-alerter-tgx4q\" (UID: \"caf9e2dc-28c5-4aa3-bfd0-b09bbfd49b0b\") " pod="openshift-network-operator/iptables-alerter-tgx4q" Sep 11 08:46:34.683113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.681601 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf-agent-certs\") pod \"konnectivity-agent-4smrv\" (UID: \"20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf\") " pod="kube-system/konnectivity-agent-4smrv" Sep 11 08:46:34.683113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.681626 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8901cf6c-2530-49cc-971f-431e24a0b88d-tmp\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.683113 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.682275 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8901cf6c-2530-49cc-971f-431e24a0b88d-etc-tuned\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.688595 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:34.688576 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 08:46:34.688718 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:34.688600 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 08:46:34.688718 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:34.688612 2562 projected.go:196] Error preparing data for projected volume kube-api-access-zlkzg for pod openshift-network-diagnostics/network-check-target-q7bqv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 08:46:34.688718 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:34.688682 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f24c0bea-46aa-432e-a51f-f25451143d07-kube-api-access-zlkzg podName:f24c0bea-46aa-432e-a51f-f25451143d07 nodeName:}" failed. No retries permitted until 2026-09-11 08:46:35.188663909 +0000 UTC m=+2.088562190 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zlkzg" (UniqueName: "kubernetes.io/projected/f24c0bea-46aa-432e-a51f-f25451143d07-kube-api-access-zlkzg") pod "network-check-target-q7bqv" (UID: "f24c0bea-46aa-432e-a51f-f25451143d07") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 08:46:34.690105 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.690077 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smc6d\" (UniqueName: \"kubernetes.io/projected/8901cf6c-2530-49cc-971f-431e24a0b88d-kube-api-access-smc6d\") pod \"tuned-nxhhd\" (UID: \"8901cf6c-2530-49cc-971f-431e24a0b88d\") " pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.690326 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.690301 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2mpq\" (UniqueName: \"kubernetes.io/projected/45e03031-b380-4f4d-8259-09eaccf50bf1-kube-api-access-w2mpq\") pod \"aws-ebs-csi-driver-node-k6cxr\" (UID: \"45e03031-b380-4f4d-8259-09eaccf50bf1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" Sep 11 08:46:34.691027 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.691003 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gthqd\" (UniqueName: \"kubernetes.io/projected/caf9e2dc-28c5-4aa3-bfd0-b09bbfd49b0b-kube-api-access-gthqd\") pod \"iptables-alerter-tgx4q\" (UID: \"caf9e2dc-28c5-4aa3-bfd0-b09bbfd49b0b\") " pod="openshift-network-operator/iptables-alerter-tgx4q" Sep 11 08:46:34.691115 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.691069 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2ck2\" (UniqueName: \"kubernetes.io/projected/4cd3f396-137e-4748-83b5-582ddc810965-kube-api-access-v2ck2\") pod \"node-resolver-cbhgc\" (UID: \"4cd3f396-137e-4748-83b5-582ddc810965\") " pod="openshift-dns/node-resolver-cbhgc" Sep 11 08:46:34.691432 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.691408 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jxhc\" (UniqueName: \"kubernetes.io/projected/c62c56b1-a68f-4a3f-a41b-23a2a08e148d-kube-api-access-6jxhc\") pod \"multus-additional-cni-plugins-wpkr7\" (UID: \"c62c56b1-a68f-4a3f-a41b-23a2a08e148d\") " pod="openshift-multus/multus-additional-cni-plugins-wpkr7" Sep 11 08:46:34.691689 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.691672 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8ckl\" (UniqueName: \"kubernetes.io/projected/731bdf3b-9131-4284-a232-93b9b73f3a89-kube-api-access-d8ckl\") pod \"multus-x9gk7\" (UID: \"731bdf3b-9131-4284-a232-93b9b73f3a89\") " pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.691764 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.691743 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rdkj\" (UniqueName: \"kubernetes.io/projected/faa7ad4f-7140-43cd-a195-d659bd3bee15-kube-api-access-4rdkj\") pod \"network-metrics-daemon-97r66\" (UID: \"faa7ad4f-7140-43cd-a195-d659bd3bee15\") " pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:46:34.789067 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.789016 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:34.798766 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:34.798742 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e30076a_a074_40b2_8c43_2b2757da92d2.slice/crio-f884cf9209dfb117d808a47c07faa2737d569d9245adb4a3d249f91351035fc0 WatchSource:0}: Error finding container f884cf9209dfb117d808a47c07faa2737d569d9245adb4a3d249f91351035fc0: Status 404 returned error can't find the container with id f884cf9209dfb117d808a47c07faa2737d569d9245adb4a3d249f91351035fc0 Sep 11 08:46:34.805122 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.805102 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9gfk6" Sep 11 08:46:34.813632 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:34.813608 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395401b2_9748_42b5_b600_4fa1f7369b1e.slice/crio-7560063aa492f6638b71b1d6bca0e82ec1d89c9232abf9313dcbab00f35ed6e4 WatchSource:0}: Error finding container 7560063aa492f6638b71b1d6bca0e82ec1d89c9232abf9313dcbab00f35ed6e4: Status 404 returned error can't find the container with id 7560063aa492f6638b71b1d6bca0e82ec1d89c9232abf9313dcbab00f35ed6e4 Sep 11 08:46:34.822151 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.822132 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" Sep 11 08:46:34.826188 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.826172 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" Sep 11 08:46:34.830642 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:34.830612 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45e03031_b380_4f4d_8259_09eaccf50bf1.slice/crio-0372e868bb8893caf97ac5e3adfff8d58aa0e198890eea1df379e17ec8a21281 WatchSource:0}: Error finding container 0372e868bb8893caf97ac5e3adfff8d58aa0e198890eea1df379e17ec8a21281: Status 404 returned error can't find the container with id 0372e868bb8893caf97ac5e3adfff8d58aa0e198890eea1df379e17ec8a21281 Sep 11 08:46:34.835261 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:34.835233 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8901cf6c_2530_49cc_971f_431e24a0b88d.slice/crio-73a2efcefceee4d025e7b71a76de3d06bc0ebe0fb061891198afdec918dee020 WatchSource:0}: Error finding container 73a2efcefceee4d025e7b71a76de3d06bc0ebe0fb061891198afdec918dee020: Status 404 returned error can't find the container with id 73a2efcefceee4d025e7b71a76de3d06bc0ebe0fb061891198afdec918dee020 Sep 11 08:46:34.839650 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.839631 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4smrv" Sep 11 08:46:34.849055 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:34.849037 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20f65aa9_c978_41d0_9ca1_9e1ddd4ef5bf.slice/crio-805a8083b6d26706207bebd4c40d222cb29a5000c9cf8fc2d1d7de06dbba4a5e WatchSource:0}: Error finding container 805a8083b6d26706207bebd4c40d222cb29a5000c9cf8fc2d1d7de06dbba4a5e: Status 404 returned error can't find the container with id 805a8083b6d26706207bebd4c40d222cb29a5000c9cf8fc2d1d7de06dbba4a5e Sep 11 08:46:34.852241 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.852222 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x9gk7" Sep 11 08:46:34.852843 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.852827 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 08:46:34.861430 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:34.861409 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod731bdf3b_9131_4284_a232_93b9b73f3a89.slice/crio-4b2b5c366bdacd9f14b3decf829862ac5aa9c997849c7d82f908f42a2eee2261 WatchSource:0}: Error finding container 4b2b5c366bdacd9f14b3decf829862ac5aa9c997849c7d82f908f42a2eee2261: Status 404 returned error can't find the container with id 4b2b5c366bdacd9f14b3decf829862ac5aa9c997849c7d82f908f42a2eee2261 Sep 11 08:46:34.869442 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.869428 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cbhgc" Sep 11 08:46:34.877292 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:34.877267 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cd3f396_137e_4748_83b5_582ddc810965.slice/crio-2dda2500d452472ce48566b93824c730a4024a8cbf589e7698fcb3e53d88a11d WatchSource:0}: Error finding container 2dda2500d452472ce48566b93824c730a4024a8cbf589e7698fcb3e53d88a11d: Status 404 returned error can't find the container with id 2dda2500d452472ce48566b93824c730a4024a8cbf589e7698fcb3e53d88a11d Sep 11 08:46:34.881472 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.881452 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tgx4q" Sep 11 08:46:34.889666 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:34.889649 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wpkr7" Sep 11 08:46:34.890114 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:34.890088 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaf9e2dc_28c5_4aa3_bfd0_b09bbfd49b0b.slice/crio-ea2cd2c411a808a8757c2c509b8ad055a4f10861eeb5448fd396e3972d3c3f53 WatchSource:0}: Error finding container ea2cd2c411a808a8757c2c509b8ad055a4f10861eeb5448fd396e3972d3c3f53: Status 404 returned error can't find the container with id ea2cd2c411a808a8757c2c509b8ad055a4f10861eeb5448fd396e3972d3c3f53 Sep 11 08:46:34.899869 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:46:34.899847 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc62c56b1_a68f_4a3f_a41b_23a2a08e148d.slice/crio-93d25c23a6f7989eb99ab3e192d056b4bbb01418a3a1421bf326e9f0fe815be1 WatchSource:0}: Error finding container 93d25c23a6f7989eb99ab3e192d056b4bbb01418a3a1421bf326e9f0fe815be1: Status 404 returned error can't find the container with id 93d25c23a6f7989eb99ab3e192d056b4bbb01418a3a1421bf326e9f0fe815be1 Sep 11 08:46:35.184608 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:35.184530 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e53c87e2-2721-4540-87c6-214593d333a4-original-pull-secret\") pod \"global-pull-secret-syncer-tldcl\" (UID: \"e53c87e2-2721-4540-87c6-214593d333a4\") " pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:35.184608 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:35.184572 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faa7ad4f-7140-43cd-a195-d659bd3bee15-metrics-certs\") pod \"network-metrics-daemon-97r66\" (UID: \"faa7ad4f-7140-43cd-a195-d659bd3bee15\") " pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:46:35.184762 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:35.184687 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 08:46:35.184762 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:35.184740 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faa7ad4f-7140-43cd-a195-d659bd3bee15-metrics-certs podName:faa7ad4f-7140-43cd-a195-d659bd3bee15 nodeName:}" failed. No retries permitted until 2026-09-11 08:46:36.18472662 +0000 UTC m=+3.084624898 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/faa7ad4f-7140-43cd-a195-d659bd3bee15-metrics-certs") pod "network-metrics-daemon-97r66" (UID: "faa7ad4f-7140-43cd-a195-d659bd3bee15") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 08:46:35.184839 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:35.184687 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 08:46:35.184839 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:35.184826 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e53c87e2-2721-4540-87c6-214593d333a4-original-pull-secret podName:e53c87e2-2721-4540-87c6-214593d333a4 nodeName:}" failed. No retries permitted until 2026-09-11 08:46:36.184812721 +0000 UTC m=+3.084711018 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e53c87e2-2721-4540-87c6-214593d333a4-original-pull-secret") pod "global-pull-secret-syncer-tldcl" (UID: "e53c87e2-2721-4540-87c6-214593d333a4") : object "kube-system"/"original-pull-secret" not registered Sep 11 08:46:35.285542 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:35.285502 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlkzg\" (UniqueName: \"kubernetes.io/projected/f24c0bea-46aa-432e-a51f-f25451143d07-kube-api-access-zlkzg\") pod \"network-check-target-q7bqv\" (UID: \"f24c0bea-46aa-432e-a51f-f25451143d07\") " pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:46:35.285727 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:35.285687 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 08:46:35.285727 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:35.285708 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 08:46:35.285727 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:35.285719 2562 projected.go:196] Error preparing data for projected volume kube-api-access-zlkzg for pod openshift-network-diagnostics/network-check-target-q7bqv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 08:46:35.285899 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:35.285780 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f24c0bea-46aa-432e-a51f-f25451143d07-kube-api-access-zlkzg podName:f24c0bea-46aa-432e-a51f-f25451143d07 nodeName:}" failed. No retries permitted until 2026-09-11 08:46:36.285761046 +0000 UTC m=+3.185659333 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zlkzg" (UniqueName: "kubernetes.io/projected/f24c0bea-46aa-432e-a51f-f25451143d07-kube-api-access-zlkzg") pod "network-check-target-q7bqv" (UID: "f24c0bea-46aa-432e-a51f-f25451143d07") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 08:46:35.489641 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:35.489560 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 08:46:35.518002 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:35.517922 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 08:41:35 +0000 UTC" deadline="2028-03-30 10:55:42.727452266 +0000 UTC" Sep 11 08:46:35.518002 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:35.517982 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13586h9m7.2094752s" Sep 11 08:46:35.575426 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:35.574841 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 08:46:35.583657 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:35.583631 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:35.583799 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:35.583781 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tldcl" podUID="e53c87e2-2721-4540-87c6-214593d333a4" Sep 11 08:46:35.598258 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:35.598162 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wpkr7" event={"ID":"c62c56b1-a68f-4a3f-a41b-23a2a08e148d","Type":"ContainerStarted","Data":"93d25c23a6f7989eb99ab3e192d056b4bbb01418a3a1421bf326e9f0fe815be1"} Sep 11 08:46:35.626601 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:35.626567 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tgx4q" event={"ID":"caf9e2dc-28c5-4aa3-bfd0-b09bbfd49b0b","Type":"ContainerStarted","Data":"ea2cd2c411a808a8757c2c509b8ad055a4f10861eeb5448fd396e3972d3c3f53"} Sep 11 08:46:35.647348 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:35.647272 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x9gk7" event={"ID":"731bdf3b-9131-4284-a232-93b9b73f3a89","Type":"ContainerStarted","Data":"4b2b5c366bdacd9f14b3decf829862ac5aa9c997849c7d82f908f42a2eee2261"} Sep 11 08:46:35.673535 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:35.673486 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" event={"ID":"8901cf6c-2530-49cc-971f-431e24a0b88d","Type":"ContainerStarted","Data":"73a2efcefceee4d025e7b71a76de3d06bc0ebe0fb061891198afdec918dee020"} Sep 11 08:46:35.695448 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:35.695319 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" event={"ID":"45e03031-b380-4f4d-8259-09eaccf50bf1","Type":"ContainerStarted","Data":"0372e868bb8893caf97ac5e3adfff8d58aa0e198890eea1df379e17ec8a21281"} Sep 11 08:46:35.717781 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:35.717689 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9gfk6" event={"ID":"395401b2-9748-42b5-b600-4fa1f7369b1e","Type":"ContainerStarted","Data":"7560063aa492f6638b71b1d6bca0e82ec1d89c9232abf9313dcbab00f35ed6e4"} Sep 11 08:46:35.729057 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:35.728948 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" event={"ID":"1e30076a-a074-40b2-8c43-2b2757da92d2","Type":"ContainerStarted","Data":"f884cf9209dfb117d808a47c07faa2737d569d9245adb4a3d249f91351035fc0"} Sep 11 08:46:35.769279 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:35.769123 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-169.ec2.internal" event={"ID":"40c4a495594d5730f78ee6e94869d17f","Type":"ContainerStarted","Data":"5a6803a755b02acd3815acea3ad6668103e91b76d74afe5b61b9f16270ec9a28"} Sep 11 08:46:35.792925 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:35.792896 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cbhgc" event={"ID":"4cd3f396-137e-4748-83b5-582ddc810965","Type":"ContainerStarted","Data":"2dda2500d452472ce48566b93824c730a4024a8cbf589e7698fcb3e53d88a11d"} Sep 11 08:46:35.814307 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:35.814227 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4smrv" event={"ID":"20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf","Type":"ContainerStarted","Data":"805a8083b6d26706207bebd4c40d222cb29a5000c9cf8fc2d1d7de06dbba4a5e"} Sep 11 08:46:35.847965 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:35.847401 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-169.ec2.internal" event={"ID":"458a6d26c1086ff6914f03deab3d15f2","Type":"ContainerStarted","Data":"3238f2bf35b3e2f00f27942faf7e1644be1c1f4e8be33514569970cb04dbfce8"} Sep 11 08:46:36.194657 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:36.194526 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e53c87e2-2721-4540-87c6-214593d333a4-original-pull-secret\") pod \"global-pull-secret-syncer-tldcl\" (UID: \"e53c87e2-2721-4540-87c6-214593d333a4\") " pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:36.194657 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:36.194583 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faa7ad4f-7140-43cd-a195-d659bd3bee15-metrics-certs\") pod \"network-metrics-daemon-97r66\" (UID: \"faa7ad4f-7140-43cd-a195-d659bd3bee15\") " pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:46:36.194885 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:36.194746 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 08:46:36.194885 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:36.194812 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faa7ad4f-7140-43cd-a195-d659bd3bee15-metrics-certs podName:faa7ad4f-7140-43cd-a195-d659bd3bee15 nodeName:}" failed. No retries permitted until 2026-09-11 08:46:38.194791012 +0000 UTC m=+5.094689289 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/faa7ad4f-7140-43cd-a195-d659bd3bee15-metrics-certs") pod "network-metrics-daemon-97r66" (UID: "faa7ad4f-7140-43cd-a195-d659bd3bee15") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 08:46:36.195260 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:36.195238 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 08:46:36.195366 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:36.195295 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e53c87e2-2721-4540-87c6-214593d333a4-original-pull-secret podName:e53c87e2-2721-4540-87c6-214593d333a4 nodeName:}" failed. No retries permitted until 2026-09-11 08:46:38.195280125 +0000 UTC m=+5.095178417 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e53c87e2-2721-4540-87c6-214593d333a4-original-pull-secret") pod "global-pull-secret-syncer-tldcl" (UID: "e53c87e2-2721-4540-87c6-214593d333a4") : object "kube-system"/"original-pull-secret" not registered Sep 11 08:46:36.295420 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:36.295388 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlkzg\" (UniqueName: \"kubernetes.io/projected/f24c0bea-46aa-432e-a51f-f25451143d07-kube-api-access-zlkzg\") pod \"network-check-target-q7bqv\" (UID: \"f24c0bea-46aa-432e-a51f-f25451143d07\") " pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:46:36.295580 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:36.295560 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 08:46:36.295654 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:36.295586 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 08:46:36.295654 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:36.295601 2562 projected.go:196] Error preparing data for projected volume kube-api-access-zlkzg for pod openshift-network-diagnostics/network-check-target-q7bqv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 08:46:36.295756 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:36.295667 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f24c0bea-46aa-432e-a51f-f25451143d07-kube-api-access-zlkzg podName:f24c0bea-46aa-432e-a51f-f25451143d07 nodeName:}" failed. No retries permitted until 2026-09-11 08:46:38.295646582 +0000 UTC m=+5.195544873 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zlkzg" (UniqueName: "kubernetes.io/projected/f24c0bea-46aa-432e-a51f-f25451143d07-kube-api-access-zlkzg") pod "network-check-target-q7bqv" (UID: "f24c0bea-46aa-432e-a51f-f25451143d07") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 08:46:36.317177 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:36.317149 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 08:46:36.518818 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:36.518727 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 08:41:35 +0000 UTC" deadline="2028-03-21 00:33:11.461056721 +0000 UTC" Sep 11 08:46:36.518818 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:36.518768 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13359h46m34.942294358s" Sep 11 08:46:36.567283 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:36.567247 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:46:36.567503 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:36.567477 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7bqv" podUID="f24c0bea-46aa-432e-a51f-f25451143d07" Sep 11 08:46:36.567917 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:36.567892 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:46:36.568094 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:36.568072 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-97r66" podUID="faa7ad4f-7140-43cd-a195-d659bd3bee15" Sep 11 08:46:37.566765 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:37.566736 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:37.567219 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:37.566873 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tldcl" podUID="e53c87e2-2721-4540-87c6-214593d333a4" Sep 11 08:46:38.214215 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:38.214130 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e53c87e2-2721-4540-87c6-214593d333a4-original-pull-secret\") pod \"global-pull-secret-syncer-tldcl\" (UID: \"e53c87e2-2721-4540-87c6-214593d333a4\") " pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:38.214395 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:38.214245 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faa7ad4f-7140-43cd-a195-d659bd3bee15-metrics-certs\") pod \"network-metrics-daemon-97r66\" (UID: \"faa7ad4f-7140-43cd-a195-d659bd3bee15\") " pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:46:38.214477 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:38.214452 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 08:46:38.214538 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:38.214520 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faa7ad4f-7140-43cd-a195-d659bd3bee15-metrics-certs podName:faa7ad4f-7140-43cd-a195-d659bd3bee15 nodeName:}" failed. No retries permitted until 2026-09-11 08:46:42.214500642 +0000 UTC m=+9.114398945 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/faa7ad4f-7140-43cd-a195-d659bd3bee15-metrics-certs") pod "network-metrics-daemon-97r66" (UID: "faa7ad4f-7140-43cd-a195-d659bd3bee15") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 08:46:38.214945 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:38.214923 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 08:46:38.215030 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:38.214982 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e53c87e2-2721-4540-87c6-214593d333a4-original-pull-secret podName:e53c87e2-2721-4540-87c6-214593d333a4 nodeName:}" failed. No retries permitted until 2026-09-11 08:46:42.214966788 +0000 UTC m=+9.114865072 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e53c87e2-2721-4540-87c6-214593d333a4-original-pull-secret") pod "global-pull-secret-syncer-tldcl" (UID: "e53c87e2-2721-4540-87c6-214593d333a4") : object "kube-system"/"original-pull-secret" not registered Sep 11 08:46:38.314930 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:38.314892 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlkzg\" (UniqueName: \"kubernetes.io/projected/f24c0bea-46aa-432e-a51f-f25451143d07-kube-api-access-zlkzg\") pod \"network-check-target-q7bqv\" (UID: \"f24c0bea-46aa-432e-a51f-f25451143d07\") " pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:46:38.315146 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:38.315112 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 08:46:38.315146 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:38.315143 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 08:46:38.315276 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:38.315157 2562 projected.go:196] Error preparing data for projected volume kube-api-access-zlkzg for pod openshift-network-diagnostics/network-check-target-q7bqv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 08:46:38.315276 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:38.315213 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f24c0bea-46aa-432e-a51f-f25451143d07-kube-api-access-zlkzg podName:f24c0bea-46aa-432e-a51f-f25451143d07 nodeName:}" failed. No retries permitted until 2026-09-11 08:46:42.315193947 +0000 UTC m=+9.215092244 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zlkzg" (UniqueName: "kubernetes.io/projected/f24c0bea-46aa-432e-a51f-f25451143d07-kube-api-access-zlkzg") pod "network-check-target-q7bqv" (UID: "f24c0bea-46aa-432e-a51f-f25451143d07") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 08:46:38.567456 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:38.566679 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:46:38.567456 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:38.566839 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-97r66" podUID="faa7ad4f-7140-43cd-a195-d659bd3bee15" Sep 11 08:46:38.567456 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:38.567234 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:46:38.567456 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:38.567331 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7bqv" podUID="f24c0bea-46aa-432e-a51f-f25451143d07" Sep 11 08:46:39.566911 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:39.566872 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:39.567091 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:39.567032 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tldcl" podUID="e53c87e2-2721-4540-87c6-214593d333a4" Sep 11 08:46:40.566914 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:40.566879 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:46:40.566914 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:40.566911 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:46:40.567481 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:40.567023 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7bqv" podUID="f24c0bea-46aa-432e-a51f-f25451143d07" Sep 11 08:46:40.567542 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:40.567497 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-97r66" podUID="faa7ad4f-7140-43cd-a195-d659bd3bee15" Sep 11 08:46:41.570813 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:41.570781 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:46:41.571321 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:41.570931 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-97r66" podUID="faa7ad4f-7140-43cd-a195-d659bd3bee15" Sep 11 08:46:41.571321 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:41.570789 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:41.571321 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:41.571223 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tldcl" podUID="e53c87e2-2721-4540-87c6-214593d333a4" Sep 11 08:46:42.251975 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:42.251944 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e53c87e2-2721-4540-87c6-214593d333a4-original-pull-secret\") pod \"global-pull-secret-syncer-tldcl\" (UID: \"e53c87e2-2721-4540-87c6-214593d333a4\") " pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:42.252141 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:42.251996 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faa7ad4f-7140-43cd-a195-d659bd3bee15-metrics-certs\") pod \"network-metrics-daemon-97r66\" (UID: \"faa7ad4f-7140-43cd-a195-d659bd3bee15\") " pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:46:42.252141 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:42.252108 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 08:46:42.252252 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:42.252177 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faa7ad4f-7140-43cd-a195-d659bd3bee15-metrics-certs podName:faa7ad4f-7140-43cd-a195-d659bd3bee15 nodeName:}" failed. No retries permitted until 2026-09-11 08:46:50.252155663 +0000 UTC m=+17.152053963 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/faa7ad4f-7140-43cd-a195-d659bd3bee15-metrics-certs") pod "network-metrics-daemon-97r66" (UID: "faa7ad4f-7140-43cd-a195-d659bd3bee15") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 08:46:42.252252 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:42.252108 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 08:46:42.252406 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:42.252333 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e53c87e2-2721-4540-87c6-214593d333a4-original-pull-secret podName:e53c87e2-2721-4540-87c6-214593d333a4 nodeName:}" failed. No retries permitted until 2026-09-11 08:46:50.252295087 +0000 UTC m=+17.152193377 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e53c87e2-2721-4540-87c6-214593d333a4-original-pull-secret") pod "global-pull-secret-syncer-tldcl" (UID: "e53c87e2-2721-4540-87c6-214593d333a4") : object "kube-system"/"original-pull-secret" not registered Sep 11 08:46:42.352433 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:42.352398 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlkzg\" (UniqueName: \"kubernetes.io/projected/f24c0bea-46aa-432e-a51f-f25451143d07-kube-api-access-zlkzg\") pod \"network-check-target-q7bqv\" (UID: \"f24c0bea-46aa-432e-a51f-f25451143d07\") " pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:46:42.352618 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:42.352580 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 08:46:42.352618 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:42.352605 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 08:46:42.352618 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:42.352616 2562 projected.go:196] Error preparing data for projected volume kube-api-access-zlkzg for pod openshift-network-diagnostics/network-check-target-q7bqv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 08:46:42.352763 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:42.352673 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f24c0bea-46aa-432e-a51f-f25451143d07-kube-api-access-zlkzg podName:f24c0bea-46aa-432e-a51f-f25451143d07 nodeName:}" failed. No retries permitted until 2026-09-11 08:46:50.352658769 +0000 UTC m=+17.252557046 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zlkzg" (UniqueName: "kubernetes.io/projected/f24c0bea-46aa-432e-a51f-f25451143d07-kube-api-access-zlkzg") pod "network-check-target-q7bqv" (UID: "f24c0bea-46aa-432e-a51f-f25451143d07") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 08:46:42.567197 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:42.567109 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:46:42.567387 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:42.567231 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7bqv" podUID="f24c0bea-46aa-432e-a51f-f25451143d07" Sep 11 08:46:43.571053 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:43.571021 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:43.571483 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:43.571175 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tldcl" podUID="e53c87e2-2721-4540-87c6-214593d333a4" Sep 11 08:46:43.571483 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:43.571290 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:46:43.571483 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:43.571435 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-97r66" podUID="faa7ad4f-7140-43cd-a195-d659bd3bee15" Sep 11 08:46:44.566766 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:44.566670 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:46:44.566955 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:44.566801 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7bqv" podUID="f24c0bea-46aa-432e-a51f-f25451143d07" Sep 11 08:46:45.567095 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:45.567055 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:46:45.567563 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:45.567194 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-97r66" podUID="faa7ad4f-7140-43cd-a195-d659bd3bee15" Sep 11 08:46:45.567563 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:45.567240 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:45.567563 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:45.567339 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tldcl" podUID="e53c87e2-2721-4540-87c6-214593d333a4" Sep 11 08:46:46.567106 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:46.567078 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:46:46.567469 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:46.567184 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7bqv" podUID="f24c0bea-46aa-432e-a51f-f25451143d07" Sep 11 08:46:47.566822 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:47.566728 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:46:47.566966 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:47.566873 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-97r66" podUID="faa7ad4f-7140-43cd-a195-d659bd3bee15" Sep 11 08:46:47.566966 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:47.566940 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:47.567088 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:47.567068 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tldcl" podUID="e53c87e2-2721-4540-87c6-214593d333a4" Sep 11 08:46:48.567100 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:48.567063 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:46:48.567605 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:48.567206 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7bqv" podUID="f24c0bea-46aa-432e-a51f-f25451143d07" Sep 11 08:46:49.569700 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:49.569668 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:46:49.570102 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:49.569667 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:49.570102 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:49.569774 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-97r66" podUID="faa7ad4f-7140-43cd-a195-d659bd3bee15" Sep 11 08:46:49.570102 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:49.569866 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tldcl" podUID="e53c87e2-2721-4540-87c6-214593d333a4" Sep 11 08:46:50.310901 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:50.310852 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faa7ad4f-7140-43cd-a195-d659bd3bee15-metrics-certs\") pod \"network-metrics-daemon-97r66\" (UID: \"faa7ad4f-7140-43cd-a195-d659bd3bee15\") " pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:46:50.311119 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:50.310949 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e53c87e2-2721-4540-87c6-214593d333a4-original-pull-secret\") pod \"global-pull-secret-syncer-tldcl\" (UID: \"e53c87e2-2721-4540-87c6-214593d333a4\") " pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:50.311119 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:50.310999 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 08:46:50.311119 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:50.311045 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 08:46:50.311119 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:50.311073 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faa7ad4f-7140-43cd-a195-d659bd3bee15-metrics-certs podName:faa7ad4f-7140-43cd-a195-d659bd3bee15 nodeName:}" failed. No retries permitted until 2026-09-11 08:47:06.31105139 +0000 UTC m=+33.210949669 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/faa7ad4f-7140-43cd-a195-d659bd3bee15-metrics-certs") pod "network-metrics-daemon-97r66" (UID: "faa7ad4f-7140-43cd-a195-d659bd3bee15") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 08:46:50.311119 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:50.311096 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e53c87e2-2721-4540-87c6-214593d333a4-original-pull-secret podName:e53c87e2-2721-4540-87c6-214593d333a4 nodeName:}" failed. No retries permitted until 2026-09-11 08:47:06.31108547 +0000 UTC m=+33.210983748 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e53c87e2-2721-4540-87c6-214593d333a4-original-pull-secret") pod "global-pull-secret-syncer-tldcl" (UID: "e53c87e2-2721-4540-87c6-214593d333a4") : object "kube-system"/"original-pull-secret" not registered Sep 11 08:46:50.411453 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:50.411428 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlkzg\" (UniqueName: \"kubernetes.io/projected/f24c0bea-46aa-432e-a51f-f25451143d07-kube-api-access-zlkzg\") pod \"network-check-target-q7bqv\" (UID: \"f24c0bea-46aa-432e-a51f-f25451143d07\") " pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:46:50.411597 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:50.411578 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 08:46:50.411644 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:50.411602 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 08:46:50.411644 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:50.411611 2562 projected.go:196] Error preparing data for projected volume kube-api-access-zlkzg for pod openshift-network-diagnostics/network-check-target-q7bqv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 08:46:50.411707 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:50.411658 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f24c0bea-46aa-432e-a51f-f25451143d07-kube-api-access-zlkzg podName:f24c0bea-46aa-432e-a51f-f25451143d07 nodeName:}" failed. No retries permitted until 2026-09-11 08:47:06.411645942 +0000 UTC m=+33.311544222 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zlkzg" (UniqueName: "kubernetes.io/projected/f24c0bea-46aa-432e-a51f-f25451143d07-kube-api-access-zlkzg") pod "network-check-target-q7bqv" (UID: "f24c0bea-46aa-432e-a51f-f25451143d07") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 08:46:50.566995 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:50.566919 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:46:50.567155 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:50.567044 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7bqv" podUID="f24c0bea-46aa-432e-a51f-f25451143d07" Sep 11 08:46:51.567361 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:51.567327 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:51.567872 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:51.567745 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tldcl" podUID="e53c87e2-2721-4540-87c6-214593d333a4" Sep 11 08:46:51.567937 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:51.567875 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:46:51.568087 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:51.568039 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-97r66" podUID="faa7ad4f-7140-43cd-a195-d659bd3bee15" Sep 11 08:46:52.566522 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:52.566489 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:46:52.566698 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:52.566603 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7bqv" podUID="f24c0bea-46aa-432e-a51f-f25451143d07" Sep 11 08:46:53.567673 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:53.567630 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:53.568135 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:53.567743 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tldcl" podUID="e53c87e2-2721-4540-87c6-214593d333a4" Sep 11 08:46:53.568135 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:53.567780 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:46:53.568135 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:53.567864 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-97r66" podUID="faa7ad4f-7140-43cd-a195-d659bd3bee15" Sep 11 08:46:54.567216 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:54.567085 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:46:54.567292 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:54.567278 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7bqv" podUID="f24c0bea-46aa-432e-a51f-f25451143d07" Sep 11 08:46:54.901398 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:54.901350 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-169.ec2.internal" event={"ID":"458a6d26c1086ff6914f03deab3d15f2","Type":"ContainerStarted","Data":"8bd28fe5c02f952ef3eeb2dd2b8f60d39c8532d5acdf2c182fa7c050b3b42722"} Sep 11 08:46:54.904849 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:54.904806 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" event={"ID":"8901cf6c-2530-49cc-971f-431e24a0b88d","Type":"ContainerStarted","Data":"b75615750f16c9f3d2a29f1f2bb9b5fa7ccd36cb8a92efac8104cb0bd414b213"} Sep 11 08:46:54.907174 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:54.907157 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/ovn-acl-logging/0.log" Sep 11 08:46:54.907571 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:54.907555 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" event={"ID":"1e30076a-a074-40b2-8c43-2b2757da92d2","Type":"ContainerStarted","Data":"316e871d3b949fdc4315931381b592c3a957a91386b7baa0726af95e61494df6"} Sep 11 08:46:54.907642 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:54.907607 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" event={"ID":"1e30076a-a074-40b2-8c43-2b2757da92d2","Type":"ContainerStarted","Data":"f58626c6da428a2e168c4e3b0a266c2e43f4aa25fb67511972a3c65a3dbd9b71"} Sep 11 08:46:54.907682 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:54.907640 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" event={"ID":"1e30076a-a074-40b2-8c43-2b2757da92d2","Type":"ContainerStarted","Data":"68694bc2bacbcaa881d43a447c507f59ad489afd7ff1dd12afb9d32d70b1ea4c"} Sep 11 08:46:54.915861 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:54.915775 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-169.ec2.internal" podStartSLOduration=19.915758299 podStartE2EDuration="19.915758299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 08:46:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:46:54.91529212 +0000 UTC m=+21.815190418" watchObservedRunningTime="2026-09-11 08:46:54.915758299 +0000 UTC m=+21.815656600" Sep 11 08:46:54.930944 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:54.930815 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nxhhd" podStartSLOduration=1.45783812 podStartE2EDuration="20.930798961s" totalImagesPullingTime="19.472960841s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 08:46:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:46:54.93074443 +0000 UTC m=+21.830642728" watchObservedRunningTime="2026-09-11 08:46:54.930798961 +0000 UTC m=+21.830697260" Sep 11 08:46:55.566632 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:55.566349 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:46:55.566722 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:55.566349 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:55.566768 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:55.566713 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-97r66" podUID="faa7ad4f-7140-43cd-a195-d659bd3bee15" Sep 11 08:46:55.566808 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:55.566779 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tldcl" podUID="e53c87e2-2721-4540-87c6-214593d333a4" Sep 11 08:46:55.912545 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:55.912517 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/ovn-acl-logging/0.log" Sep 11 08:46:55.913000 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:55.912847 2562 generic.go:356] "Generic (PLEG): container finished" podID="1e30076a-a074-40b2-8c43-2b2757da92d2" containerID="f58626c6da428a2e168c4e3b0a266c2e43f4aa25fb67511972a3c65a3dbd9b71" exitCode=1 Sep 11 08:46:55.913000 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:55.912904 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" event={"ID":"1e30076a-a074-40b2-8c43-2b2757da92d2","Type":"ContainerStarted","Data":"ed1ab79cea62177e7c0cd4094ae85221d740700bc0b3116fa1ad556b73be1eee"} Sep 11 08:46:55.913000 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:55.912971 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" event={"ID":"1e30076a-a074-40b2-8c43-2b2757da92d2","Type":"ContainerDied","Data":"f58626c6da428a2e168c4e3b0a266c2e43f4aa25fb67511972a3c65a3dbd9b71"} Sep 11 08:46:55.913131 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:55.913019 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" event={"ID":"1e30076a-a074-40b2-8c43-2b2757da92d2","Type":"ContainerStarted","Data":"3815acaaa8c40c204634c86bff906fd6778f2dea3833cd9ca150e2a7851d5d42"} Sep 11 08:46:55.913131 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:55.913053 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" event={"ID":"1e30076a-a074-40b2-8c43-2b2757da92d2","Type":"ContainerStarted","Data":"be5e1d1a4bfe42e451aea0e26dcc8974f3d93ded47b6eeb7378066c692643946"} Sep 11 08:46:55.914392 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:55.914360 2562 generic.go:356] "Generic (PLEG): container finished" podID="40c4a495594d5730f78ee6e94869d17f" containerID="96acbf91a1fef6dbf7fe5fb710816eb6af39720c370be8b1c650283903e5fdd0" exitCode=0 Sep 11 08:46:55.914478 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:55.914410 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-169.ec2.internal" event={"ID":"40c4a495594d5730f78ee6e94869d17f","Type":"ContainerDied","Data":"96acbf91a1fef6dbf7fe5fb710816eb6af39720c370be8b1c650283903e5fdd0"} Sep 11 08:46:55.915911 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:55.915881 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cbhgc" event={"ID":"4cd3f396-137e-4748-83b5-582ddc810965","Type":"ContainerStarted","Data":"e77af3b8ec5957a706b5b0c5ffa0da16db1c0753d2b5de843af769388f93c1b9"} Sep 11 08:46:55.917435 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:55.917402 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4smrv" event={"ID":"20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf","Type":"ContainerStarted","Data":"c0872889bed086ff7b2dc7356a86210b324c144d46fd28305d0d13cc17fb5611"} Sep 11 08:46:55.918788 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:55.918768 2562 generic.go:356] "Generic (PLEG): container finished" podID="c62c56b1-a68f-4a3f-a41b-23a2a08e148d" containerID="3f0203375f626ccb1cc7eb425e7fd64ce342bf297b8f8b3c68c91e176ddf7cbc" exitCode=0 Sep 11 08:46:55.918891 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:55.918826 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wpkr7" event={"ID":"c62c56b1-a68f-4a3f-a41b-23a2a08e148d","Type":"ContainerDied","Data":"3f0203375f626ccb1cc7eb425e7fd64ce342bf297b8f8b3c68c91e176ddf7cbc"} Sep 11 08:46:55.920387 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:55.920364 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tgx4q" event={"ID":"caf9e2dc-28c5-4aa3-bfd0-b09bbfd49b0b","Type":"ContainerStarted","Data":"1c4870c761a476c80404c8b589e64f08b69732a39b03a55606a1cdb2713c0d39"} Sep 11 08:46:55.921967 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:55.921946 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x9gk7" event={"ID":"731bdf3b-9131-4284-a232-93b9b73f3a89","Type":"ContainerStarted","Data":"7eb06594589e3d34e619c5db107be2185ce1ebab6703dc8b456eec5da306bc5c"} Sep 11 08:46:55.923337 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:55.923294 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" event={"ID":"45e03031-b380-4f4d-8259-09eaccf50bf1","Type":"ContainerStarted","Data":"23fddf7ec1bb0ebd63b78f489bc2c50147aed83ba6a787b5c4b1ecd4c731feeb"} Sep 11 08:46:55.924620 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:55.924603 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9gfk6" event={"ID":"395401b2-9748-42b5-b600-4fa1f7369b1e","Type":"ContainerStarted","Data":"38dfa432d5ce4d499ed2e4c63ddba8e1c9e0f37481dada53351c080d8bc4e8cd"} Sep 11 08:46:55.939475 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:55.939434 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tgx4q" podStartSLOduration=2.5568662399999997 podStartE2EDuration="21.939421892s" totalImagesPullingTime="19.382555652s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 08:46:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:46:55.939047982 +0000 UTC m=+22.838946280" watchObservedRunningTime="2026-09-11 08:46:55.939421892 +0000 UTC m=+22.839320189" Sep 11 08:46:55.969001 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:55.968960 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4smrv" podStartSLOduration=2.5427252129999998 podStartE2EDuration="21.968945352s" totalImagesPullingTime="19.426220139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:46:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:46:55.968505378 +0000 UTC m=+22.868403680" watchObservedRunningTime="2026-09-11 08:46:55.968945352 +0000 UTC m=+22.868843657" Sep 11 08:46:55.982337 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:55.982283 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-cbhgc" podStartSLOduration=2.553584792 podStartE2EDuration="21.982270869s" totalImagesPullingTime="19.428686077s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 08:46:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:46:55.982140958 +0000 UTC m=+22.882039256" watchObservedRunningTime="2026-09-11 08:46:55.982270869 +0000 UTC m=+22.882169167" Sep 11 08:46:55.997184 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:55.997150 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9gfk6" podStartSLOduration=2.535308993 podStartE2EDuration="21.997137683s" totalImagesPullingTime="19.46182869s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 08:46:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:46:55.996798029 +0000 UTC m=+22.896696329" watchObservedRunningTime="2026-09-11 08:46:55.997137683 +0000 UTC m=+22.897035980" Sep 11 08:46:56.013917 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:56.013882 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-x9gk7" podStartSLOduration=1.943833772 podStartE2EDuration="22.013870245s" totalImagesPullingTime="20.070036473s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 08:46:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:46:56.013622694 +0000 UTC m=+22.913520992" watchObservedRunningTime="2026-09-11 08:46:56.013870245 +0000 UTC m=+22.913768542" Sep 11 08:46:56.080541 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:56.080504 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:46:56.157787 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:56.157744 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:46:56.238337 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:56.238315 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 11 08:46:56.258537 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:56.258512 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:46:56.350086 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:56.350064 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:46:56.566490 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:56.566418 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:46:56.566635 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:56.566521 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7bqv" podUID="f24c0bea-46aa-432e-a51f-f25451143d07" Sep 11 08:46:56.671581 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:56.671483 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T08:46:56.238331667Z","UUID":"b888b432-cc5d-4c57-88fd-43c31e5dcd46","Handler":null,"Name":"","Endpoint":""} Sep 11 08:46:56.671581 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:56.671546 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4smrv" Sep 11 08:46:56.672057 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:56.672038 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4smrv" Sep 11 08:46:56.673073 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:56.673057 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 11 08:46:56.673175 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:56.673087 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 11 08:46:56.929348 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:56.929252 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" event={"ID":"45e03031-b380-4f4d-8259-09eaccf50bf1","Type":"ContainerStarted","Data":"a0b83d0816b136321279fc5908c261d3a6e2ad3f9735454fa8390f9602ca300d"} Sep 11 08:46:56.931231 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:56.931203 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-169.ec2.internal" event={"ID":"40c4a495594d5730f78ee6e94869d17f","Type":"ContainerStarted","Data":"32acbe0db25c511ba9cca0270b0da5dafa966ce26becbc200d22e1aa66bed857"} Sep 11 08:46:56.932048 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:56.932026 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4smrv" Sep 11 08:46:56.932631 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:56.932615 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4smrv" Sep 11 08:46:56.945945 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:56.945888 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-169.ec2.internal" podStartSLOduration=21.945868058 podStartE2EDuration="21.945868058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 08:46:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:46:56.945340618 +0000 UTC m=+23.845238917" watchObservedRunningTime="2026-09-11 08:46:56.945868058 +0000 UTC m=+23.845766384" Sep 11 08:46:57.009383 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:57.009350 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:46:57.476668 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:57.476416 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:46:57.567401 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:57.567378 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:57.567534 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:57.567503 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tldcl" podUID="e53c87e2-2721-4540-87c6-214593d333a4" Sep 11 08:46:57.567590 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:57.567378 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:46:57.567669 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:57.567649 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-97r66" podUID="faa7ad4f-7140-43cd-a195-d659bd3bee15" Sep 11 08:46:57.577837 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:57.577818 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:46:57.676465 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:57.676426 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:46:57.935417 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:57.935296 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" event={"ID":"45e03031-b380-4f4d-8259-09eaccf50bf1","Type":"ContainerStarted","Data":"40a0d23c7499f4e0e9b3d70b4b9e92f203f15bb2779c10b87abcd62bdc806bc3"} Sep 11 08:46:57.938685 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:57.938654 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/ovn-acl-logging/0.log" Sep 11 08:46:57.939088 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:57.939071 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" event={"ID":"1e30076a-a074-40b2-8c43-2b2757da92d2","Type":"ContainerStarted","Data":"c2e7d555b2e38547ee293e0cc28b940ad9014c98a1a3842978739d0b803c17de"} Sep 11 08:46:58.211258 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:58.211185 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:46:58.276812 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:58.276787 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:46:58.434076 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:58.434015 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:46:58.566696 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:58.566627 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:46:58.566817 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:58.566744 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7bqv" podUID="f24c0bea-46aa-432e-a51f-f25451143d07" Sep 11 08:46:58.608789 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:58.608764 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:46:58.708979 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:58.708951 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:46:59.567103 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:59.567073 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:46:59.567551 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:59.567080 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:46:59.567551 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:59.567204 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tldcl" podUID="e53c87e2-2721-4540-87c6-214593d333a4" Sep 11 08:46:59.567551 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:46:59.567248 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-97r66" podUID="faa7ad4f-7140-43cd-a195-d659bd3bee15" Sep 11 08:46:59.825764 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:59.825676 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:46:59.885042 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:59.885010 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:46:59.947056 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:59.947035 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/ovn-acl-logging/0.log" Sep 11 08:46:59.947521 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:59.947484 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" event={"ID":"1e30076a-a074-40b2-8c43-2b2757da92d2","Type":"ContainerStarted","Data":"4dc79fde05b70cf4e2c8d5eff472d48f7bb0b46f77a57a965cf31dc20cd62ab3"} Sep 11 08:46:59.947781 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:59.947760 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:59.947851 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:59.947839 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:59.947961 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:59.947948 2562 scope.go:117] "RemoveContainer" containerID="f58626c6da428a2e168c4e3b0a266c2e43f4aa25fb67511972a3c65a3dbd9b71" Sep 11 08:46:59.967081 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:59.967057 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:46:59.977842 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:46:59.977794 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6cxr" podStartSLOduration=4.226969549 podStartE2EDuration="25.977779507s" totalImagesPullingTime="21.750809958s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 08:46:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:46:57.964612646 +0000 UTC m=+24.864510945" watchObservedRunningTime="2026-09-11 08:46:59.977779507 +0000 UTC m=+26.877677816" Sep 11 08:47:00.566572 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:00.566527 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:47:00.566739 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:00.566654 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7bqv" podUID="f24c0bea-46aa-432e-a51f-f25451143d07" Sep 11 08:47:00.954550 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:00.954485 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/ovn-acl-logging/0.log" Sep 11 08:47:00.954979 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:00.954930 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" event={"ID":"1e30076a-a074-40b2-8c43-2b2757da92d2","Type":"ContainerStarted","Data":"a8d671e417de1d0278a2663188f8d9623c07aa595e7c6db7a3f53e8e5d385859"} Sep 11 08:47:00.955170 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:00.955146 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:47:00.973577 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:00.973543 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:47:01.016462 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:01.016398 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" podStartSLOduration=7.488079129 podStartE2EDuration="27.016378713s" totalImagesPullingTime="19.528299584s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 08:46:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:47:01.01481219 +0000 UTC m=+27.914710487" watchObservedRunningTime="2026-09-11 08:47:01.016378713 +0000 UTC m=+27.916277011" Sep 11 08:47:01.213040 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:01.212978 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:01.331289 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:01.331254 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:01.409953 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:01.409928 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q7bqv"] Sep 11 08:47:01.410081 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:01.410043 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:47:01.410139 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:01.410122 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7bqv" podUID="f24c0bea-46aa-432e-a51f-f25451143d07" Sep 11 08:47:01.415635 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:01.415606 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tldcl"] Sep 11 08:47:01.415752 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:01.415710 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:47:01.415848 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:01.415812 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tldcl" podUID="e53c87e2-2721-4540-87c6-214593d333a4" Sep 11 08:47:01.440515 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:01.440488 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-97r66"] Sep 11 08:47:01.440639 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:01.440595 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:47:01.440714 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:01.440692 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-97r66" podUID="faa7ad4f-7140-43cd-a195-d659bd3bee15" Sep 11 08:47:02.603926 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:02.603900 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:02.673693 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:02.673625 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:02.960716 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:02.960649 2562 generic.go:356] "Generic (PLEG): container finished" podID="c62c56b1-a68f-4a3f-a41b-23a2a08e148d" containerID="ddc670beca743226a48f5fe4eaeeae7e02045096d35347aaa9d1de01e46777d0" exitCode=0 Sep 11 08:47:02.960840 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:02.960732 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wpkr7" event={"ID":"c62c56b1-a68f-4a3f-a41b-23a2a08e148d","Type":"ContainerDied","Data":"ddc670beca743226a48f5fe4eaeeae7e02045096d35347aaa9d1de01e46777d0"} Sep 11 08:47:03.568015 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:03.567973 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:47:03.568164 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:03.568069 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:47:03.568164 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:03.568113 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:47:03.568164 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:03.568106 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7bqv" podUID="f24c0bea-46aa-432e-a51f-f25451143d07" Sep 11 08:47:03.568274 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:03.568213 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tldcl" podUID="e53c87e2-2721-4540-87c6-214593d333a4" Sep 11 08:47:03.568392 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:03.568364 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-97r66" podUID="faa7ad4f-7140-43cd-a195-d659bd3bee15" Sep 11 08:47:04.967813 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:04.967617 2562 generic.go:356] "Generic (PLEG): container finished" podID="c62c56b1-a68f-4a3f-a41b-23a2a08e148d" containerID="bf5ae8e8252b411fec87c27dc8d83c4295a671799d5546fdaf2165c0b703d9ae" exitCode=0 Sep 11 08:47:04.968218 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:04.967706 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wpkr7" event={"ID":"c62c56b1-a68f-4a3f-a41b-23a2a08e148d","Type":"ContainerDied","Data":"bf5ae8e8252b411fec87c27dc8d83c4295a671799d5546fdaf2165c0b703d9ae"} Sep 11 08:47:05.567415 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:05.567388 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:47:05.567564 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:05.567422 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:47:05.567564 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:05.567384 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:47:05.567564 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:05.567489 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tldcl" podUID="e53c87e2-2721-4540-87c6-214593d333a4" Sep 11 08:47:05.567667 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:05.567563 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7bqv" podUID="f24c0bea-46aa-432e-a51f-f25451143d07" Sep 11 08:47:05.567667 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:05.567632 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-97r66" podUID="faa7ad4f-7140-43cd-a195-d659bd3bee15" Sep 11 08:47:06.326488 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:06.326420 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e53c87e2-2721-4540-87c6-214593d333a4-original-pull-secret\") pod \"global-pull-secret-syncer-tldcl\" (UID: \"e53c87e2-2721-4540-87c6-214593d333a4\") " pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:47:06.326488 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:06.326456 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faa7ad4f-7140-43cd-a195-d659bd3bee15-metrics-certs\") pod \"network-metrics-daemon-97r66\" (UID: \"faa7ad4f-7140-43cd-a195-d659bd3bee15\") " pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:47:06.326882 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:06.326546 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 08:47:06.326882 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:06.326594 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faa7ad4f-7140-43cd-a195-d659bd3bee15-metrics-certs podName:faa7ad4f-7140-43cd-a195-d659bd3bee15 nodeName:}" failed. No retries permitted until 2026-09-11 08:47:38.326579915 +0000 UTC m=+65.226478192 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/faa7ad4f-7140-43cd-a195-d659bd3bee15-metrics-certs") pod "network-metrics-daemon-97r66" (UID: "faa7ad4f-7140-43cd-a195-d659bd3bee15") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 08:47:06.326882 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:06.326547 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 08:47:06.326882 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:06.326624 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e53c87e2-2721-4540-87c6-214593d333a4-original-pull-secret podName:e53c87e2-2721-4540-87c6-214593d333a4 nodeName:}" failed. No retries permitted until 2026-09-11 08:47:38.326618225 +0000 UTC m=+65.226516501 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e53c87e2-2721-4540-87c6-214593d333a4-original-pull-secret") pod "global-pull-secret-syncer-tldcl" (UID: "e53c87e2-2721-4540-87c6-214593d333a4") : object "kube-system"/"original-pull-secret" not registered Sep 11 08:47:06.427031 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:06.427004 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlkzg\" (UniqueName: \"kubernetes.io/projected/f24c0bea-46aa-432e-a51f-f25451143d07-kube-api-access-zlkzg\") pod \"network-check-target-q7bqv\" (UID: \"f24c0bea-46aa-432e-a51f-f25451143d07\") " pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:47:06.427146 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:06.427123 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 08:47:06.427146 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:06.427138 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 08:47:06.427214 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:06.427148 2562 projected.go:196] Error preparing data for projected volume kube-api-access-zlkzg for pod openshift-network-diagnostics/network-check-target-q7bqv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 08:47:06.427214 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:06.427187 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f24c0bea-46aa-432e-a51f-f25451143d07-kube-api-access-zlkzg podName:f24c0bea-46aa-432e-a51f-f25451143d07 nodeName:}" failed. No retries permitted until 2026-09-11 08:47:38.427175338 +0000 UTC m=+65.327073613 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zlkzg" (UniqueName: "kubernetes.io/projected/f24c0bea-46aa-432e-a51f-f25451143d07-kube-api-access-zlkzg") pod "network-check-target-q7bqv" (UID: "f24c0bea-46aa-432e-a51f-f25451143d07") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 08:47:06.975381 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:06.975344 2562 generic.go:356] "Generic (PLEG): container finished" podID="c62c56b1-a68f-4a3f-a41b-23a2a08e148d" containerID="6679ff5a02d3df29eb11aacd76149f4209649e0e191d5ed96320a45f8dd94a37" exitCode=0 Sep 11 08:47:06.975530 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:06.975391 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wpkr7" event={"ID":"c62c56b1-a68f-4a3f-a41b-23a2a08e148d","Type":"ContainerDied","Data":"6679ff5a02d3df29eb11aacd76149f4209649e0e191d5ed96320a45f8dd94a37"} Sep 11 08:47:07.566691 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.566661 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:47:07.567317 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.566704 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:47:07.567317 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.566662 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:47:07.567317 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:07.566777 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7bqv" podUID="f24c0bea-46aa-432e-a51f-f25451143d07" Sep 11 08:47:07.567317 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:07.566846 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-97r66" podUID="faa7ad4f-7140-43cd-a195-d659bd3bee15" Sep 11 08:47:07.567317 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:07.566937 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tldcl" podUID="e53c87e2-2721-4540-87c6-214593d333a4" Sep 11 08:47:07.838864 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.838795 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-169.ec2.internal" event="NodeReady" Sep 11 08:47:07.839096 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.838903 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 11 08:47:07.888703 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.888670 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6b6b7648d-q99h8"] Sep 11 08:47:07.890827 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.890803 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:07.893818 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.893789 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 11 08:47:07.894059 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.894041 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 11 08:47:07.894499 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.894419 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 11 08:47:07.894499 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.894418 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-l57fw\"" Sep 11 08:47:07.902010 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.900595 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 11 08:47:07.909759 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.909736 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-js7h2"] Sep 11 08:47:07.911797 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.911779 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7c6vh"] Sep 11 08:47:07.911892 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.911869 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-js7h2" Sep 11 08:47:07.913781 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.913760 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7c6vh" Sep 11 08:47:07.915149 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.914943 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 11 08:47:07.915149 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.915000 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 11 08:47:07.915339 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.915262 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-q7bhf\"" Sep 11 08:47:07.916144 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.915918 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 11 08:47:07.916252 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.916198 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 11 08:47:07.916392 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.916276 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 11 08:47:07.916392 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.916350 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7fqkq\"" Sep 11 08:47:07.916564 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.916543 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b6b7648d-q99h8"] Sep 11 08:47:07.927621 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.927598 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7c6vh"] Sep 11 08:47:07.929538 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.929520 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-js7h2"] Sep 11 08:47:07.988638 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:07.988613 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:08.037800 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.037770 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx6c6\" (UniqueName: \"kubernetes.io/projected/0cf62b5d-a365-4686-a79d-9301c5f5297b-kube-api-access-vx6c6\") pod \"dns-default-7c6vh\" (UID: \"0cf62b5d-a365-4686-a79d-9301c5f5297b\") " pod="openshift-dns/dns-default-7c6vh" Sep 11 08:47:08.037926 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.037809 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2f97\" (UniqueName: \"kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-kube-api-access-x2f97\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:08.037926 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.037846 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0cf62b5d-a365-4686-a79d-9301c5f5297b-config-volume\") pod \"dns-default-7c6vh\" (UID: \"0cf62b5d-a365-4686-a79d-9301c5f5297b\") " pod="openshift-dns/dns-default-7c6vh" Sep 11 08:47:08.037926 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.037893 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-tls\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:08.038092 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.037953 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-certificates\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:08.038092 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.037991 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb966255-7c76-47bb-aed7-bd2671e8013a-cert\") pod \"ingress-canary-js7h2\" (UID: \"cb966255-7c76-47bb-aed7-bd2671e8013a\") " pod="openshift-ingress-canary/ingress-canary-js7h2" Sep 11 08:47:08.038092 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.038020 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dclsp\" (UniqueName: \"kubernetes.io/projected/cb966255-7c76-47bb-aed7-bd2671e8013a-kube-api-access-dclsp\") pod \"ingress-canary-js7h2\" (UID: \"cb966255-7c76-47bb-aed7-bd2671e8013a\") " pod="openshift-ingress-canary/ingress-canary-js7h2" Sep 11 08:47:08.038092 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.038043 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0cf62b5d-a365-4686-a79d-9301c5f5297b-metrics-tls\") pod \"dns-default-7c6vh\" (UID: \"0cf62b5d-a365-4686-a79d-9301c5f5297b\") " pod="openshift-dns/dns-default-7c6vh" Sep 11 08:47:08.038269 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.038103 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cf62b5d-a365-4686-a79d-9301c5f5297b-tmp-dir\") pod \"dns-default-7c6vh\" (UID: \"0cf62b5d-a365-4686-a79d-9301c5f5297b\") " pod="openshift-dns/dns-default-7c6vh" Sep 11 08:47:08.038269 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.038166 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-image-registry-private-configuration\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:08.038269 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.038200 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-trusted-ca\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:08.038269 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.038229 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-ca-trust-extracted\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:08.038269 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.038258 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-bound-sa-token\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:08.038477 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.038320 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-installation-pull-secrets\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:08.062933 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.062896 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:08.139583 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.139519 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-ca-trust-extracted\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:08.139583 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.139557 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-bound-sa-token\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:08.139583 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.139579 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-installation-pull-secrets\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:08.139796 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.139611 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx6c6\" (UniqueName: \"kubernetes.io/projected/0cf62b5d-a365-4686-a79d-9301c5f5297b-kube-api-access-vx6c6\") pod \"dns-default-7c6vh\" (UID: \"0cf62b5d-a365-4686-a79d-9301c5f5297b\") " pod="openshift-dns/dns-default-7c6vh" Sep 11 08:47:08.139796 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.139638 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2f97\" (UniqueName: \"kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-kube-api-access-x2f97\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:08.139796 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.139671 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0cf62b5d-a365-4686-a79d-9301c5f5297b-config-volume\") pod \"dns-default-7c6vh\" (UID: \"0cf62b5d-a365-4686-a79d-9301c5f5297b\") " pod="openshift-dns/dns-default-7c6vh" Sep 11 08:47:08.139796 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.139712 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-tls\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:08.139796 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.139742 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-certificates\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:08.139796 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.139767 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb966255-7c76-47bb-aed7-bd2671e8013a-cert\") pod \"ingress-canary-js7h2\" (UID: \"cb966255-7c76-47bb-aed7-bd2671e8013a\") " pod="openshift-ingress-canary/ingress-canary-js7h2" Sep 11 08:47:08.139796 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.139794 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dclsp\" (UniqueName: \"kubernetes.io/projected/cb966255-7c76-47bb-aed7-bd2671e8013a-kube-api-access-dclsp\") pod \"ingress-canary-js7h2\" (UID: \"cb966255-7c76-47bb-aed7-bd2671e8013a\") " pod="openshift-ingress-canary/ingress-canary-js7h2" Sep 11 08:47:08.140125 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.139818 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0cf62b5d-a365-4686-a79d-9301c5f5297b-metrics-tls\") pod \"dns-default-7c6vh\" (UID: \"0cf62b5d-a365-4686-a79d-9301c5f5297b\") " pod="openshift-dns/dns-default-7c6vh" Sep 11 08:47:08.140125 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.139872 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cf62b5d-a365-4686-a79d-9301c5f5297b-tmp-dir\") pod \"dns-default-7c6vh\" (UID: \"0cf62b5d-a365-4686-a79d-9301c5f5297b\") " pod="openshift-dns/dns-default-7c6vh" Sep 11 08:47:08.140125 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:08.139885 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 08:47:08.140125 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:08.139901 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b6b7648d-q99h8: secret "image-registry-tls" not found Sep 11 08:47:08.140125 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.139901 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-ca-trust-extracted\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:08.140125 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.139917 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-image-registry-private-configuration\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:08.140125 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:08.139955 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-tls podName:c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f nodeName:}" failed. No retries permitted until 2026-09-11 08:47:08.639935022 +0000 UTC m=+35.539833313 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-tls") pod "image-registry-6b6b7648d-q99h8" (UID: "c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f") : secret "image-registry-tls" not found Sep 11 08:47:08.140125 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.139975 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-trusted-ca\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:08.140599 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:08.140213 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 08:47:08.140599 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:08.140271 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cf62b5d-a365-4686-a79d-9301c5f5297b-metrics-tls podName:0cf62b5d-a365-4686-a79d-9301c5f5297b nodeName:}" failed. No retries permitted until 2026-09-11 08:47:08.640256817 +0000 UTC m=+35.540155108 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0cf62b5d-a365-4686-a79d-9301c5f5297b-metrics-tls") pod "dns-default-7c6vh" (UID: "0cf62b5d-a365-4686-a79d-9301c5f5297b") : secret "dns-default-metrics-tls" not found Sep 11 08:47:08.140599 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.140582 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cf62b5d-a365-4686-a79d-9301c5f5297b-tmp-dir\") pod \"dns-default-7c6vh\" (UID: \"0cf62b5d-a365-4686-a79d-9301c5f5297b\") " pod="openshift-dns/dns-default-7c6vh" Sep 11 08:47:08.140751 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.140582 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0cf62b5d-a365-4686-a79d-9301c5f5297b-config-volume\") pod \"dns-default-7c6vh\" (UID: \"0cf62b5d-a365-4686-a79d-9301c5f5297b\") " pod="openshift-dns/dns-default-7c6vh" Sep 11 08:47:08.141217 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.141175 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-certificates\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:08.141437 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.141417 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-trusted-ca\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:08.145121 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.145096 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-image-registry-private-configuration\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:08.145220 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.145102 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-installation-pull-secrets\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:08.145220 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.145140 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb966255-7c76-47bb-aed7-bd2671e8013a-cert\") pod \"ingress-canary-js7h2\" (UID: \"cb966255-7c76-47bb-aed7-bd2671e8013a\") " pod="openshift-ingress-canary/ingress-canary-js7h2" Sep 11 08:47:08.148704 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.148660 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-bound-sa-token\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:08.151342 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.150914 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dclsp\" (UniqueName: \"kubernetes.io/projected/cb966255-7c76-47bb-aed7-bd2671e8013a-kube-api-access-dclsp\") pod \"ingress-canary-js7h2\" (UID: \"cb966255-7c76-47bb-aed7-bd2671e8013a\") " pod="openshift-ingress-canary/ingress-canary-js7h2" Sep 11 08:47:08.151342 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.150927 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2f97\" (UniqueName: \"kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-kube-api-access-x2f97\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:08.152014 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.151991 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx6c6\" (UniqueName: \"kubernetes.io/projected/0cf62b5d-a365-4686-a79d-9301c5f5297b-kube-api-access-vx6c6\") pod \"dns-default-7c6vh\" (UID: \"0cf62b5d-a365-4686-a79d-9301c5f5297b\") " pod="openshift-dns/dns-default-7c6vh" Sep 11 08:47:08.161873 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.161853 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:08.224115 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.224080 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-js7h2" Sep 11 08:47:08.512047 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.511824 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-js7h2"] Sep 11 08:47:08.515710 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:47:08.515677 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb966255_7c76_47bb_aed7_bd2671e8013a.slice/crio-764307708add56ed472dbe355ee8998de87b2011c667ed9eeda7f129857ff94f WatchSource:0}: Error finding container 764307708add56ed472dbe355ee8998de87b2011c667ed9eeda7f129857ff94f: Status 404 returned error can't find the container with id 764307708add56ed472dbe355ee8998de87b2011c667ed9eeda7f129857ff94f Sep 11 08:47:08.627714 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.627678 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:08.644652 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.644629 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-tls\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:08.644784 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.644665 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0cf62b5d-a365-4686-a79d-9301c5f5297b-metrics-tls\") pod \"dns-default-7c6vh\" (UID: \"0cf62b5d-a365-4686-a79d-9301c5f5297b\") " pod="openshift-dns/dns-default-7c6vh" Sep 11 08:47:08.644843 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:08.644785 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 08:47:08.644843 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:08.644796 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 08:47:08.644843 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:08.644817 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b6b7648d-q99h8: secret "image-registry-tls" not found Sep 11 08:47:08.644843 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:08.644837 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cf62b5d-a365-4686-a79d-9301c5f5297b-metrics-tls podName:0cf62b5d-a365-4686-a79d-9301c5f5297b nodeName:}" failed. No retries permitted until 2026-09-11 08:47:09.644821592 +0000 UTC m=+36.544719885 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0cf62b5d-a365-4686-a79d-9301c5f5297b-metrics-tls") pod "dns-default-7c6vh" (UID: "0cf62b5d-a365-4686-a79d-9301c5f5297b") : secret "dns-default-metrics-tls" not found Sep 11 08:47:08.645023 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:08.644867 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-tls podName:c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f nodeName:}" failed. No retries permitted until 2026-09-11 08:47:09.644850124 +0000 UTC m=+36.544748415 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-tls") pod "image-registry-6b6b7648d-q99h8" (UID: "c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f") : secret "image-registry-tls" not found Sep 11 08:47:08.727562 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.727521 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:08.981173 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:08.981086 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-js7h2" event={"ID":"cb966255-7c76-47bb-aed7-bd2671e8013a","Type":"ContainerStarted","Data":"764307708add56ed472dbe355ee8998de87b2011c667ed9eeda7f129857ff94f"} Sep 11 08:47:09.124068 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:09.124033 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:09.183472 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:09.183438 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:09.513634 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:09.513562 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:09.567392 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:09.567145 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:47:09.567392 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:09.567347 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:47:09.567631 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:09.567351 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:47:09.570145 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:09.570122 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 11 08:47:09.570328 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:09.570124 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xm9k4\"" Sep 11 08:47:09.570426 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:09.570406 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 11 08:47:09.570491 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:09.570440 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 11 08:47:09.570544 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:09.570526 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wbz7z\"" Sep 11 08:47:09.570600 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:09.570575 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 11 08:47:09.604856 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:09.604831 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:09.651954 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:09.651923 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-tls\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:09.652401 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:09.651967 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0cf62b5d-a365-4686-a79d-9301c5f5297b-metrics-tls\") pod \"dns-default-7c6vh\" (UID: \"0cf62b5d-a365-4686-a79d-9301c5f5297b\") " pod="openshift-dns/dns-default-7c6vh" Sep 11 08:47:09.652401 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:09.652143 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 08:47:09.652401 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:09.652163 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b6b7648d-q99h8: secret "image-registry-tls" not found Sep 11 08:47:09.652401 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:09.652221 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-tls podName:c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f nodeName:}" failed. No retries permitted until 2026-09-11 08:47:11.652201607 +0000 UTC m=+38.552099905 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-tls") pod "image-registry-6b6b7648d-q99h8" (UID: "c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f") : secret "image-registry-tls" not found Sep 11 08:47:09.652592 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:09.652488 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 08:47:09.652592 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:09.652532 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cf62b5d-a365-4686-a79d-9301c5f5297b-metrics-tls podName:0cf62b5d-a365-4686-a79d-9301c5f5297b nodeName:}" failed. No retries permitted until 2026-09-11 08:47:11.652516441 +0000 UTC m=+38.552414732 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0cf62b5d-a365-4686-a79d-9301c5f5297b-metrics-tls") pod "dns-default-7c6vh" (UID: "0cf62b5d-a365-4686-a79d-9301c5f5297b") : secret "dns-default-metrics-tls" not found Sep 11 08:47:09.942868 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:09.942780 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:10.051636 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.051595 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:10.116833 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.116761 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:10.220843 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.220748 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:10.344505 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.344468 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jwpjd"] Sep 11 08:47:10.347651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.347618 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jwpjd" Sep 11 08:47:10.350894 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.350868 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:10.352079 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.352057 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 11 08:47:10.352350 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.352328 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 11 08:47:10.352856 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.352836 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xbkxl\"" Sep 11 08:47:10.356201 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.356177 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 11 08:47:10.365042 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.365021 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 11 08:47:10.367053 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.367032 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jwpjd"] Sep 11 08:47:10.430791 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.430757 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:10.457987 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.457949 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-crio-socket\") pod \"insights-runtime-extractor-jwpjd\" (UID: \"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7\") " pod="openshift-insights/insights-runtime-extractor-jwpjd" Sep 11 08:47:10.457987 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.457975 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jwpjd\" (UID: \"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7\") " pod="openshift-insights/insights-runtime-extractor-jwpjd" Sep 11 08:47:10.458184 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.457996 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jwpjd\" (UID: \"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7\") " pod="openshift-insights/insights-runtime-extractor-jwpjd" Sep 11 08:47:10.458184 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.458063 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6947\" (UniqueName: \"kubernetes.io/projected/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-kube-api-access-t6947\") pod \"insights-runtime-extractor-jwpjd\" (UID: \"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7\") " pod="openshift-insights/insights-runtime-extractor-jwpjd" Sep 11 08:47:10.458184 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.458115 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-data-volume\") pod \"insights-runtime-extractor-jwpjd\" (UID: \"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7\") " pod="openshift-insights/insights-runtime-extractor-jwpjd" Sep 11 08:47:10.558892 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.558820 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-data-volume\") pod \"insights-runtime-extractor-jwpjd\" (UID: \"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7\") " pod="openshift-insights/insights-runtime-extractor-jwpjd" Sep 11 08:47:10.559033 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.558915 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-crio-socket\") pod \"insights-runtime-extractor-jwpjd\" (UID: \"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7\") " pod="openshift-insights/insights-runtime-extractor-jwpjd" Sep 11 08:47:10.559033 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.558945 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jwpjd\" (UID: \"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7\") " pod="openshift-insights/insights-runtime-extractor-jwpjd" Sep 11 08:47:10.559033 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.558970 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jwpjd\" (UID: \"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7\") " pod="openshift-insights/insights-runtime-extractor-jwpjd" Sep 11 08:47:10.559033 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.559010 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6947\" (UniqueName: \"kubernetes.io/projected/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-kube-api-access-t6947\") pod \"insights-runtime-extractor-jwpjd\" (UID: \"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7\") " pod="openshift-insights/insights-runtime-extractor-jwpjd" Sep 11 08:47:10.559323 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:10.559046 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 11 08:47:10.559323 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:10.559143 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-kube-rbac-proxy-cm podName:79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7 nodeName:}" failed. No retries permitted until 2026-09-11 08:47:11.059118382 +0000 UTC m=+37.959016679 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jwpjd" (UID: "79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7") : configmap "kube-rbac-proxy" not found Sep 11 08:47:10.559323 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.559161 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-data-volume\") pod \"insights-runtime-extractor-jwpjd\" (UID: \"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7\") " pod="openshift-insights/insights-runtime-extractor-jwpjd" Sep 11 08:47:10.559323 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.559163 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-crio-socket\") pod \"insights-runtime-extractor-jwpjd\" (UID: \"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7\") " pod="openshift-insights/insights-runtime-extractor-jwpjd" Sep 11 08:47:10.559323 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:10.559182 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 08:47:10.559581 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:10.559358 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-insights-runtime-extractor-tls podName:79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7 nodeName:}" failed. No retries permitted until 2026-09-11 08:47:11.059337074 +0000 UTC m=+37.959235354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jwpjd" (UID: "79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7") : secret "insights-runtime-extractor-tls" not found Sep 11 08:47:10.568448 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:10.568415 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6947\" (UniqueName: \"kubernetes.io/projected/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-kube-api-access-t6947\") pod \"insights-runtime-extractor-jwpjd\" (UID: \"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7\") " pod="openshift-insights/insights-runtime-extractor-jwpjd" Sep 11 08:47:11.059623 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:11.059581 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:11.062616 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:11.062588 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jwpjd\" (UID: \"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7\") " pod="openshift-insights/insights-runtime-extractor-jwpjd" Sep 11 08:47:11.062750 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:11.062631 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jwpjd\" (UID: \"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7\") " pod="openshift-insights/insights-runtime-extractor-jwpjd" Sep 11 08:47:11.062750 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:11.062723 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 08:47:11.062875 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:11.062792 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-insights-runtime-extractor-tls podName:79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7 nodeName:}" failed. No retries permitted until 2026-09-11 08:47:12.06277631 +0000 UTC m=+38.962674601 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jwpjd" (UID: "79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7") : secret "insights-runtime-extractor-tls" not found Sep 11 08:47:11.063180 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:11.063158 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jwpjd\" (UID: \"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7\") " pod="openshift-insights/insights-runtime-extractor-jwpjd" Sep 11 08:47:11.667702 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:11.667664 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-tls\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:11.667873 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:11.667719 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0cf62b5d-a365-4686-a79d-9301c5f5297b-metrics-tls\") pod \"dns-default-7c6vh\" (UID: \"0cf62b5d-a365-4686-a79d-9301c5f5297b\") " pod="openshift-dns/dns-default-7c6vh" Sep 11 08:47:11.667873 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:11.667825 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 08:47:11.667873 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:11.667834 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 08:47:11.667873 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:11.667857 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b6b7648d-q99h8: secret "image-registry-tls" not found Sep 11 08:47:11.668070 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:11.667888 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cf62b5d-a365-4686-a79d-9301c5f5297b-metrics-tls podName:0cf62b5d-a365-4686-a79d-9301c5f5297b nodeName:}" failed. No retries permitted until 2026-09-11 08:47:15.66786966 +0000 UTC m=+42.567767938 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0cf62b5d-a365-4686-a79d-9301c5f5297b-metrics-tls") pod "dns-default-7c6vh" (UID: "0cf62b5d-a365-4686-a79d-9301c5f5297b") : secret "dns-default-metrics-tls" not found Sep 11 08:47:11.668070 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:11.667910 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-tls podName:c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f nodeName:}" failed. No retries permitted until 2026-09-11 08:47:15.667899847 +0000 UTC m=+42.567798131 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-tls") pod "image-registry-6b6b7648d-q99h8" (UID: "c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f") : secret "image-registry-tls" not found Sep 11 08:47:11.818525 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:11.818446 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-8552h"] Sep 11 08:47:11.820110 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:11.820078 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:11.832260 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:11.832236 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-8552h"] Sep 11 08:47:11.832436 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:11.832375 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-8552h" Sep 11 08:47:11.834618 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:11.834563 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-h5m74\"" Sep 11 08:47:11.834618 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:11.834586 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 11 08:47:11.834773 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:11.834644 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 11 08:47:11.834979 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:11.834963 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 11 08:47:11.835051 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:11.835002 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 11 08:47:11.876766 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:11.876724 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:11.970164 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:11.970128 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/94c5491c-2394-444d-b9d9-09ef9fa4ed20-signing-cabundle\") pod \"service-ca-568869fdc5-8552h\" (UID: \"94c5491c-2394-444d-b9d9-09ef9fa4ed20\") " pod="openshift-service-ca/service-ca-568869fdc5-8552h" Sep 11 08:47:11.970387 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:11.970175 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5d2b\" (UniqueName: \"kubernetes.io/projected/94c5491c-2394-444d-b9d9-09ef9fa4ed20-kube-api-access-c5d2b\") pod \"service-ca-568869fdc5-8552h\" (UID: \"94c5491c-2394-444d-b9d9-09ef9fa4ed20\") " pod="openshift-service-ca/service-ca-568869fdc5-8552h" Sep 11 08:47:11.970387 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:11.970285 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/94c5491c-2394-444d-b9d9-09ef9fa4ed20-signing-key\") pod \"service-ca-568869fdc5-8552h\" (UID: \"94c5491c-2394-444d-b9d9-09ef9fa4ed20\") " pod="openshift-service-ca/service-ca-568869fdc5-8552h" Sep 11 08:47:12.071359 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:12.071275 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/94c5491c-2394-444d-b9d9-09ef9fa4ed20-signing-cabundle\") pod \"service-ca-568869fdc5-8552h\" (UID: \"94c5491c-2394-444d-b9d9-09ef9fa4ed20\") " pod="openshift-service-ca/service-ca-568869fdc5-8552h" Sep 11 08:47:12.071359 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:12.071338 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5d2b\" (UniqueName: \"kubernetes.io/projected/94c5491c-2394-444d-b9d9-09ef9fa4ed20-kube-api-access-c5d2b\") pod \"service-ca-568869fdc5-8552h\" (UID: \"94c5491c-2394-444d-b9d9-09ef9fa4ed20\") " pod="openshift-service-ca/service-ca-568869fdc5-8552h" Sep 11 08:47:12.071989 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:12.071390 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/94c5491c-2394-444d-b9d9-09ef9fa4ed20-signing-key\") pod \"service-ca-568869fdc5-8552h\" (UID: \"94c5491c-2394-444d-b9d9-09ef9fa4ed20\") " pod="openshift-service-ca/service-ca-568869fdc5-8552h" Sep 11 08:47:12.071989 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:12.071409 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jwpjd\" (UID: \"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7\") " pod="openshift-insights/insights-runtime-extractor-jwpjd" Sep 11 08:47:12.071989 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:12.071511 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 08:47:12.071989 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:12.071578 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-insights-runtime-extractor-tls podName:79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7 nodeName:}" failed. No retries permitted until 2026-09-11 08:47:14.071559314 +0000 UTC m=+40.971457594 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jwpjd" (UID: "79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7") : secret "insights-runtime-extractor-tls" not found Sep 11 08:47:12.072194 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:12.072171 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/94c5491c-2394-444d-b9d9-09ef9fa4ed20-signing-cabundle\") pod \"service-ca-568869fdc5-8552h\" (UID: \"94c5491c-2394-444d-b9d9-09ef9fa4ed20\") " pod="openshift-service-ca/service-ca-568869fdc5-8552h" Sep 11 08:47:12.073767 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:12.073749 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/94c5491c-2394-444d-b9d9-09ef9fa4ed20-signing-key\") pod \"service-ca-568869fdc5-8552h\" (UID: \"94c5491c-2394-444d-b9d9-09ef9fa4ed20\") " pod="openshift-service-ca/service-ca-568869fdc5-8552h" Sep 11 08:47:12.079725 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:12.079697 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5d2b\" (UniqueName: \"kubernetes.io/projected/94c5491c-2394-444d-b9d9-09ef9fa4ed20-kube-api-access-c5d2b\") pod \"service-ca-568869fdc5-8552h\" (UID: \"94c5491c-2394-444d-b9d9-09ef9fa4ed20\") " pod="openshift-service-ca/service-ca-568869fdc5-8552h" Sep 11 08:47:12.143983 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:12.143961 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-8552h" Sep 11 08:47:12.609531 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:12.609451 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-8552h"] Sep 11 08:47:12.626188 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:47:12.626153 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94c5491c_2394_444d_b9d9_09ef9fa4ed20.slice/crio-2a9891c7d4fd57f03776e8b24fcd9b2f9a3429d2148af32d18080b7436bd9d15 WatchSource:0}: Error finding container 2a9891c7d4fd57f03776e8b24fcd9b2f9a3429d2148af32d18080b7436bd9d15: Status 404 returned error can't find the container with id 2a9891c7d4fd57f03776e8b24fcd9b2f9a3429d2148af32d18080b7436bd9d15 Sep 11 08:47:12.757688 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:12.757635 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:12.993919 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:12.993889 2562 generic.go:356] "Generic (PLEG): container finished" podID="c62c56b1-a68f-4a3f-a41b-23a2a08e148d" containerID="9f4b763c0ae9ddd5563e1df42c54472429b776c5593f4cdb5c1083cdd5cc3d05" exitCode=0 Sep 11 08:47:12.994093 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:12.993976 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wpkr7" event={"ID":"c62c56b1-a68f-4a3f-a41b-23a2a08e148d","Type":"ContainerDied","Data":"9f4b763c0ae9ddd5563e1df42c54472429b776c5593f4cdb5c1083cdd5cc3d05"} Sep 11 08:47:12.995525 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:12.995502 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-js7h2" event={"ID":"cb966255-7c76-47bb-aed7-bd2671e8013a","Type":"ContainerStarted","Data":"e3fb8c5bd0a34811878445df374ca52b442326ccbca47464a5d82eeda1d7ac1a"} Sep 11 08:47:12.996680 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:12.996660 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-8552h" event={"ID":"94c5491c-2394-444d-b9d9-09ef9fa4ed20","Type":"ContainerStarted","Data":"2a9891c7d4fd57f03776e8b24fcd9b2f9a3429d2148af32d18080b7436bd9d15"} Sep 11 08:47:13.037080 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:13.037024 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-js7h2" podStartSLOduration=2.199187759 podStartE2EDuration="6.037003715s" totalImagesPullingTime="3.837815956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:47:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:47:13.036128133 +0000 UTC m=+39.936026500" watchObservedRunningTime="2026-09-11 08:47:13.037003715 +0000 UTC m=+39.936902013" Sep 11 08:47:13.069597 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:13.069574 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cbhgc_4cd3f396-137e-4748-83b5-582ddc810965/dns-node-resolver/0.log" Sep 11 08:47:14.003287 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:14.003252 2562 generic.go:356] "Generic (PLEG): container finished" podID="c62c56b1-a68f-4a3f-a41b-23a2a08e148d" containerID="f3c78b56e8f3bb4254abf325235c355b9d036cf700af848b427d4cefd02babbd" exitCode=0 Sep 11 08:47:14.003718 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:14.003336 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wpkr7" event={"ID":"c62c56b1-a68f-4a3f-a41b-23a2a08e148d","Type":"ContainerDied","Data":"f3c78b56e8f3bb4254abf325235c355b9d036cf700af848b427d4cefd02babbd"} Sep 11 08:47:14.068573 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:14.068546 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9gfk6_395401b2-9748-42b5-b600-4fa1f7369b1e/node-ca/0.log" Sep 11 08:47:14.089164 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:14.089132 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jwpjd\" (UID: \"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7\") " pod="openshift-insights/insights-runtime-extractor-jwpjd" Sep 11 08:47:14.089520 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:14.089414 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 08:47:14.089520 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:14.089478 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-insights-runtime-extractor-tls podName:79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7 nodeName:}" failed. No retries permitted until 2026-09-11 08:47:18.089458199 +0000 UTC m=+44.989356486 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jwpjd" (UID: "79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7") : secret "insights-runtime-extractor-tls" not found Sep 11 08:47:14.198922 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:14.198870 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:14.261850 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:14.261748 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:15.007369 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:15.007331 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-8552h" event={"ID":"94c5491c-2394-444d-b9d9-09ef9fa4ed20","Type":"ContainerStarted","Data":"3267bb4afee7947c426292a13b57f5cb3cdc9d054e03813b1cd1122c47ac91fa"} Sep 11 08:47:15.010831 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:15.010807 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wpkr7" event={"ID":"c62c56b1-a68f-4a3f-a41b-23a2a08e148d","Type":"ContainerStarted","Data":"40f65ac7ee769062b2afcd323f561778264f7f0dedc22bd50098e118751ebf5c"} Sep 11 08:47:15.022738 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:15.022697 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-8552h" podStartSLOduration=1.886406652 podStartE2EDuration="4.022681861s" totalImagesPullingTime="2.136275209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:47:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:47:15.021852061 +0000 UTC m=+41.921750359" watchObservedRunningTime="2026-09-11 08:47:15.022681861 +0000 UTC m=+41.922580156" Sep 11 08:47:15.044824 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:15.042464 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wpkr7" podStartSLOduration=8.160571083 podStartE2EDuration="41.042445214s" totalImagesPullingTime="32.881874131s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 08:46:34 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:47:15.041615027 +0000 UTC m=+41.941513318" watchObservedRunningTime="2026-09-11 08:47:15.042445214 +0000 UTC m=+41.942343512" Sep 11 08:47:15.071051 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:15.071032 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-js7h2_cb966255-7c76-47bb-aed7-bd2671e8013a/serve-healthcheck-canary/0.log" Sep 11 08:47:15.552713 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:15.552686 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:15.699642 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:15.699613 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-tls\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:15.699800 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:15.699654 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0cf62b5d-a365-4686-a79d-9301c5f5297b-metrics-tls\") pod \"dns-default-7c6vh\" (UID: \"0cf62b5d-a365-4686-a79d-9301c5f5297b\") " pod="openshift-dns/dns-default-7c6vh" Sep 11 08:47:15.699800 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:15.699760 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 08:47:15.699800 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:15.699778 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b6b7648d-q99h8: secret "image-registry-tls" not found Sep 11 08:47:15.699800 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:15.699781 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 08:47:15.699953 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:15.699841 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-tls podName:c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f nodeName:}" failed. No retries permitted until 2026-09-11 08:47:23.699824448 +0000 UTC m=+50.599722744 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-tls") pod "image-registry-6b6b7648d-q99h8" (UID: "c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f") : secret "image-registry-tls" not found Sep 11 08:47:15.699953 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:15.699855 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cf62b5d-a365-4686-a79d-9301c5f5297b-metrics-tls podName:0cf62b5d-a365-4686-a79d-9301c5f5297b nodeName:}" failed. No retries permitted until 2026-09-11 08:47:23.699848777 +0000 UTC m=+50.599747053 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0cf62b5d-a365-4686-a79d-9301c5f5297b-metrics-tls") pod "dns-default-7c6vh" (UID: "0cf62b5d-a365-4686-a79d-9301c5f5297b") : secret "dns-default-metrics-tls" not found Sep 11 08:47:16.062651 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:16.062609 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:16.290687 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:16.290656 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:16.395352 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:16.395259 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:16.490559 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:16.490531 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:16.590773 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:16.590747 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:16.772452 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:16.772414 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:17.758105 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:17.758070 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:18.117554 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:18.117473 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jwpjd\" (UID: \"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7\") " pod="openshift-insights/insights-runtime-extractor-jwpjd" Sep 11 08:47:18.117693 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:18.117589 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 08:47:18.117693 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:18.117648 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-insights-runtime-extractor-tls podName:79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7 nodeName:}" failed. No retries permitted until 2026-09-11 08:47:26.117629848 +0000 UTC m=+53.017528136 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jwpjd" (UID: "79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7") : secret "insights-runtime-extractor-tls" not found Sep 11 08:47:18.158634 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:18.158605 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:18.359380 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:18.359344 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:18.558971 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:18.558949 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:18.759238 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:18.759213 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:19.666821 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:19.666772 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:19.767395 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:19.767365 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:19.879912 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:19.879889 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:20.159037 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:20.158950 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:20.360522 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:20.360492 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:20.564658 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:20.564633 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:20.622369 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:20.622338 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:20.763164 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:20.763136 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:21.758169 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:21.758135 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:21.961312 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:21.961284 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:22.329698 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:22.329668 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:22.383367 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:22.383340 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:22.553047 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:22.553017 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:22.957481 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:22.957452 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:23.015913 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:23.015893 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:23.757411 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:23.757380 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-tls\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:23.757565 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:23.757418 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0cf62b5d-a365-4686-a79d-9301c5f5297b-metrics-tls\") pod \"dns-default-7c6vh\" (UID: \"0cf62b5d-a365-4686-a79d-9301c5f5297b\") " pod="openshift-dns/dns-default-7c6vh" Sep 11 08:47:23.759795 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:23.759770 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0cf62b5d-a365-4686-a79d-9301c5f5297b-metrics-tls\") pod \"dns-default-7c6vh\" (UID: \"0cf62b5d-a365-4686-a79d-9301c5f5297b\") " pod="openshift-dns/dns-default-7c6vh" Sep 11 08:47:23.759901 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:23.759845 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-tls\") pod \"image-registry-6b6b7648d-q99h8\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:23.761223 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:23.761201 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:23.802375 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:23.802352 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:23.830489 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:23.830468 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7c6vh" Sep 11 08:47:23.961677 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:23.961649 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:24.106428 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:24.106404 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b6b7648d-q99h8"] Sep 11 08:47:24.126527 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:24.126506 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7c6vh"] Sep 11 08:47:24.128901 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:47:24.128877 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cf62b5d_a365_4686_a79d_9301c5f5297b.slice/crio-2721c0f686169a27a0ca973b0a43970d29798ebd5e962bc8384f1c4a9edb6853 WatchSource:0}: Error finding container 2721c0f686169a27a0ca973b0a43970d29798ebd5e962bc8384f1c4a9edb6853: Status 404 returned error can't find the container with id 2721c0f686169a27a0ca973b0a43970d29798ebd5e962bc8384f1c4a9edb6853 Sep 11 08:47:24.364687 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:24.364613 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:24.573957 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:24.573919 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:24.725485 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:24.725411 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:24.966949 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:24.966912 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:25.039536 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:25.039503 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" event={"ID":"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f","Type":"ContainerStarted","Data":"b2ee8a07b95ff88dc7818668c42cfe898e99e90c3d6b8bc0bbef605d5fe2cfd5"} Sep 11 08:47:25.039692 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:25.039562 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" event={"ID":"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f","Type":"ContainerStarted","Data":"bf31c3aa87a30008e80725d753ebee9aebef143b4b16333787f9e20accb874d6"} Sep 11 08:47:25.039762 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:25.039707 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:25.041151 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:25.041128 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7c6vh" event={"ID":"0cf62b5d-a365-4686-a79d-9301c5f5297b","Type":"ContainerStarted","Data":"2721c0f686169a27a0ca973b0a43970d29798ebd5e962bc8384f1c4a9edb6853"} Sep 11 08:47:25.060400 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:25.060354 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" podStartSLOduration=24.060335723 podStartE2EDuration="24.060335723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:47:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:47:25.059979273 +0000 UTC m=+51.959877573" watchObservedRunningTime="2026-09-11 08:47:25.060335723 +0000 UTC m=+51.960234016" Sep 11 08:47:25.559595 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:25.559560 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:25.958488 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:25.958414 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:26.154010 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:26.153974 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:26.177770 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:26.177733 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jwpjd\" (UID: \"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7\") " pod="openshift-insights/insights-runtime-extractor-jwpjd" Sep 11 08:47:26.180206 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:26.180179 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jwpjd\" (UID: \"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7\") " pod="openshift-insights/insights-runtime-extractor-jwpjd" Sep 11 08:47:26.260362 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:26.260324 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jwpjd" Sep 11 08:47:26.357291 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:26.357247 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:26.613959 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:26.613879 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jwpjd"] Sep 11 08:47:26.639577 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:47:26.639549 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79c653d6_bad2_4ddf_ad33_97c2b4cd2ef7.slice/crio-d75342b700e121f0a8103ca60a20e086c338a3bbef48f4aef28751264620ad08 WatchSource:0}: Error finding container d75342b700e121f0a8103ca60a20e086c338a3bbef48f4aef28751264620ad08: Status 404 returned error can't find the container with id d75342b700e121f0a8103ca60a20e086c338a3bbef48f4aef28751264620ad08 Sep 11 08:47:26.760585 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:26.760563 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:27.049179 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:27.049145 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7c6vh" event={"ID":"0cf62b5d-a365-4686-a79d-9301c5f5297b","Type":"ContainerStarted","Data":"9c500bd9324afce40c80d02199e5d6e246d1037d2a48136aee63cf5c14498bbd"} Sep 11 08:47:27.049340 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:27.049201 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7c6vh" event={"ID":"0cf62b5d-a365-4686-a79d-9301c5f5297b","Type":"ContainerStarted","Data":"220470222c29eeaaa4c7c765bc79b11a499bd67d87720ee30b20e6f79870c597"} Sep 11 08:47:27.049340 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:27.049230 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7c6vh" Sep 11 08:47:27.050614 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:27.050589 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jwpjd" event={"ID":"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7","Type":"ContainerStarted","Data":"ef4c5a637c56b0ceb4555a26e497b80c5342eda038c2a6acffea8527c3b594da"} Sep 11 08:47:27.050713 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:27.050635 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jwpjd" event={"ID":"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7","Type":"ContainerStarted","Data":"d75342b700e121f0a8103ca60a20e086c338a3bbef48f4aef28751264620ad08"} Sep 11 08:47:27.065165 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:27.065121 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7c6vh" podStartSLOduration=17.530463237 podStartE2EDuration="20.065105462s" totalImagesPullingTime="2.534642225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:47:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:47:27.064368931 +0000 UTC m=+53.964267231" watchObservedRunningTime="2026-09-11 08:47:27.065105462 +0000 UTC m=+53.965003760" Sep 11 08:47:28.057596 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:28.057565 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jwpjd" event={"ID":"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7","Type":"ContainerStarted","Data":"aed4b47da0a57fe35f1e8b27c3477e9f99ce94f9910ccfd8340f4cf71172ff41"} Sep 11 08:47:29.062647 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:29.062614 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jwpjd" event={"ID":"79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7","Type":"ContainerStarted","Data":"71f359672d5fd00166a68f7871b848c9502ef8e0befd4240bfcdc177a574d325"} Sep 11 08:47:29.081002 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:29.080946 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jwpjd" podStartSLOduration=16.94145061 podStartE2EDuration="19.080928323s" totalImagesPullingTime="2.139477713s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 08:47:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:47:29.080457901 +0000 UTC m=+55.980356200" watchObservedRunningTime="2026-09-11 08:47:29.080928323 +0000 UTC m=+55.980826626" Sep 11 08:47:32.978575 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:32.978549 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n6jlz" Sep 11 08:47:34.106326 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.106277 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-588858659b-smc86"] Sep 11 08:47:34.140109 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.140082 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4"] Sep 11 08:47:34.140247 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.140191 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-588858659b-smc86" Sep 11 08:47:34.143165 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.143134 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 11 08:47:34.143331 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.143225 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 11 08:47:34.143539 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.143525 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 11 08:47:34.143803 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.143790 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 11 08:47:34.153873 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.153854 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-588858659b-smc86"] Sep 11 08:47:34.153965 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.153882 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4"] Sep 11 08:47:34.153965 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.153948 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" Sep 11 08:47:34.158118 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.158099 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Sep 11 08:47:34.158204 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.158159 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Sep 11 08:47:34.158283 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.158267 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Sep 11 08:47:34.158383 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.158370 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Sep 11 08:47:34.225479 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.225451 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f90272b1-37f2-4869-8b76-c19979dec4c6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-c5bd94d4-g5gr4\" (UID: \"f90272b1-37f2-4869-8b76-c19979dec4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" Sep 11 08:47:34.225597 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.225504 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7kl5\" (UniqueName: \"kubernetes.io/projected/9a91437b-b2d0-4055-a1ab-eabc5252c1c2-kube-api-access-x7kl5\") pod \"klusterlet-addon-workmgr-588858659b-smc86\" (UID: \"9a91437b-b2d0-4055-a1ab-eabc5252c1c2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-588858659b-smc86" Sep 11 08:47:34.225597 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.225559 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f90272b1-37f2-4869-8b76-c19979dec4c6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-c5bd94d4-g5gr4\" (UID: \"f90272b1-37f2-4869-8b76-c19979dec4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" Sep 11 08:47:34.225689 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.225603 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f90272b1-37f2-4869-8b76-c19979dec4c6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-c5bd94d4-g5gr4\" (UID: \"f90272b1-37f2-4869-8b76-c19979dec4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" Sep 11 08:47:34.225689 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.225644 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9a91437b-b2d0-4055-a1ab-eabc5252c1c2-klusterlet-config\") pod \"klusterlet-addon-workmgr-588858659b-smc86\" (UID: \"9a91437b-b2d0-4055-a1ab-eabc5252c1c2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-588858659b-smc86" Sep 11 08:47:34.225786 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.225697 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9a91437b-b2d0-4055-a1ab-eabc5252c1c2-tmp\") pod \"klusterlet-addon-workmgr-588858659b-smc86\" (UID: \"9a91437b-b2d0-4055-a1ab-eabc5252c1c2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-588858659b-smc86" Sep 11 08:47:34.225786 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.225747 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f90272b1-37f2-4869-8b76-c19979dec4c6-ca\") pod \"cluster-proxy-proxy-agent-c5bd94d4-g5gr4\" (UID: \"f90272b1-37f2-4869-8b76-c19979dec4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" Sep 11 08:47:34.225786 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.225766 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vlqf\" (UniqueName: \"kubernetes.io/projected/f90272b1-37f2-4869-8b76-c19979dec4c6-kube-api-access-9vlqf\") pod \"cluster-proxy-proxy-agent-c5bd94d4-g5gr4\" (UID: \"f90272b1-37f2-4869-8b76-c19979dec4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" Sep 11 08:47:34.225888 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.225819 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f90272b1-37f2-4869-8b76-c19979dec4c6-hub\") pod \"cluster-proxy-proxy-agent-c5bd94d4-g5gr4\" (UID: \"f90272b1-37f2-4869-8b76-c19979dec4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" Sep 11 08:47:34.326187 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.326165 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f90272b1-37f2-4869-8b76-c19979dec4c6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-c5bd94d4-g5gr4\" (UID: \"f90272b1-37f2-4869-8b76-c19979dec4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" Sep 11 08:47:34.326292 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.326189 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9a91437b-b2d0-4055-a1ab-eabc5252c1c2-klusterlet-config\") pod \"klusterlet-addon-workmgr-588858659b-smc86\" (UID: \"9a91437b-b2d0-4055-a1ab-eabc5252c1c2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-588858659b-smc86" Sep 11 08:47:34.326292 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.326209 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9a91437b-b2d0-4055-a1ab-eabc5252c1c2-tmp\") pod \"klusterlet-addon-workmgr-588858659b-smc86\" (UID: \"9a91437b-b2d0-4055-a1ab-eabc5252c1c2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-588858659b-smc86" Sep 11 08:47:34.326292 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.326245 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f90272b1-37f2-4869-8b76-c19979dec4c6-ca\") pod \"cluster-proxy-proxy-agent-c5bd94d4-g5gr4\" (UID: \"f90272b1-37f2-4869-8b76-c19979dec4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" Sep 11 08:47:34.326292 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.326272 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vlqf\" (UniqueName: \"kubernetes.io/projected/f90272b1-37f2-4869-8b76-c19979dec4c6-kube-api-access-9vlqf\") pod \"cluster-proxy-proxy-agent-c5bd94d4-g5gr4\" (UID: \"f90272b1-37f2-4869-8b76-c19979dec4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" Sep 11 08:47:34.326546 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.326297 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f90272b1-37f2-4869-8b76-c19979dec4c6-hub\") pod \"cluster-proxy-proxy-agent-c5bd94d4-g5gr4\" (UID: \"f90272b1-37f2-4869-8b76-c19979dec4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" Sep 11 08:47:34.326546 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.326354 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f90272b1-37f2-4869-8b76-c19979dec4c6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-c5bd94d4-g5gr4\" (UID: \"f90272b1-37f2-4869-8b76-c19979dec4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" Sep 11 08:47:34.326546 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.326399 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7kl5\" (UniqueName: \"kubernetes.io/projected/9a91437b-b2d0-4055-a1ab-eabc5252c1c2-kube-api-access-x7kl5\") pod \"klusterlet-addon-workmgr-588858659b-smc86\" (UID: \"9a91437b-b2d0-4055-a1ab-eabc5252c1c2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-588858659b-smc86" Sep 11 08:47:34.326546 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.326442 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f90272b1-37f2-4869-8b76-c19979dec4c6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-c5bd94d4-g5gr4\" (UID: \"f90272b1-37f2-4869-8b76-c19979dec4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" Sep 11 08:47:34.327096 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.327049 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f90272b1-37f2-4869-8b76-c19979dec4c6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-c5bd94d4-g5gr4\" (UID: \"f90272b1-37f2-4869-8b76-c19979dec4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" Sep 11 08:47:34.327227 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.327204 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9a91437b-b2d0-4055-a1ab-eabc5252c1c2-tmp\") pod \"klusterlet-addon-workmgr-588858659b-smc86\" (UID: \"9a91437b-b2d0-4055-a1ab-eabc5252c1c2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-588858659b-smc86" Sep 11 08:47:34.328000 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.327970 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b6b7648d-q99h8"] Sep 11 08:47:34.329282 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.329259 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9a91437b-b2d0-4055-a1ab-eabc5252c1c2-klusterlet-config\") pod \"klusterlet-addon-workmgr-588858659b-smc86\" (UID: \"9a91437b-b2d0-4055-a1ab-eabc5252c1c2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-588858659b-smc86" Sep 11 08:47:34.329415 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.329292 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f90272b1-37f2-4869-8b76-c19979dec4c6-ca\") pod \"cluster-proxy-proxy-agent-c5bd94d4-g5gr4\" (UID: \"f90272b1-37f2-4869-8b76-c19979dec4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" Sep 11 08:47:34.329484 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.329451 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f90272b1-37f2-4869-8b76-c19979dec4c6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-c5bd94d4-g5gr4\" (UID: \"f90272b1-37f2-4869-8b76-c19979dec4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" Sep 11 08:47:34.329697 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.329674 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f90272b1-37f2-4869-8b76-c19979dec4c6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-c5bd94d4-g5gr4\" (UID: \"f90272b1-37f2-4869-8b76-c19979dec4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" Sep 11 08:47:34.329778 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.329697 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f90272b1-37f2-4869-8b76-c19979dec4c6-hub\") pod \"cluster-proxy-proxy-agent-c5bd94d4-g5gr4\" (UID: \"f90272b1-37f2-4869-8b76-c19979dec4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" Sep 11 08:47:34.385726 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.385674 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7kl5\" (UniqueName: \"kubernetes.io/projected/9a91437b-b2d0-4055-a1ab-eabc5252c1c2-kube-api-access-x7kl5\") pod \"klusterlet-addon-workmgr-588858659b-smc86\" (UID: \"9a91437b-b2d0-4055-a1ab-eabc5252c1c2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-588858659b-smc86" Sep 11 08:47:34.407861 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.407836 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vlqf\" (UniqueName: \"kubernetes.io/projected/f90272b1-37f2-4869-8b76-c19979dec4c6-kube-api-access-9vlqf\") pod \"cluster-proxy-proxy-agent-c5bd94d4-g5gr4\" (UID: \"f90272b1-37f2-4869-8b76-c19979dec4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" Sep 11 08:47:34.449801 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.449780 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-588858659b-smc86" Sep 11 08:47:34.469979 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.469913 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" Sep 11 08:47:34.493963 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.493933 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:34.594145 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.594096 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:34.719607 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.719533 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:34.728193 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.728172 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-588858659b-smc86"] Sep 11 08:47:34.731200 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:47:34.731177 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a91437b_b2d0_4055_a1ab_eabc5252c1c2.slice/crio-657307ee3b2924f739987acb1afec487a3b46413d99fb7b227ad7cbcb191cd05 WatchSource:0}: Error finding container 657307ee3b2924f739987acb1afec487a3b46413d99fb7b227ad7cbcb191cd05: Status 404 returned error can't find the container with id 657307ee3b2924f739987acb1afec487a3b46413d99fb7b227ad7cbcb191cd05 Sep 11 08:47:34.758658 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.758633 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4"] Sep 11 08:47:34.761751 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:47:34.761732 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf90272b1_37f2_4869_8b76_c19979dec4c6.slice/crio-676a3b05686579ee18fcb0f29764d28c88c2a8c7fd068d0a4764569ec8d5ae7e WatchSource:0}: Error finding container 676a3b05686579ee18fcb0f29764d28c88c2a8c7fd068d0a4764569ec8d5ae7e: Status 404 returned error can't find the container with id 676a3b05686579ee18fcb0f29764d28c88c2a8c7fd068d0a4764569ec8d5ae7e Sep 11 08:47:34.825364 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.825338 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:34.926618 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:34.926581 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:35.082218 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:35.082187 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-588858659b-smc86" event={"ID":"9a91437b-b2d0-4055-a1ab-eabc5252c1c2","Type":"ContainerStarted","Data":"657307ee3b2924f739987acb1afec487a3b46413d99fb7b227ad7cbcb191cd05"} Sep 11 08:47:35.083344 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:35.083322 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" event={"ID":"f90272b1-37f2-4869-8b76-c19979dec4c6","Type":"ContainerStarted","Data":"676a3b05686579ee18fcb0f29764d28c88c2a8c7fd068d0a4764569ec8d5ae7e"} Sep 11 08:47:35.929504 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:35.929465 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:36.026819 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:36.026787 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:36.113092 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:36.113060 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:36.266958 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:36.266923 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:36.678429 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:36.678353 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:36.832661 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:36.832564 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:37.024016 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:37.023934 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:37.060042 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:37.060017 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7c6vh" Sep 11 08:47:37.126499 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:37.126462 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:37.273850 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:37.273767 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:37.355612 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:37.355500 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:37.992580 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:37.992545 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:38.091566 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:38.091535 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:38.359729 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:38.359648 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faa7ad4f-7140-43cd-a195-d659bd3bee15-metrics-certs\") pod \"network-metrics-daemon-97r66\" (UID: \"faa7ad4f-7140-43cd-a195-d659bd3bee15\") " pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:47:38.359729 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:38.359703 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e53c87e2-2721-4540-87c6-214593d333a4-original-pull-secret\") pod \"global-pull-secret-syncer-tldcl\" (UID: \"e53c87e2-2721-4540-87c6-214593d333a4\") " pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:47:38.362193 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:38.362165 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 11 08:47:38.362323 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:38.362226 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 11 08:47:38.373294 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:38.373272 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e53c87e2-2721-4540-87c6-214593d333a4-original-pull-secret\") pod \"global-pull-secret-syncer-tldcl\" (UID: \"e53c87e2-2721-4540-87c6-214593d333a4\") " pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:47:38.373294 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:38.373290 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faa7ad4f-7140-43cd-a195-d659bd3bee15-metrics-certs\") pod \"network-metrics-daemon-97r66\" (UID: \"faa7ad4f-7140-43cd-a195-d659bd3bee15\") " pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:47:38.383426 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:38.383405 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xm9k4\"" Sep 11 08:47:38.390203 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:38.390173 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tldcl" Sep 11 08:47:38.391737 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:38.391719 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-97r66" Sep 11 08:47:38.460463 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:38.460426 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlkzg\" (UniqueName: \"kubernetes.io/projected/f24c0bea-46aa-432e-a51f-f25451143d07-kube-api-access-zlkzg\") pod \"network-check-target-q7bqv\" (UID: \"f24c0bea-46aa-432e-a51f-f25451143d07\") " pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:47:38.462754 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:38.462729 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 11 08:47:38.473757 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:38.473735 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 11 08:47:38.484847 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:38.484825 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlkzg\" (UniqueName: \"kubernetes.io/projected/f24c0bea-46aa-432e-a51f-f25451143d07-kube-api-access-zlkzg\") pod \"network-check-target-q7bqv\" (UID: \"f24c0bea-46aa-432e-a51f-f25451143d07\") " pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:47:38.699811 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:38.699732 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wbz7z\"" Sep 11 08:47:38.708054 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:38.708029 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:47:40.114280 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:40.114244 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:40.178784 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:40.178756 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:40.285916 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:40.285888 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:40.474975 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:40.474947 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q7bqv"] Sep 11 08:47:40.478202 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:40.478178 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-97r66"] Sep 11 08:47:40.478538 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:47:40.478509 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf24c0bea_46aa_432e_a51f_f25451143d07.slice/crio-e92198155b11a370fa99981edc21846616f37ac8616c017b4fbf6e13b5a79999 WatchSource:0}: Error finding container e92198155b11a370fa99981edc21846616f37ac8616c017b4fbf6e13b5a79999: Status 404 returned error can't find the container with id e92198155b11a370fa99981edc21846616f37ac8616c017b4fbf6e13b5a79999 Sep 11 08:47:40.481428 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:47:40.481399 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaa7ad4f_7140_43cd_a195_d659bd3bee15.slice/crio-734b335af1fac0b20e66ee9429c665424e6d92d3fb939d8315a76c987bce073d WatchSource:0}: Error finding container 734b335af1fac0b20e66ee9429c665424e6d92d3fb939d8315a76c987bce073d: Status 404 returned error can't find the container with id 734b335af1fac0b20e66ee9429c665424e6d92d3fb939d8315a76c987bce073d Sep 11 08:47:40.495662 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:40.495644 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tldcl"] Sep 11 08:47:40.498034 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:47:40.498017 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode53c87e2_2721_4540_87c6_214593d333a4.slice/crio-08bab6e88ee878fdf5c864031e013bbf637013b5bfb95e4f9d441c219888f7d9 WatchSource:0}: Error finding container 08bab6e88ee878fdf5c864031e013bbf637013b5bfb95e4f9d441c219888f7d9: Status 404 returned error can't find the container with id 08bab6e88ee878fdf5c864031e013bbf637013b5bfb95e4f9d441c219888f7d9 Sep 11 08:47:41.108666 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:41.108632 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tldcl" event={"ID":"e53c87e2-2721-4540-87c6-214593d333a4","Type":"ContainerStarted","Data":"08bab6e88ee878fdf5c864031e013bbf637013b5bfb95e4f9d441c219888f7d9"} Sep 11 08:47:41.110364 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:41.110333 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q7bqv" event={"ID":"f24c0bea-46aa-432e-a51f-f25451143d07","Type":"ContainerStarted","Data":"e92198155b11a370fa99981edc21846616f37ac8616c017b4fbf6e13b5a79999"} Sep 11 08:47:41.112283 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:41.112251 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-588858659b-smc86" event={"ID":"9a91437b-b2d0-4055-a1ab-eabc5252c1c2","Type":"ContainerStarted","Data":"05726dacc1b4b85e7a83bb6cdfcf321a9d47c57421e004cee1d9e06872d956fe"} Sep 11 08:47:41.112551 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:41.112501 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-588858659b-smc86" Sep 11 08:47:41.114413 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:41.114394 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-588858659b-smc86" Sep 11 08:47:41.116156 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:41.116124 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" event={"ID":"f90272b1-37f2-4869-8b76-c19979dec4c6","Type":"ContainerStarted","Data":"45229ba91e05910d7fe320ae09291adbdd825347f038cd0062455110733838e1"} Sep 11 08:47:41.116321 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:41.116177 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" event={"ID":"f90272b1-37f2-4869-8b76-c19979dec4c6","Type":"ContainerStarted","Data":"b72cfb3943e0b52b01689e50abce64f1f124bea5e0e2dcd9f574312050fed757"} Sep 11 08:47:41.116321 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:41.116197 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" event={"ID":"f90272b1-37f2-4869-8b76-c19979dec4c6","Type":"ContainerStarted","Data":"a8f0a4ca2dd4a8c31f4c93c9ee2a11acc5ad1b5764d0dff0776db57d9e98cbc3"} Sep 11 08:47:41.118410 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:41.118384 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-97r66" event={"ID":"faa7ad4f-7140-43cd-a195-d659bd3bee15","Type":"ContainerStarted","Data":"734b335af1fac0b20e66ee9429c665424e6d92d3fb939d8315a76c987bce073d"} Sep 11 08:47:41.146424 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:41.146343 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-588858659b-smc86" podStartSLOduration=1.659667245 podStartE2EDuration="7.146324671s" totalImagesPullingTime="5.486657426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:47:41.144749356 +0000 UTC m=+68.044647656" watchObservedRunningTime="2026-09-11 08:47:41.146324671 +0000 UTC m=+68.046222971" Sep 11 08:47:42.124951 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:42.124872 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-97r66" event={"ID":"faa7ad4f-7140-43cd-a195-d659bd3bee15","Type":"ContainerStarted","Data":"3a7c8f03d2e0fa9d0a80e7ce5a75242a17bd8cf865f2f4b98bce463095107f0e"} Sep 11 08:47:42.124951 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:42.124944 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-97r66" event={"ID":"faa7ad4f-7140-43cd-a195-d659bd3bee15","Type":"ContainerStarted","Data":"dc51b1536541435fadafad486ec6c47dbb7334592bbb4ea5d956680db4fd5173"} Sep 11 08:47:42.154196 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:42.151380 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-c5bd94d4-g5gr4" podStartSLOduration=2.6932088480000003 podStartE2EDuration="8.151350024s" totalImagesPullingTime="5.458141176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:47:41.234280983 +0000 UTC m=+68.134179284" watchObservedRunningTime="2026-09-11 08:47:42.151350024 +0000 UTC m=+69.051248322" Sep 11 08:47:42.154196 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:42.152499 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-97r66" podStartSLOduration=67.142052115 podStartE2EDuration="1m8.152484013s" totalImagesPullingTime="1.010431898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:46:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:47:42.148635374 +0000 UTC m=+69.048533664" watchObservedRunningTime="2026-09-11 08:47:42.152484013 +0000 UTC m=+69.052382313" Sep 11 08:47:42.658115 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:42.658075 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:42.717685 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:42.717653 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:43.685358 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:43.685258 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:43.756141 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:43.756097 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:44.278743 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.278708 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:44.335325 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.335269 2562 patch_prober.go:28] interesting pod/image-registry-6b6b7648d-q99h8 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 11 08:47:44.335488 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.335346 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" podUID="c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 08:47:44.486466 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.486398 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:44.546326 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.546240 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-w6llw"] Sep 11 08:47:44.550104 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.550075 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.552172 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.552150 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 11 08:47:44.552297 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.552150 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 11 08:47:44.552447 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.552431 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 11 08:47:44.552967 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.552634 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2w95z\"" Sep 11 08:47:44.552967 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.552684 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 11 08:47:44.552967 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.552694 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 11 08:47:44.552967 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.552737 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 11 08:47:44.582749 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.582723 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:44.605766 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.605742 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3bf195fe-d614-4941-b271-d8821fc90554-node-exporter-textfile\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.605886 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.605780 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3bf195fe-d614-4941-b271-d8821fc90554-node-exporter-wtmp\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.605886 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.605821 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3bf195fe-d614-4941-b271-d8821fc90554-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.605886 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.605858 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3bf195fe-d614-4941-b271-d8821fc90554-node-exporter-accelerators-collector-config\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.605886 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.605883 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84vjr\" (UniqueName: \"kubernetes.io/projected/3bf195fe-d614-4941-b271-d8821fc90554-kube-api-access-84vjr\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.606153 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.605911 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3bf195fe-d614-4941-b271-d8821fc90554-sys\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.606153 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.605935 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3bf195fe-d614-4941-b271-d8821fc90554-metrics-client-ca\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.606153 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.605964 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3bf195fe-d614-4941-b271-d8821fc90554-root\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.606153 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.605985 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3bf195fe-d614-4941-b271-d8821fc90554-node-exporter-tls\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.655892 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.655857 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:44.706925 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.706893 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3bf195fe-d614-4941-b271-d8821fc90554-root\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.707338 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.706928 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3bf195fe-d614-4941-b271-d8821fc90554-node-exporter-tls\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.707338 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.707011 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3bf195fe-d614-4941-b271-d8821fc90554-root\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.707338 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:44.707042 2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 11 08:47:44.707338 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.707064 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3bf195fe-d614-4941-b271-d8821fc90554-node-exporter-textfile\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.707338 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:44.707108 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bf195fe-d614-4941-b271-d8821fc90554-node-exporter-tls podName:3bf195fe-d614-4941-b271-d8821fc90554 nodeName:}" failed. No retries permitted until 2026-09-11 08:47:45.207086746 +0000 UTC m=+72.106985028 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3bf195fe-d614-4941-b271-d8821fc90554-node-exporter-tls") pod "node-exporter-w6llw" (UID: "3bf195fe-d614-4941-b271-d8821fc90554") : secret "node-exporter-tls" not found Sep 11 08:47:44.707338 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.707124 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3bf195fe-d614-4941-b271-d8821fc90554-node-exporter-wtmp\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.707338 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.707173 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3bf195fe-d614-4941-b271-d8821fc90554-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.707338 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.707224 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3bf195fe-d614-4941-b271-d8821fc90554-node-exporter-accelerators-collector-config\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.707338 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.707249 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84vjr\" (UniqueName: \"kubernetes.io/projected/3bf195fe-d614-4941-b271-d8821fc90554-kube-api-access-84vjr\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.707338 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.707282 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3bf195fe-d614-4941-b271-d8821fc90554-sys\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.707338 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.707324 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3bf195fe-d614-4941-b271-d8821fc90554-metrics-client-ca\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.707872 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.707436 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3bf195fe-d614-4941-b271-d8821fc90554-node-exporter-textfile\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.707872 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.707806 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3bf195fe-d614-4941-b271-d8821fc90554-node-exporter-accelerators-collector-config\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.707872 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.707852 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3bf195fe-d614-4941-b271-d8821fc90554-node-exporter-wtmp\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.708001 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.707880 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3bf195fe-d614-4941-b271-d8821fc90554-metrics-client-ca\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.708001 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.707908 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3bf195fe-d614-4941-b271-d8821fc90554-sys\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.710188 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.710168 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3bf195fe-d614-4941-b271-d8821fc90554-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.716346 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.716325 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84vjr\" (UniqueName: \"kubernetes.io/projected/3bf195fe-d614-4941-b271-d8821fc90554-kube-api-access-84vjr\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:44.787662 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.787623 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:44.889286 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:44.889210 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:45.135418 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:45.135385 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:45.211430 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:45.211348 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3bf195fe-d614-4941-b271-d8821fc90554-node-exporter-tls\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:45.211587 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:45.211538 2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 11 08:47:45.211653 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:47:45.211598 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bf195fe-d614-4941-b271-d8821fc90554-node-exporter-tls podName:3bf195fe-d614-4941-b271-d8821fc90554 nodeName:}" failed. No retries permitted until 2026-09-11 08:47:46.211579393 +0000 UTC m=+73.111477683 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3bf195fe-d614-4941-b271-d8821fc90554-node-exporter-tls") pod "node-exporter-w6llw" (UID: "3bf195fe-d614-4941-b271-d8821fc90554") : secret "node-exporter-tls" not found Sep 11 08:47:45.295460 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:45.295421 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:46.141427 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:46.141389 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tldcl" event={"ID":"e53c87e2-2721-4540-87c6-214593d333a4","Type":"ContainerStarted","Data":"3f205618d20b18fd565491af50992774e1e37fa7933524bdfce0c8ba33897881"} Sep 11 08:47:46.143222 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:46.143190 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q7bqv" event={"ID":"f24c0bea-46aa-432e-a51f-f25451143d07","Type":"ContainerStarted","Data":"26c74bc1d09059aab9807ebec3801b37e7bd3b89a60d99ba0be213ad7652a858"} Sep 11 08:47:46.143388 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:46.143245 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:47:46.158568 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:46.158521 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tldcl" podStartSLOduration=67.215235032 podStartE2EDuration="1m12.158506167s" totalImagesPullingTime="4.943271135s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 08:46:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:47:46.157413469 +0000 UTC m=+73.057311768" watchObservedRunningTime="2026-09-11 08:47:46.158506167 +0000 UTC m=+73.058404465" Sep 11 08:47:46.175080 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:46.175042 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-q7bqv" podStartSLOduration=67.212359869 podStartE2EDuration="1m12.175029741s" totalImagesPullingTime="4.962669872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:46:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:47:46.173629681 +0000 UTC m=+73.073527979" watchObservedRunningTime="2026-09-11 08:47:46.175029741 +0000 UTC m=+73.074928039" Sep 11 08:47:46.218534 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:46.218509 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3bf195fe-d614-4941-b271-d8821fc90554-node-exporter-tls\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:46.220976 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:46.220956 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3bf195fe-d614-4941-b271-d8821fc90554-node-exporter-tls\") pod \"node-exporter-w6llw\" (UID: \"3bf195fe-d614-4941-b271-d8821fc90554\") " pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:46.241278 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:46.241258 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:46.342997 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:46.342956 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:46.362419 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:46.362400 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-w6llw" Sep 11 08:47:46.395315 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:47:46.395284 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bf195fe_d614_4941_b271_d8821fc90554.slice/crio-d1019835298cf1c5cd823427189777df2f6eb76eedec731d253f15b0a107bd54 WatchSource:0}: Error finding container d1019835298cf1c5cd823427189777df2f6eb76eedec731d253f15b0a107bd54: Status 404 returned error can't find the container with id d1019835298cf1c5cd823427189777df2f6eb76eedec731d253f15b0a107bd54 Sep 11 08:47:47.150320 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:47.150265 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w6llw" event={"ID":"3bf195fe-d614-4941-b271-d8821fc90554","Type":"ContainerStarted","Data":"d1019835298cf1c5cd823427189777df2f6eb76eedec731d253f15b0a107bd54"} Sep 11 08:47:47.473704 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:47.473618 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:47.576732 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:47.576670 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:47.673623 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:47.673596 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:47.775728 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:47.775701 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:47.876146 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:47.876126 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:47.976943 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:47.976923 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:48.077109 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:48.077056 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:48.155421 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:48.155393 2562 generic.go:356] "Generic (PLEG): container finished" podID="3bf195fe-d614-4941-b271-d8821fc90554" containerID="abb6a6e5a3f947e86a366dc34465b337d58f6b088e95dee9ccff181d06650290" exitCode=0 Sep 11 08:47:48.155689 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:48.155464 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w6llw" event={"ID":"3bf195fe-d614-4941-b271-d8821fc90554","Type":"ContainerDied","Data":"abb6a6e5a3f947e86a366dc34465b337d58f6b088e95dee9ccff181d06650290"} Sep 11 08:47:48.176754 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:48.176732 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:48.277825 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:48.277802 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:48.381096 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:48.381071 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:48.763209 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:48.763183 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:48.862414 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:48.862388 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:48.964417 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:48.964392 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:49.161185 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:49.161125 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w6llw" event={"ID":"3bf195fe-d614-4941-b271-d8821fc90554","Type":"ContainerStarted","Data":"834ad2e319e58824e0447f793bcd631fd95589b2455086801f65b074e1dff482"} Sep 11 08:47:49.161185 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:49.161175 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-w6llw" event={"ID":"3bf195fe-d614-4941-b271-d8821fc90554","Type":"ContainerStarted","Data":"e9b092ad424d256747b66824efc93a7c4be7a86ec47f611a45ae14b2a91bdfcd"} Sep 11 08:47:49.186496 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:49.186434 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-w6llw" podStartSLOduration=3.588794766 podStartE2EDuration="5.186416247s" totalImagesPullingTime="1.50717378s" totalInitContainerRuntime="90.447701ms" isStatefulPod=true podCreationTimestamp="2026-09-11 08:47:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:47:49.183804545 +0000 UTC m=+76.083702842" watchObservedRunningTime="2026-09-11 08:47:49.186416247 +0000 UTC m=+76.086314560" Sep 11 08:47:49.300883 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:49.300853 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:49.395667 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:49.395637 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:49.692384 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:49.692354 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:49.797020 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:49.796997 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:49.943598 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:49.943542 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:50.140916 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:50.140889 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:50.340407 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:50.340373 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:50.626984 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:50.626914 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:51.503228 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:51.503194 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:51.603070 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:51.603040 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:51.829684 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:51.829617 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:51.928318 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:51.928284 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:52.700656 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:52.700608 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:52.982577 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:52.982538 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:53.080701 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:53.080663 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:54.332937 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:54.332909 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:55.981360 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:55.981319 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:56.044364 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:56.044337 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:57.680681 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:57.680651 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:57.783890 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:57.783856 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:57.885399 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:57.885373 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:59.346223 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.346188 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" podUID="c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f" containerName="registry" containerID="cri-o://b2ee8a07b95ff88dc7818668c42cfe898e99e90c3d6b8bc0bbef605d5fe2cfd5" gracePeriod=30 Sep 11 08:47:59.600647 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.600596 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:47:59.715472 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.715442 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-image-registry-private-configuration\") pod \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " Sep 11 08:47:59.715674 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.715657 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-installation-pull-secrets\") pod \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " Sep 11 08:47:59.715809 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.715796 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-certificates\") pod \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " Sep 11 08:47:59.715913 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.715901 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-trusted-ca\") pod \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " Sep 11 08:47:59.716011 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.715999 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-ca-trust-extracted\") pod \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " Sep 11 08:47:59.716116 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.716104 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-tls\") pod \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " Sep 11 08:47:59.716216 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.716205 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-bound-sa-token\") pod \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " Sep 11 08:47:59.716356 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.716342 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2f97\" (UniqueName: \"kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-kube-api-access-x2f97\") pod \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\" (UID: \"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f\") " Sep 11 08:47:59.719371 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.719334 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f" (UID: "c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:47:59.719921 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.719630 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-kube-api-access-x2f97" (OuterVolumeSpecName: "kube-api-access-x2f97") pod "c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f" (UID: "c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f"). InnerVolumeSpecName "kube-api-access-x2f97". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 08:47:59.719921 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.719680 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f" (UID: "c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 08:47:59.721582 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.721557 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f" (UID: "c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:47:59.721660 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.721561 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f" (UID: "c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 08:47:59.721716 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.721646 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f" (UID: "c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 08:47:59.721861 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.721841 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f" (UID: "c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 08:47:59.727749 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.727724 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f" (UID: "c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 08:47:59.764688 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.764665 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:47:59.817887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.817865 2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-installation-pull-secrets\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 08:47:59.817887 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.817885 2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-certificates\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 08:47:59.817993 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.817894 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-trusted-ca\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 08:47:59.817993 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.817902 2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-ca-trust-extracted\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 08:47:59.817993 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.817910 2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-registry-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 08:47:59.817993 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.817918 2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-bound-sa-token\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 08:47:59.817993 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.817926 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2f97\" (UniqueName: \"kubernetes.io/projected/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-kube-api-access-x2f97\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 08:47:59.817993 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.817934 2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f-image-registry-private-configuration\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 08:47:59.828840 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:47:59.828816 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:48:00.197679 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:00.197650 2562 generic.go:356] "Generic (PLEG): container finished" podID="c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f" containerID="b2ee8a07b95ff88dc7818668c42cfe898e99e90c3d6b8bc0bbef605d5fe2cfd5" exitCode=0 Sep 11 08:48:00.197844 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:00.197693 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" event={"ID":"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f","Type":"ContainerDied","Data":"b2ee8a07b95ff88dc7818668c42cfe898e99e90c3d6b8bc0bbef605d5fe2cfd5"} Sep 11 08:48:00.197844 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:00.197705 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" Sep 11 08:48:00.197844 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:00.197762 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b6b7648d-q99h8" event={"ID":"c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f","Type":"ContainerDied","Data":"bf31c3aa87a30008e80725d753ebee9aebef143b4b16333787f9e20accb874d6"} Sep 11 08:48:00.198661 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:00.198646 2562 scope.go:117] "RemoveContainer" containerID="b2ee8a07b95ff88dc7818668c42cfe898e99e90c3d6b8bc0bbef605d5fe2cfd5" Sep 11 08:48:00.207770 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:00.207758 2562 scope.go:117] "RemoveContainer" containerID="b2ee8a07b95ff88dc7818668c42cfe898e99e90c3d6b8bc0bbef605d5fe2cfd5" Sep 11 08:48:00.208052 ip-10-0-141-169 kubenswrapper[2562]: E0911 08:48:00.208032 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2ee8a07b95ff88dc7818668c42cfe898e99e90c3d6b8bc0bbef605d5fe2cfd5\": container with ID starting with b2ee8a07b95ff88dc7818668c42cfe898e99e90c3d6b8bc0bbef605d5fe2cfd5 not found: ID does not exist" containerID="b2ee8a07b95ff88dc7818668c42cfe898e99e90c3d6b8bc0bbef605d5fe2cfd5" Sep 11 08:48:00.208089 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:00.208061 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2ee8a07b95ff88dc7818668c42cfe898e99e90c3d6b8bc0bbef605d5fe2cfd5"} err="failed to get container status \"b2ee8a07b95ff88dc7818668c42cfe898e99e90c3d6b8bc0bbef605d5fe2cfd5\": rpc error: code = NotFound desc = could not find container \"b2ee8a07b95ff88dc7818668c42cfe898e99e90c3d6b8bc0bbef605d5fe2cfd5\": container with ID starting with b2ee8a07b95ff88dc7818668c42cfe898e99e90c3d6b8bc0bbef605d5fe2cfd5 not found: ID does not exist" Sep 11 08:48:00.217386 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:00.217367 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b6b7648d-q99h8"] Sep 11 08:48:00.221225 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:00.221205 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6b6b7648d-q99h8"] Sep 11 08:48:01.570984 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:01.570944 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f" path="/var/lib/kubelet/pods/c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f/volumes" Sep 11 08:48:02.370288 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:02.370260 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:48:02.471437 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:02.471408 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:48:04.129142 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:04.129086 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:48:04.188150 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:04.188120 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:48:04.769590 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:04.769562 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:48:04.870170 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:04.870123 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:48:09.092049 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:09.092017 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:48:09.163596 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:09.163575 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:48:13.099746 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:13.099718 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:48:13.161193 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:13.161161 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:48:14.008610 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:14.008576 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:48:14.109763 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:14.109732 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:48:14.211225 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:14.211194 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:48:14.344255 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:14.344178 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:48:15.594218 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:15.594187 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:48:15.694713 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:15.694690 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:48:15.749119 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:15.749094 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:48:17.153424 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:17.153400 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-q7bqv" Sep 11 08:48:17.860626 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:17.860596 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:48:17.929009 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:17.928980 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:48:37.967348 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:37.967292 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:48:38.068142 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:38.068113 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:48:47.920747 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:47.920712 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:48:47.986441 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:48:47.986411 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:49:03.761377 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:49:03.761346 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:49:04.436425 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:49:04.436383 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:49:13.245003 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:49:13.244968 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:49:13.387198 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:49:13.387149 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:49:13.442573 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:49:13.442537 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:49:14.568470 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:49:14.568420 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:49:14.646463 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:49:14.646436 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:49:24.272639 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:49:24.272611 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:49:24.373040 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:49:24.373009 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:49:24.474229 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:49:24.474198 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:49:50.477573 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:49:50.477504 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:49:57.696519 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:49:57.696485 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:49:57.757714 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:49:57.757686 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:50:18.994617 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:50:18.994583 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:50:19.095813 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:50:19.095784 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:50:30.918146 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:50:30.918113 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:50:30.979398 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:50:30.979373 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:50:37.978673 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:50:37.978632 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:50:38.079491 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:50:38.079465 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:50:46.362542 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:50:46.362502 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:50:48.222472 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:50:48.222407 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:50:48.292736 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:50:48.292708 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:50:48.376072 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:50:48.376046 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:50:48.496418 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:50:48.496387 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:50:49.083745 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:50:49.083715 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:50:58.646543 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:50:58.646435 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:50:58.698242 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:50:58.698207 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:51:33.480518 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:51:33.480488 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/ovn-acl-logging/0.log" Sep 11 08:51:33.482956 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:51:33.482935 2562 kubelet.go:1648] "Image garbage collection succeeded" Sep 11 08:51:33.483722 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:51:33.483706 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/ovn-acl-logging/0.log" Sep 11 08:52:40.286665 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:52:40.286584 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:52:41.835516 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:52:41.835485 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:52:59.185359 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:52:59.185328 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:53:01.676012 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:53:01.675975 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:53:01.737310 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:53:01.737281 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:53:10.226151 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:53:10.226119 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:53:10.327048 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:53:10.327017 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:53:18.168077 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:53:18.168040 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:53:18.266406 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:53:18.266380 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:53:18.368156 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:53:18.368129 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:53:18.761965 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:53:18.761934 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:53:18.860862 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:53:18.860806 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:53:19.110545 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:53:19.110460 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:53:19.237354 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:53:19.237323 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:53:19.394614 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:53:19.394542 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:53:19.636644 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:53:19.636619 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:54:05.653041 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:54:05.652963 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:55:07.139794 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:55:07.139760 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:55:07.204719 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:55:07.204686 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:55:07.403072 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:55:07.402991 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:55:07.505663 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:55:07.505634 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:55:07.604185 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:55:07.604150 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:55:07.705233 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:55:07.705164 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:55:07.803432 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:55:07.803405 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:55:08.063828 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:55:08.063808 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:55:29.967939 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:55:29.967869 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:56:31.917179 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:56:31.917143 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:56:31.976799 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:56:31.976772 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:56:32.367701 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:56:32.367639 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:56:32.473079 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:56:32.473045 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:56:32.648250 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:56:32.648167 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:56:32.747483 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:56:32.747456 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:56:32.848463 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:56:32.848423 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:56:33.501192 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:56:33.501162 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/ovn-acl-logging/0.log" Sep 11 08:56:33.501608 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:56:33.501249 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/ovn-acl-logging/0.log" Sep 11 08:57:06.482193 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:57:06.482119 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:57:13.381958 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:57:13.381929 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:57:52.337112 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:57:52.337080 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:58:04.481104 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:04.481069 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:58:14.777132 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:14.777068 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:58:14.907523 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:14.907485 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:58:15.014382 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:15.014345 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:58:15.123581 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:15.123512 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:58:15.446097 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:15.446027 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:58:20.064764 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:20.064728 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:58:57.559151 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:57.559072 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:58:57.798976 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:57.798945 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-69bbd798b6-zstb2"] Sep 11 08:58:57.799204 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:57.799187 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f" containerName="registry" Sep 11 08:58:57.799280 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:57.799208 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f" containerName="registry" Sep 11 08:58:57.799374 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:57.799280 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c52a4b71-b0ca-4350-a61b-ca7adc4fdb0f" containerName="registry" Sep 11 08:58:57.801279 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:57.801254 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-zstb2" Sep 11 08:58:57.803366 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:57.803342 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:58:57.803515 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:57.803494 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 11 08:58:57.803590 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:57.803535 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 11 08:58:57.803590 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:57.803550 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-kqd8r\"" Sep 11 08:58:57.809976 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:57.809930 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-zstb2"] Sep 11 08:58:57.876605 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:57.876571 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:58:57.898595 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:57.898575 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6gvv\" (UniqueName: \"kubernetes.io/projected/e0ac5480-9e59-415d-a8f4-7c391468e2b4-kube-api-access-s6gvv\") pod \"redis-69bbd798b6-zstb2\" (UID: \"e0ac5480-9e59-415d-a8f4-7c391468e2b4\") " pod="test-ns-feast/redis-69bbd798b6-zstb2" Sep 11 08:58:57.999294 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:57.999272 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6gvv\" (UniqueName: \"kubernetes.io/projected/e0ac5480-9e59-415d-a8f4-7c391468e2b4-kube-api-access-s6gvv\") pod \"redis-69bbd798b6-zstb2\" (UID: \"e0ac5480-9e59-415d-a8f4-7c391468e2b4\") " pod="test-ns-feast/redis-69bbd798b6-zstb2" Sep 11 08:58:58.007187 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:58.007158 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6gvv\" (UniqueName: \"kubernetes.io/projected/e0ac5480-9e59-415d-a8f4-7c391468e2b4-kube-api-access-s6gvv\") pod \"redis-69bbd798b6-zstb2\" (UID: \"e0ac5480-9e59-415d-a8f4-7c391468e2b4\") " pod="test-ns-feast/redis-69bbd798b6-zstb2" Sep 11 08:58:58.110209 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:58.110149 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-zstb2" Sep 11 08:58:58.373458 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:58.373391 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-zstb2"] Sep 11 08:58:58.376665 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:58:58.376637 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0ac5480_9e59_415d_a8f4_7c391468e2b4.slice/crio-01b99f4c7eb0465ef42919fc20322f2ece8fd9a77b6a1ff6d8f6eceeeb5bd812 WatchSource:0}: Error finding container 01b99f4c7eb0465ef42919fc20322f2ece8fd9a77b6a1ff6d8f6eceeeb5bd812: Status 404 returned error can't find the container with id 01b99f4c7eb0465ef42919fc20322f2ece8fd9a77b6a1ff6d8f6eceeeb5bd812 Sep 11 08:58:58.378866 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:58.378849 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 11 08:58:59.300722 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:58:59.300685 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-zstb2" event={"ID":"e0ac5480-9e59-415d-a8f4-7c391468e2b4","Type":"ContainerStarted","Data":"01b99f4c7eb0465ef42919fc20322f2ece8fd9a77b6a1ff6d8f6eceeeb5bd812"} Sep 11 08:59:00.305033 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:00.304994 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-zstb2" event={"ID":"e0ac5480-9e59-415d-a8f4-7c391468e2b4","Type":"ContainerStarted","Data":"2b9337c524d039b7ac23fdfb2ade8f76df54acc3b3496e88a2d740b1d4e541cb"} Sep 11 08:59:00.322981 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:00.320187 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/redis-69bbd798b6-zstb2" podStartSLOduration=1.892151707 podStartE2EDuration="3.320168957s" totalImagesPullingTime="1.42801725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:58:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:59:00.317984037 +0000 UTC m=+747.217882338" watchObservedRunningTime="2026-09-11 08:59:00.320168957 +0000 UTC m=+747.220067256" Sep 11 08:59:06.477230 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:06.477196 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:59:06.578464 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:06.578420 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:59:06.684390 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:06.684356 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:59:06.792202 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:06.792127 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:59:06.892370 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:06.892342 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:59:13.715679 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:13.715647 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:59:31.825492 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:31.825453 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 08:59:33.237739 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:33.237711 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-test-apply-gc48h"] Sep 11 08:59:33.240685 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:33.240669 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-gc48h" Sep 11 08:59:33.242546 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:33.242524 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-cronjob-dockercfg-sdrhw\"" Sep 11 08:59:33.242771 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:33.242753 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 11 08:59:33.247523 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:33.247500 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-gc48h"] Sep 11 08:59:33.314059 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:33.314037 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cln79\" (UniqueName: \"kubernetes.io/projected/a1369ac2-f702-4d53-84bd-2b47c17d39ca-kube-api-access-cln79\") pod \"feast-test-apply-gc48h\" (UID: \"a1369ac2-f702-4d53-84bd-2b47c17d39ca\") " pod="test-ns-feast/feast-test-apply-gc48h" Sep 11 08:59:33.414408 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:33.414386 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cln79\" (UniqueName: \"kubernetes.io/projected/a1369ac2-f702-4d53-84bd-2b47c17d39ca-kube-api-access-cln79\") pod \"feast-test-apply-gc48h\" (UID: \"a1369ac2-f702-4d53-84bd-2b47c17d39ca\") " pod="test-ns-feast/feast-test-apply-gc48h" Sep 11 08:59:33.422219 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:33.422196 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cln79\" (UniqueName: \"kubernetes.io/projected/a1369ac2-f702-4d53-84bd-2b47c17d39ca-kube-api-access-cln79\") pod \"feast-test-apply-gc48h\" (UID: \"a1369ac2-f702-4d53-84bd-2b47c17d39ca\") " pod="test-ns-feast/feast-test-apply-gc48h" Sep 11 08:59:33.551684 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:33.551629 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-cronjob-dockercfg-sdrhw\"" Sep 11 08:59:33.560437 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:33.560410 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-gc48h" Sep 11 08:59:33.826204 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:33.826124 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-gc48h"] Sep 11 08:59:33.828760 ip-10-0-141-169 kubenswrapper[2562]: W0911 08:59:33.828735 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1369ac2_f702_4d53_84bd_2b47c17d39ca.slice/crio-edcc80c08bcbadf979e4d80489d1213b35aa0dc77053ed2eb14aba2d12fdd1cb WatchSource:0}: Error finding container edcc80c08bcbadf979e4d80489d1213b35aa0dc77053ed2eb14aba2d12fdd1cb: Status 404 returned error can't find the container with id edcc80c08bcbadf979e4d80489d1213b35aa0dc77053ed2eb14aba2d12fdd1cb Sep 11 08:59:34.414504 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:34.414468 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-gc48h" event={"ID":"a1369ac2-f702-4d53-84bd-2b47c17d39ca","Type":"ContainerStarted","Data":"edcc80c08bcbadf979e4d80489d1213b35aa0dc77053ed2eb14aba2d12fdd1cb"} Sep 11 08:59:38.451921 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:38.451903 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 11 08:59:39.435491 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:39.435458 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-gc48h" event={"ID":"a1369ac2-f702-4d53-84bd-2b47c17d39ca","Type":"ContainerStarted","Data":"6bd2be973d8f36be7ac3919620d6b5b432349236b0415c8327c1907bcc7a64bc"} Sep 11 08:59:45.457910 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:45.457874 2562 generic.go:356] "Generic (PLEG): container finished" podID="a1369ac2-f702-4d53-84bd-2b47c17d39ca" containerID="6bd2be973d8f36be7ac3919620d6b5b432349236b0415c8327c1907bcc7a64bc" exitCode=0 Sep 11 08:59:45.458288 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:45.457949 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-gc48h" event={"ID":"a1369ac2-f702-4d53-84bd-2b47c17d39ca","Type":"ContainerDied","Data":"6bd2be973d8f36be7ac3919620d6b5b432349236b0415c8327c1907bcc7a64bc"} Sep 11 08:59:46.463576 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:46.463542 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-gc48h" event={"ID":"a1369ac2-f702-4d53-84bd-2b47c17d39ca","Type":"ContainerStarted","Data":"6358d9a95301b924de3196b0fa7bf6d9f53118521f74ebf653e15e4141986ad6"} Sep 11 08:59:46.478696 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:46.478646 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-test-apply-gc48h" podStartSLOduration=2.4018358810000002 podStartE2EDuration="13.478630516s" totalImagesPullingTime="4.619059542s" totalInitContainerRuntime="6.457735093s" isStatefulPod=false podCreationTimestamp="2026-09-11 08:59:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 08:59:46.478037533 +0000 UTC m=+793.377935832" watchObservedRunningTime="2026-09-11 08:59:46.478630516 +0000 UTC m=+793.378528812" Sep 11 08:59:55.497543 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:55.497506 2562 generic.go:356] "Generic (PLEG): container finished" podID="a1369ac2-f702-4d53-84bd-2b47c17d39ca" containerID="6358d9a95301b924de3196b0fa7bf6d9f53118521f74ebf653e15e4141986ad6" exitCode=0 Sep 11 08:59:55.497901 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:55.497581 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-gc48h" event={"ID":"a1369ac2-f702-4d53-84bd-2b47c17d39ca","Type":"ContainerDied","Data":"6358d9a95301b924de3196b0fa7bf6d9f53118521f74ebf653e15e4141986ad6"} Sep 11 08:59:56.626313 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:56.626279 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-gc48h" Sep 11 08:59:56.657371 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:56.657347 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cln79\" (UniqueName: \"kubernetes.io/projected/a1369ac2-f702-4d53-84bd-2b47c17d39ca-kube-api-access-cln79\") pod \"a1369ac2-f702-4d53-84bd-2b47c17d39ca\" (UID: \"a1369ac2-f702-4d53-84bd-2b47c17d39ca\") " Sep 11 08:59:56.659334 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:56.659314 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1369ac2-f702-4d53-84bd-2b47c17d39ca-kube-api-access-cln79" (OuterVolumeSpecName: "kube-api-access-cln79") pod "a1369ac2-f702-4d53-84bd-2b47c17d39ca" (UID: "a1369ac2-f702-4d53-84bd-2b47c17d39ca"). InnerVolumeSpecName "kube-api-access-cln79". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 08:59:56.757952 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:56.757881 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cln79\" (UniqueName: \"kubernetes.io/projected/a1369ac2-f702-4d53-84bd-2b47c17d39ca-kube-api-access-cln79\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 08:59:57.506530 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:57.506497 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-gc48h" event={"ID":"a1369ac2-f702-4d53-84bd-2b47c17d39ca","Type":"ContainerDied","Data":"edcc80c08bcbadf979e4d80489d1213b35aa0dc77053ed2eb14aba2d12fdd1cb"} Sep 11 08:59:57.506530 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:57.506537 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="edcc80c08bcbadf979e4d80489d1213b35aa0dc77053ed2eb14aba2d12fdd1cb" Sep 11 08:59:57.506750 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:57.506508 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-gc48h" Sep 11 08:59:57.663502 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:57.663474 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-gc48h_a1369ac2-f702-4d53-84bd-2b47c17d39ca/feast-0/0.log" Sep 11 08:59:57.677170 ip-10-0-141-169 kubenswrapper[2562]: I0911 08:59:57.677153 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-gc48h_a1369ac2-f702-4d53-84bd-2b47c17d39ca/feast-1/0.log" Sep 11 09:00:29.451644 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:00:29.451607 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:00:34.441650 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:00:34.441615 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-test-apply-gc48h"] Sep 11 09:00:34.449768 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:00:34.449744 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-test-apply-gc48h"] Sep 11 09:00:34.465020 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:00:34.464996 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-zstb2"] Sep 11 09:00:34.465191 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:00:34.465150 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/redis-69bbd798b6-zstb2" podUID="e0ac5480-9e59-415d-a8f4-7c391468e2b4" containerName="redis" containerID="cri-o://2b9337c524d039b7ac23fdfb2ade8f76df54acc3b3496e88a2d740b1d4e541cb" gracePeriod=30 Sep 11 09:00:34.531255 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:00:34.531234 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:00:34.629875 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:00:34.629841 2562 generic.go:356] "Generic (PLEG): container finished" podID="e0ac5480-9e59-415d-a8f4-7c391468e2b4" containerID="2b9337c524d039b7ac23fdfb2ade8f76df54acc3b3496e88a2d740b1d4e541cb" exitCode=0 Sep 11 09:00:34.630012 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:00:34.629912 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-zstb2" event={"ID":"e0ac5480-9e59-415d-a8f4-7c391468e2b4","Type":"ContainerDied","Data":"2b9337c524d039b7ac23fdfb2ade8f76df54acc3b3496e88a2d740b1d4e541cb"} Sep 11 09:00:34.700298 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:00:34.700249 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-zstb2" Sep 11 09:00:34.795592 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:00:34.795563 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6gvv\" (UniqueName: \"kubernetes.io/projected/e0ac5480-9e59-415d-a8f4-7c391468e2b4-kube-api-access-s6gvv\") pod \"e0ac5480-9e59-415d-a8f4-7c391468e2b4\" (UID: \"e0ac5480-9e59-415d-a8f4-7c391468e2b4\") " Sep 11 09:00:34.797471 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:00:34.797440 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0ac5480-9e59-415d-a8f4-7c391468e2b4-kube-api-access-s6gvv" (OuterVolumeSpecName: "kube-api-access-s6gvv") pod "e0ac5480-9e59-415d-a8f4-7c391468e2b4" (UID: "e0ac5480-9e59-415d-a8f4-7c391468e2b4"). InnerVolumeSpecName "kube-api-access-s6gvv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 09:00:34.896443 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:00:34.896417 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6gvv\" (UniqueName: \"kubernetes.io/projected/e0ac5480-9e59-415d-a8f4-7c391468e2b4-kube-api-access-s6gvv\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:00:35.571229 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:00:35.571198 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1369ac2-f702-4d53-84bd-2b47c17d39ca" path="/var/lib/kubelet/pods/a1369ac2-f702-4d53-84bd-2b47c17d39ca/volumes" Sep 11 09:00:35.634367 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:00:35.634345 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-zstb2" Sep 11 09:00:35.634464 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:00:35.634346 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-zstb2" event={"ID":"e0ac5480-9e59-415d-a8f4-7c391468e2b4","Type":"ContainerDied","Data":"01b99f4c7eb0465ef42919fc20322f2ece8fd9a77b6a1ff6d8f6eceeeb5bd812"} Sep 11 09:00:35.634464 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:00:35.634460 2562 scope.go:117] "RemoveContainer" containerID="2b9337c524d039b7ac23fdfb2ade8f76df54acc3b3496e88a2d740b1d4e541cb" Sep 11 09:00:35.647404 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:00:35.647384 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-zstb2"] Sep 11 09:00:35.650619 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:00:35.650601 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-69bbd798b6-zstb2"] Sep 11 09:00:37.572447 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:00:37.572412 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0ac5480-9e59-415d-a8f4-7c391468e2b4" path="/var/lib/kubelet/pods/e0ac5480-9e59-415d-a8f4-7c391468e2b4/volumes" Sep 11 09:01:11.615587 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:01:11.615553 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:01:11.715274 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:01:11.715241 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:01:12.471885 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:01:12.471849 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:01:33.519647 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:01:33.519569 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/ovn-acl-logging/0.log" Sep 11 09:01:33.520744 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:01:33.520719 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/ovn-acl-logging/0.log" Sep 11 09:02:27.514749 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:02:27.514716 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:03:17.158858 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:03:17.158817 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:03:18.612184 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:03:18.612143 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:03:22.779438 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:03:22.779407 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:03:28.235071 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:03:28.235044 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:03:28.330783 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:03:28.330753 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:03:28.468615 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:03:28.468584 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:03:28.550555 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:03:28.550481 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:03:50.049254 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:03:50.049219 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:03:54.111736 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:03:54.111699 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:03:54.188933 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:03:54.188901 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:03:54.270599 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:03:54.270576 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:03:54.829135 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:03:54.829106 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:04:45.923141 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:04:45.923109 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:04:46.031210 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:04:46.031173 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:05:39.681056 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:39.681020 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:05:41.805969 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:41.805938 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:05:42.304613 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:42.304582 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:05:42.597713 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:42.597621 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:05:42.782728 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:42.782692 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:05:44.855782 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:44.855747 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:05:47.448414 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:47.448378 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:05:47.514035 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:47.514010 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:05:55.266162 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:55.266114 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:05:55.384457 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:55.384415 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:05:55.961991 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:55.961958 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-znm7z"] Sep 11 09:05:55.962223 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:55.962207 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0ac5480-9e59-415d-a8f4-7c391468e2b4" containerName="redis" Sep 11 09:05:55.962223 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:55.962225 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0ac5480-9e59-415d-a8f4-7c391468e2b4" containerName="redis" Sep 11 09:05:55.962359 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:55.962247 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1369ac2-f702-4d53-84bd-2b47c17d39ca" containerName="feast-1" Sep 11 09:05:55.962359 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:55.962255 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1369ac2-f702-4d53-84bd-2b47c17d39ca" containerName="feast-1" Sep 11 09:05:55.962359 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:55.962267 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1369ac2-f702-4d53-84bd-2b47c17d39ca" containerName="feast-0" Sep 11 09:05:55.962359 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:55.962274 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1369ac2-f702-4d53-84bd-2b47c17d39ca" containerName="feast-0" Sep 11 09:05:55.962359 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:55.962330 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0ac5480-9e59-415d-a8f4-7c391468e2b4" containerName="redis" Sep 11 09:05:55.962359 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:55.962341 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1369ac2-f702-4d53-84bd-2b47c17d39ca" containerName="feast-1" Sep 11 09:05:55.962710 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:55.962687 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:05:55.964939 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:55.964924 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-znm7z" Sep 11 09:05:55.974596 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:55.974580 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 11 09:05:55.974806 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:55.974793 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 11 09:05:55.974860 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:55.974807 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-bttrw\"" Sep 11 09:05:55.997507 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:55.997489 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-znm7z"] Sep 11 09:05:56.099516 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:56.099487 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbsmg\" (UniqueName: \"kubernetes.io/projected/0749a41c-e0d9-4e10-9c2c-fc26a7ec41ba-kube-api-access-sbsmg\") pod \"feast-operator-controller-manager-689b877b4b-znm7z\" (UID: \"0749a41c-e0d9-4e10-9c2c-fc26a7ec41ba\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-znm7z" Sep 11 09:05:56.199830 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:56.199765 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbsmg\" (UniqueName: \"kubernetes.io/projected/0749a41c-e0d9-4e10-9c2c-fc26a7ec41ba-kube-api-access-sbsmg\") pod \"feast-operator-controller-manager-689b877b4b-znm7z\" (UID: \"0749a41c-e0d9-4e10-9c2c-fc26a7ec41ba\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-znm7z" Sep 11 09:05:56.222499 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:56.222454 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbsmg\" (UniqueName: \"kubernetes.io/projected/0749a41c-e0d9-4e10-9c2c-fc26a7ec41ba-kube-api-access-sbsmg\") pod \"feast-operator-controller-manager-689b877b4b-znm7z\" (UID: \"0749a41c-e0d9-4e10-9c2c-fc26a7ec41ba\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-znm7z" Sep 11 09:05:56.273904 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:56.273883 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-znm7z" Sep 11 09:05:56.566413 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:56.566374 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-znm7z"] Sep 11 09:05:56.569501 ip-10-0-141-169 kubenswrapper[2562]: W0911 09:05:56.569473 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0749a41c_e0d9_4e10_9c2c_fc26a7ec41ba.slice/crio-7218a3171a9b84818e007bb85aaf47fc6f35175e4881e6fc49ba0f5b5c495d54 WatchSource:0}: Error finding container 7218a3171a9b84818e007bb85aaf47fc6f35175e4881e6fc49ba0f5b5c495d54: Status 404 returned error can't find the container with id 7218a3171a9b84818e007bb85aaf47fc6f35175e4881e6fc49ba0f5b5c495d54 Sep 11 09:05:56.571530 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:56.571512 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 11 09:05:56.663729 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:56.663699 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-znm7z" event={"ID":"0749a41c-e0d9-4e10-9c2c-fc26a7ec41ba","Type":"ContainerStarted","Data":"7218a3171a9b84818e007bb85aaf47fc6f35175e4881e6fc49ba0f5b5c495d54"} Sep 11 09:05:58.671802 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:58.671767 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-znm7z" event={"ID":"0749a41c-e0d9-4e10-9c2c-fc26a7ec41ba","Type":"ContainerStarted","Data":"90bd4f6fed5ee59e93cc5aa74ad7ef044104d4daaa0b0652bcb12306bf18b383"} Sep 11 09:05:58.672151 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:58.671864 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-znm7z" Sep 11 09:05:58.705680 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:05:58.705601 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-znm7z" podStartSLOduration=1.8358577889999999 podStartE2EDuration="3.705589021s" totalImagesPullingTime="1.869731232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 09:05:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 09:05:58.704616639 +0000 UTC m=+1165.604514937" watchObservedRunningTime="2026-09-11 09:05:58.705589021 +0000 UTC m=+1165.605487318" Sep 11 09:06:09.676587 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:09.676559 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-znm7z" Sep 11 09:06:10.184899 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:10.184864 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:06:10.251698 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:10.251673 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:06:10.748931 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:10.748898 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:06:10.852829 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:10.852783 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:06:10.873852 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:10.873826 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn"] Sep 11 09:06:10.878973 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:10.878956 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:10.881592 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:10.881423 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 11 09:06:10.881592 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:10.881494 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 11 09:06:10.881592 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:10.881511 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 11 09:06:10.881802 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:10.881510 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 11 09:06:10.881802 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:10.881675 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 11 09:06:10.881802 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:10.881776 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-9dv7f\"" Sep 11 09:06:10.882207 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:10.882190 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 11 09:06:10.897589 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:10.897571 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn"] Sep 11 09:06:10.981600 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:10.981572 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:06:11.005661 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:11.005605 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqqhs\" (UniqueName: \"kubernetes.io/projected/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-kube-api-access-bqqhs\") pod \"feast-simple-feast-setup-57576b8fcf-xgkqn\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:11.005661 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:11.005635 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xgkqn\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:11.005788 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:11.005664 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xgkqn\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:11.005788 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:11.005689 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-xgkqn\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:11.005788 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:11.005757 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xgkqn\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:11.005906 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:11.005799 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xgkqn\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:11.083012 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:11.082987 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:06:11.106469 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:11.106446 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqqhs\" (UniqueName: \"kubernetes.io/projected/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-kube-api-access-bqqhs\") pod \"feast-simple-feast-setup-57576b8fcf-xgkqn\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:11.106554 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:11.106473 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xgkqn\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:11.106554 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:11.106497 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xgkqn\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:11.106554 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:11.106524 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-xgkqn\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:11.106554 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:11.106550 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xgkqn\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:11.106762 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:11.106579 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xgkqn\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:11.106762 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:06:11.106682 2562 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 11 09:06:11.106762 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:06:11.106754 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-ui-tls podName:3ce0c87c-5b9c-4401-91b8-7204efafb8e1 nodeName:}" failed. No retries permitted until 2026-09-11 09:06:11.606735682 +0000 UTC m=+1178.506633969 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-ui-tls") pod "feast-simple-feast-setup-57576b8fcf-xgkqn" (UID: "3ce0c87c-5b9c-4401-91b8-7204efafb8e1") : secret "feast-simple-feast-setup-ui-tls" not found Sep 11 09:06:11.107014 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:11.106985 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-xgkqn\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:11.108894 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:11.108875 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xgkqn\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:11.109019 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:11.109002 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xgkqn\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:11.109074 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:11.109055 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xgkqn\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:11.115152 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:11.115127 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqqhs\" (UniqueName: \"kubernetes.io/projected/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-kube-api-access-bqqhs\") pod \"feast-simple-feast-setup-57576b8fcf-xgkqn\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:11.182897 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:11.182879 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:06:11.283379 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:11.283330 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:06:11.611005 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:11.610928 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xgkqn\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:11.613287 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:11.613269 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-xgkqn\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:11.788375 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:11.788342 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:12.063151 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:12.063121 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn"] Sep 11 09:06:12.065769 ip-10-0-141-169 kubenswrapper[2562]: W0911 09:06:12.065741 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ce0c87c_5b9c_4401_91b8_7204efafb8e1.slice/crio-d7eed1d50bb5386b0714b995bb633e0d838fdce1a740f7495bd4185ab6a3e598 WatchSource:0}: Error finding container d7eed1d50bb5386b0714b995bb633e0d838fdce1a740f7495bd4185ab6a3e598: Status 404 returned error can't find the container with id d7eed1d50bb5386b0714b995bb633e0d838fdce1a740f7495bd4185ab6a3e598 Sep 11 09:06:12.718882 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:12.718850 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" event={"ID":"3ce0c87c-5b9c-4401-91b8-7204efafb8e1","Type":"ContainerStarted","Data":"d7eed1d50bb5386b0714b995bb633e0d838fdce1a740f7495bd4185ab6a3e598"} Sep 11 09:06:22.326454 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:22.326386 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:06:22.425909 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:22.425853 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:06:22.623723 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:22.623624 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:06:25.774270 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:25.774234 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" event={"ID":"3ce0c87c-5b9c-4401-91b8-7204efafb8e1","Type":"ContainerStarted","Data":"93f5d5ea81bb9e87f8479492eba5eba51996ea2c9f4255039576647f9392baec"} Sep 11 09:06:30.796455 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:30.796424 2562 generic.go:356] "Generic (PLEG): container finished" podID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerID="93f5d5ea81bb9e87f8479492eba5eba51996ea2c9f4255039576647f9392baec" exitCode=0 Sep 11 09:06:30.796814 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:30.796496 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" event={"ID":"3ce0c87c-5b9c-4401-91b8-7204efafb8e1","Type":"ContainerDied","Data":"93f5d5ea81bb9e87f8479492eba5eba51996ea2c9f4255039576647f9392baec"} Sep 11 09:06:31.802132 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:31.802104 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" event={"ID":"3ce0c87c-5b9c-4401-91b8-7204efafb8e1","Type":"ContainerStarted","Data":"3c7dfb87ef4c7f35437753fe715ffcae2c2fc4441db5fca6f72ccbd9404d48b5"} Sep 11 09:06:33.541360 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:33.541336 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/ovn-acl-logging/0.log" Sep 11 09:06:33.542071 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:33.542054 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/ovn-acl-logging/0.log" Sep 11 09:06:33.565457 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:33.565434 2562 scope.go:117] "RemoveContainer" containerID="6358d9a95301b924de3196b0fa7bf6d9f53118521f74ebf653e15e4141986ad6" Sep 11 09:06:33.573246 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:33.573228 2562 scope.go:117] "RemoveContainer" containerID="6bd2be973d8f36be7ac3919620d6b5b432349236b0415c8327c1907bcc7a64bc" Sep 11 09:06:36.820716 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:36.820639 2562 generic.go:356] "Generic (PLEG): container finished" podID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerID="3c7dfb87ef4c7f35437753fe715ffcae2c2fc4441db5fca6f72ccbd9404d48b5" exitCode=0 Sep 11 09:06:36.820716 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:36.820673 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" event={"ID":"3ce0c87c-5b9c-4401-91b8-7204efafb8e1","Type":"ContainerDied","Data":"3c7dfb87ef4c7f35437753fe715ffcae2c2fc4441db5fca6f72ccbd9404d48b5"} Sep 11 09:06:37.830316 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:37.830257 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" event={"ID":"3ce0c87c-5b9c-4401-91b8-7204efafb8e1","Type":"ContainerStarted","Data":"df1729f940fbf378c45be5806f06e95a88288c0866d0b9ce746d398a6fa48682"} Sep 11 09:06:37.830808 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:37.830342 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" event={"ID":"3ce0c87c-5b9c-4401-91b8-7204efafb8e1","Type":"ContainerStarted","Data":"7102a3196b6d2f9db4d24d99e456691024612d81aa5d8aa28f5b9f4e7085a4b0"} Sep 11 09:06:37.830808 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:37.830367 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" event={"ID":"3ce0c87c-5b9c-4401-91b8-7204efafb8e1","Type":"ContainerStarted","Data":"1238aac031c1398f68fbe774511e93442c2db1a420b06a21442ee0f2ab4fbdc0"} Sep 11 09:06:37.830808 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:37.830385 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" event={"ID":"3ce0c87c-5b9c-4401-91b8-7204efafb8e1","Type":"ContainerStarted","Data":"d23411544c8de44c1faf1db7a46d7dfd7d7b71ad2f85137d05f4de1ca62ea083"} Sep 11 09:06:37.854821 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:37.853545 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" podStartSLOduration=15.015387785 podStartE2EDuration="27.853523867s" totalImagesPullingTime="12.838136082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 09:06:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 09:06:37.853295421 +0000 UTC m=+1204.753193747" watchObservedRunningTime="2026-09-11 09:06:37.853523867 +0000 UTC m=+1204.753422189" Sep 11 09:06:38.788447 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:38.788412 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:38.788724 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:38.788700 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:38.788724 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:38.788732 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:38.788914 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:38.788751 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:38.790682 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:38.790639 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.16:8816: connect: connection refused" Sep 11 09:06:38.790797 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:38.790668 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.16:6571: connect: connection refused" Sep 11 09:06:38.790797 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:38.790689 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="online" probeResult="failure" output="Get \"https://10.133.0.16:6567/health\": dial tcp 10.133.0.16:6567: connect: connection refused" Sep 11 09:06:38.790797 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:38.790750 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.16:8443: connect: connection refused" Sep 11 09:06:39.674463 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:39.674427 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:06:41.789816 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:41.789436 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:41.789816 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:41.789510 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:41.789816 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:41.789531 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:41.789816 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:41.789753 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.16:8816: connect: connection refused" Sep 11 09:06:41.790623 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:41.789761 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.16:8443: connect: connection refused" Sep 11 09:06:41.790623 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:41.789765 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="online" probeResult="failure" output="Get \"https://10.133.0.16:6567/health\": dial tcp 10.133.0.16:6567: connect: connection refused" Sep 11 09:06:41.790623 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:41.790101 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:41.790623 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:41.790160 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.16:6571: connect: connection refused" Sep 11 09:06:44.789251 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:44.789207 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.16:8443: connect: connection refused" Sep 11 09:06:44.790142 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:44.790122 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:44.790702 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:44.790680 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:44.790939 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:44.790919 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:44.797397 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:44.797377 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:44.863767 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:44.863747 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:44.867255 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:44.867232 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:47.790177 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:47.790145 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:06:47.874163 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:06:47.874137 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:07:23.893433 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:23.893397 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:07:23.992938 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:23.992906 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:07:25.596980 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:25.596941 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:07:49.270668 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.270633 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn"] Sep 11 09:07:49.273195 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.270864 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="registry" containerID="cri-o://d23411544c8de44c1faf1db7a46d7dfd7d7b71ad2f85137d05f4de1ca62ea083" gracePeriod=30 Sep 11 09:07:49.273195 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.270907 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="offline" containerID="cri-o://7102a3196b6d2f9db4d24d99e456691024612d81aa5d8aa28f5b9f4e7085a4b0" gracePeriod=30 Sep 11 09:07:49.273195 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.270922 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="ui" containerID="cri-o://df1729f940fbf378c45be5806f06e95a88288c0866d0b9ce746d398a6fa48682" gracePeriod=30 Sep 11 09:07:49.273195 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.270957 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="online" containerID="cri-o://1238aac031c1398f68fbe774511e93442c2db1a420b06a21442ee0f2ab4fbdc0" gracePeriod=30 Sep 11 09:07:49.281425 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.281387 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:07:49.340662 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.340629 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:07:49.572376 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.570337 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:07:49.668777 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.668732 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:07:49.705207 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.705167 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682"] Sep 11 09:07:49.709360 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.709337 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:07:49.711694 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.711516 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-2jjf8\"" Sep 11 09:07:49.727031 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.727006 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682"] Sep 11 09:07:49.781365 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.781320 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-mv682\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:07:49.781573 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.781402 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-mv682\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:07:49.781573 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.781432 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-mv682\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:07:49.781573 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.781505 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5xdq\" (UniqueName: \"kubernetes.io/projected/e5dad503-5e52-4a1b-89fd-d7abec29726c-kube-api-access-d5xdq\") pod \"feast-simple-feast-setup-57576b8fcf-mv682\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:07:49.781733 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.781593 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e5dad503-5e52-4a1b-89fd-d7abec29726c-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-mv682\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:07:49.781733 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.781641 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-mv682\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:07:49.882243 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.881982 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-mv682\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:07:49.882243 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.882044 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-mv682\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:07:49.882243 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.882070 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-mv682\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:07:49.882243 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.882136 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5xdq\" (UniqueName: \"kubernetes.io/projected/e5dad503-5e52-4a1b-89fd-d7abec29726c-kube-api-access-d5xdq\") pod \"feast-simple-feast-setup-57576b8fcf-mv682\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:07:49.882243 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.882183 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e5dad503-5e52-4a1b-89fd-d7abec29726c-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-mv682\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:07:49.882243 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.882209 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-mv682\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:07:49.882619 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:07:49.882367 2562 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 11 09:07:49.882619 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:07:49.882437 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-ui-tls podName:e5dad503-5e52-4a1b-89fd-d7abec29726c nodeName:}" failed. No retries permitted until 2026-09-11 09:07:50.382415086 +0000 UTC m=+1277.282313366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-ui-tls") pod "feast-simple-feast-setup-57576b8fcf-mv682" (UID: "e5dad503-5e52-4a1b-89fd-d7abec29726c") : secret "feast-simple-feast-setup-ui-tls" not found Sep 11 09:07:49.882619 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:07:49.882556 2562 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 11 09:07:49.882619 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:07:49.882593 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-offline-tls podName:e5dad503-5e52-4a1b-89fd-d7abec29726c nodeName:}" failed. No retries permitted until 2026-09-11 09:07:50.382580005 +0000 UTC m=+1277.282478282 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-offline-tls") pod "feast-simple-feast-setup-57576b8fcf-mv682" (UID: "e5dad503-5e52-4a1b-89fd-d7abec29726c") : secret "feast-simple-feast-setup-offline-tls" not found Sep 11 09:07:49.882806 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:07:49.882662 2562 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Sep 11 09:07:49.882806 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:07:49.882694 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-online-tls podName:e5dad503-5e52-4a1b-89fd-d7abec29726c nodeName:}" failed. No retries permitted until 2026-09-11 09:07:50.382683637 +0000 UTC m=+1277.282581913 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-online-tls") pod "feast-simple-feast-setup-57576b8fcf-mv682" (UID: "e5dad503-5e52-4a1b-89fd-d7abec29726c") : secret "feast-simple-feast-setup-online-tls" not found Sep 11 09:07:49.882806 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:07:49.882367 2562 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 11 09:07:49.882806 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:07:49.882727 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-registry-tls podName:e5dad503-5e52-4a1b-89fd-d7abec29726c nodeName:}" failed. No retries permitted until 2026-09-11 09:07:50.38271831 +0000 UTC m=+1277.282616588 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-registry-tls") pod "feast-simple-feast-setup-57576b8fcf-mv682" (UID: "e5dad503-5e52-4a1b-89fd-d7abec29726c") : secret "feast-simple-feast-setup-registry-tls" not found Sep 11 09:07:49.882985 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.882962 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e5dad503-5e52-4a1b-89fd-d7abec29726c-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-mv682\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:07:49.895732 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.895682 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5xdq\" (UniqueName: \"kubernetes.io/projected/e5dad503-5e52-4a1b-89fd-d7abec29726c-kube-api-access-d5xdq\") pod \"feast-simple-feast-setup-57576b8fcf-mv682\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:07:49.991451 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:49.991424 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:07:50.085159 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:50.085132 2562 generic.go:356] "Generic (PLEG): container finished" podID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerID="df1729f940fbf378c45be5806f06e95a88288c0866d0b9ce746d398a6fa48682" exitCode=0 Sep 11 09:07:50.085296 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:50.085203 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" event={"ID":"3ce0c87c-5b9c-4401-91b8-7204efafb8e1","Type":"ContainerDied","Data":"df1729f940fbf378c45be5806f06e95a88288c0866d0b9ce746d398a6fa48682"} Sep 11 09:07:50.120420 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:50.120358 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:07:50.258936 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:50.258905 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:07:50.377513 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:50.377423 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:07:50.385696 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:50.385661 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-mv682\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:07:50.385832 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:50.385703 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-mv682\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:07:50.385832 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:50.385775 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-mv682\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:07:50.385832 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:50.385807 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-mv682\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:07:50.388383 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:50.388357 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-mv682\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:07:50.388383 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:50.388377 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-mv682\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:07:50.388555 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:50.388438 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-mv682\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:07:50.388599 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:50.388556 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-mv682\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:07:50.621615 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:50.621585 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:07:50.895520 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:50.895440 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682"] Sep 11 09:07:50.899652 ip-10-0-141-169 kubenswrapper[2562]: W0911 09:07:50.899628 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode5dad503_5e52_4a1b_89fd_d7abec29726c.slice/crio-d26ef43ec55ae602ce8576c97da85576096f0840e05f3c310f78471e2080999a WatchSource:0}: Error finding container d26ef43ec55ae602ce8576c97da85576096f0840e05f3c310f78471e2080999a: Status 404 returned error can't find the container with id d26ef43ec55ae602ce8576c97da85576096f0840e05f3c310f78471e2080999a Sep 11 09:07:51.091174 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:51.091139 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" event={"ID":"e5dad503-5e52-4a1b-89fd-d7abec29726c","Type":"ContainerStarted","Data":"e8e02ea272d4114740215a06aca90baff7fcf2b4ffbf08d0880a07fbaa8421f3"} Sep 11 09:07:51.091358 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:51.091188 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" event={"ID":"e5dad503-5e52-4a1b-89fd-d7abec29726c","Type":"ContainerStarted","Data":"d26ef43ec55ae602ce8576c97da85576096f0840e05f3c310f78471e2080999a"} Sep 11 09:07:51.094226 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:51.094203 2562 generic.go:356] "Generic (PLEG): container finished" podID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerID="1238aac031c1398f68fbe774511e93442c2db1a420b06a21442ee0f2ab4fbdc0" exitCode=0 Sep 11 09:07:51.094352 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:51.094232 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" event={"ID":"3ce0c87c-5b9c-4401-91b8-7204efafb8e1","Type":"ContainerDied","Data":"1238aac031c1398f68fbe774511e93442c2db1a420b06a21442ee0f2ab4fbdc0"} Sep 11 09:07:54.864139 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:54.864099 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="online" probeResult="failure" output="Get \"https://10.133.0.16:6567/health\": dial tcp 10.133.0.16:6567: connect: connection refused" Sep 11 09:07:56.114528 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:56.114499 2562 generic.go:356] "Generic (PLEG): container finished" podID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerID="e8e02ea272d4114740215a06aca90baff7fcf2b4ffbf08d0880a07fbaa8421f3" exitCode=0 Sep 11 09:07:56.114899 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:56.114581 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" event={"ID":"e5dad503-5e52-4a1b-89fd-d7abec29726c","Type":"ContainerDied","Data":"e8e02ea272d4114740215a06aca90baff7fcf2b4ffbf08d0880a07fbaa8421f3"} Sep 11 09:07:57.120158 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:57.120122 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" event={"ID":"e5dad503-5e52-4a1b-89fd-d7abec29726c","Type":"ContainerStarted","Data":"b0256e8023ba7788be209eabfa65a95e11ba529542a211d8644b3692c9ff016f"} Sep 11 09:07:57.874052 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:07:57.874017 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.16:8443: connect: connection refused" Sep 11 09:08:02.138477 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:02.138403 2562 generic.go:356] "Generic (PLEG): container finished" podID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerID="b0256e8023ba7788be209eabfa65a95e11ba529542a211d8644b3692c9ff016f" exitCode=0 Sep 11 09:08:02.138791 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:02.138477 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" event={"ID":"e5dad503-5e52-4a1b-89fd-d7abec29726c","Type":"ContainerDied","Data":"b0256e8023ba7788be209eabfa65a95e11ba529542a211d8644b3692c9ff016f"} Sep 11 09:08:03.147710 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:03.147666 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" event={"ID":"e5dad503-5e52-4a1b-89fd-d7abec29726c","Type":"ContainerStarted","Data":"debdd6d352804c726a8243359fc189b74dcda5d2fcc149a5ad5c5c4e8838f522"} Sep 11 09:08:03.148188 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:03.147732 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" event={"ID":"e5dad503-5e52-4a1b-89fd-d7abec29726c","Type":"ContainerStarted","Data":"751b0efef9bc55ae19f4f4c410a487a76d99e27eb705847743ebc0cc496c2ef8"} Sep 11 09:08:03.148188 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:03.147753 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" event={"ID":"e5dad503-5e52-4a1b-89fd-d7abec29726c","Type":"ContainerStarted","Data":"7051bcee3fe8e8ee6b73fc6067503a10502a18d832214d19eea4eefb1cd64eaf"} Sep 11 09:08:03.148188 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:03.147771 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" event={"ID":"e5dad503-5e52-4a1b-89fd-d7abec29726c","Type":"ContainerStarted","Data":"49ed1824e563bb3b0bb2f6e423172b3ce578b0fb28de9204876effd5f95f33dc"} Sep 11 09:08:04.864114 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:04.864059 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="online" probeResult="failure" output="Get \"https://10.133.0.16:6567/health\": dial tcp 10.133.0.16:6567: connect: connection refused" Sep 11 09:08:05.622836 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:05.622802 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:08:05.623115 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:05.623085 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:08:05.623115 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:05.623119 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:08:05.623326 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:05.623137 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:08:05.624811 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:05.624779 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.17:8816: connect: connection refused" Sep 11 09:08:05.625002 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:05.624817 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.17:6571: connect: connection refused" Sep 11 09:08:05.625086 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:05.624819 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.17:8443: connect: connection refused" Sep 11 09:08:05.625086 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:05.624883 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.17:6567/health\": dial tcp 10.133.0.17:6567: connect: connection refused" Sep 11 09:08:07.874338 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:07.874269 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.16:8443: connect: connection refused" Sep 11 09:08:08.622184 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:08.622142 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.17:8443: connect: connection refused" Sep 11 09:08:08.622489 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:08.622142 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.17:8816: connect: connection refused" Sep 11 09:08:08.622580 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:08.622142 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.17:6571: connect: connection refused" Sep 11 09:08:08.622580 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:08.622316 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.17:6567/health\": dial tcp 10.133.0.17:6567: connect: connection refused" Sep 11 09:08:10.622082 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:10.622047 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:08:10.622488 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:10.622100 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:08:10.622488 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:10.622113 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:08:10.622488 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:10.622127 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:08:11.623625 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:11.623596 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:08:11.624047 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:11.623733 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:08:11.624047 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:11.623956 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:08:11.628711 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:11.628691 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:08:11.652310 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:11.652265 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" podStartSLOduration=22.652247581 podStartE2EDuration="22.652247581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 09:07:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 09:08:03.173063992 +0000 UTC m=+1290.072962291" watchObservedRunningTime="2026-09-11 09:08:11.652247581 +0000 UTC m=+1298.552145880" Sep 11 09:08:12.198530 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:12.198496 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:08:12.198693 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:12.198575 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:08:12.198773 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:12.198758 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:08:12.201087 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:12.201069 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:08:14.863422 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:14.863378 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="online" probeResult="failure" output="Get \"https://10.133.0.16:6567/health\": dial tcp 10.133.0.16:6567: connect: connection refused" Sep 11 09:08:14.863822 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:14.863516 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:08:16.536219 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:16.536184 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:08:17.874346 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:17.874285 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.16:8443: connect: connection refused" Sep 11 09:08:17.874709 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:17.874441 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:08:19.908180 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:19.908159 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:08:20.005213 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.005185 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-ui-tls\") pod \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " Sep 11 09:08:20.005362 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.005229 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-feast-data\") pod \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " Sep 11 09:08:20.005362 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.005247 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-online-tls\") pod \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " Sep 11 09:08:20.005362 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.005284 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqqhs\" (UniqueName: \"kubernetes.io/projected/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-kube-api-access-bqqhs\") pod \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " Sep 11 09:08:20.005362 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.005319 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-registry-tls\") pod \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " Sep 11 09:08:20.005362 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.005338 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-offline-tls\") pod \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\" (UID: \"3ce0c87c-5b9c-4401-91b8-7204efafb8e1\") " Sep 11 09:08:20.005757 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.005728 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-feast-data" (OuterVolumeSpecName: "feast-data") pod "3ce0c87c-5b9c-4401-91b8-7204efafb8e1" (UID: "3ce0c87c-5b9c-4401-91b8-7204efafb8e1"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 09:08:20.007595 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.007567 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "3ce0c87c-5b9c-4401-91b8-7204efafb8e1" (UID: "3ce0c87c-5b9c-4401-91b8-7204efafb8e1"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 09:08:20.007726 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.007708 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "3ce0c87c-5b9c-4401-91b8-7204efafb8e1" (UID: "3ce0c87c-5b9c-4401-91b8-7204efafb8e1"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 09:08:20.007819 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.007802 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-kube-api-access-bqqhs" (OuterVolumeSpecName: "kube-api-access-bqqhs") pod "3ce0c87c-5b9c-4401-91b8-7204efafb8e1" (UID: "3ce0c87c-5b9c-4401-91b8-7204efafb8e1"). InnerVolumeSpecName "kube-api-access-bqqhs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 09:08:20.007875 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.007833 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-online-tls" (OuterVolumeSpecName: "online-tls") pod "3ce0c87c-5b9c-4401-91b8-7204efafb8e1" (UID: "3ce0c87c-5b9c-4401-91b8-7204efafb8e1"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 09:08:20.007875 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.007849 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "3ce0c87c-5b9c-4401-91b8-7204efafb8e1" (UID: "3ce0c87c-5b9c-4401-91b8-7204efafb8e1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 09:08:20.106711 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.106657 2562 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-ui-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:08:20.106711 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.106676 2562 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-feast-data\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:08:20.106711 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.106686 2562 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-online-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:08:20.106711 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.106694 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqqhs\" (UniqueName: \"kubernetes.io/projected/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-kube-api-access-bqqhs\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:08:20.106711 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.106703 2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-registry-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:08:20.106711 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.106712 2562 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3ce0c87c-5b9c-4401-91b8-7204efafb8e1-offline-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:08:20.227931 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.227905 2562 generic.go:356] "Generic (PLEG): container finished" podID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerID="7102a3196b6d2f9db4d24d99e456691024612d81aa5d8aa28f5b9f4e7085a4b0" exitCode=137 Sep 11 09:08:20.227931 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.227926 2562 generic.go:356] "Generic (PLEG): container finished" podID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerID="d23411544c8de44c1faf1db7a46d7dfd7d7b71ad2f85137d05f4de1ca62ea083" exitCode=137 Sep 11 09:08:20.228100 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.227959 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" event={"ID":"3ce0c87c-5b9c-4401-91b8-7204efafb8e1","Type":"ContainerDied","Data":"7102a3196b6d2f9db4d24d99e456691024612d81aa5d8aa28f5b9f4e7085a4b0"} Sep 11 09:08:20.228100 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.227978 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" Sep 11 09:08:20.228100 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.227995 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" event={"ID":"3ce0c87c-5b9c-4401-91b8-7204efafb8e1","Type":"ContainerDied","Data":"d23411544c8de44c1faf1db7a46d7dfd7d7b71ad2f85137d05f4de1ca62ea083"} Sep 11 09:08:20.228100 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.228008 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn" event={"ID":"3ce0c87c-5b9c-4401-91b8-7204efafb8e1","Type":"ContainerDied","Data":"d7eed1d50bb5386b0714b995bb633e0d838fdce1a740f7495bd4185ab6a3e598"} Sep 11 09:08:20.228100 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.228024 2562 scope.go:117] "RemoveContainer" containerID="df1729f940fbf378c45be5806f06e95a88288c0866d0b9ce746d398a6fa48682" Sep 11 09:08:20.236371 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.236190 2562 scope.go:117] "RemoveContainer" containerID="7102a3196b6d2f9db4d24d99e456691024612d81aa5d8aa28f5b9f4e7085a4b0" Sep 11 09:08:20.243297 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.243280 2562 scope.go:117] "RemoveContainer" containerID="1238aac031c1398f68fbe774511e93442c2db1a420b06a21442ee0f2ab4fbdc0" Sep 11 09:08:20.248792 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.248773 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn"] Sep 11 09:08:20.250978 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.250963 2562 scope.go:117] "RemoveContainer" containerID="d23411544c8de44c1faf1db7a46d7dfd7d7b71ad2f85137d05f4de1ca62ea083" Sep 11 09:08:20.255111 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.255092 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-xgkqn"] Sep 11 09:08:20.258148 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.258132 2562 scope.go:117] "RemoveContainer" containerID="3c7dfb87ef4c7f35437753fe715ffcae2c2fc4441db5fca6f72ccbd9404d48b5" Sep 11 09:08:20.264574 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.264559 2562 scope.go:117] "RemoveContainer" containerID="93f5d5ea81bb9e87f8479492eba5eba51996ea2c9f4255039576647f9392baec" Sep 11 09:08:20.272525 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.272508 2562 scope.go:117] "RemoveContainer" containerID="df1729f940fbf378c45be5806f06e95a88288c0866d0b9ce746d398a6fa48682" Sep 11 09:08:20.272778 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:08:20.272764 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df1729f940fbf378c45be5806f06e95a88288c0866d0b9ce746d398a6fa48682\": container with ID starting with df1729f940fbf378c45be5806f06e95a88288c0866d0b9ce746d398a6fa48682 not found: ID does not exist" containerID="df1729f940fbf378c45be5806f06e95a88288c0866d0b9ce746d398a6fa48682" Sep 11 09:08:20.272834 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.272785 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df1729f940fbf378c45be5806f06e95a88288c0866d0b9ce746d398a6fa48682"} err="failed to get container status \"df1729f940fbf378c45be5806f06e95a88288c0866d0b9ce746d398a6fa48682\": rpc error: code = NotFound desc = could not find container \"df1729f940fbf378c45be5806f06e95a88288c0866d0b9ce746d398a6fa48682\": container with ID starting with df1729f940fbf378c45be5806f06e95a88288c0866d0b9ce746d398a6fa48682 not found: ID does not exist" Sep 11 09:08:20.272834 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.272800 2562 scope.go:117] "RemoveContainer" containerID="7102a3196b6d2f9db4d24d99e456691024612d81aa5d8aa28f5b9f4e7085a4b0" Sep 11 09:08:20.273013 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:08:20.272998 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7102a3196b6d2f9db4d24d99e456691024612d81aa5d8aa28f5b9f4e7085a4b0\": container with ID starting with 7102a3196b6d2f9db4d24d99e456691024612d81aa5d8aa28f5b9f4e7085a4b0 not found: ID does not exist" containerID="7102a3196b6d2f9db4d24d99e456691024612d81aa5d8aa28f5b9f4e7085a4b0" Sep 11 09:08:20.273053 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.273017 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7102a3196b6d2f9db4d24d99e456691024612d81aa5d8aa28f5b9f4e7085a4b0"} err="failed to get container status \"7102a3196b6d2f9db4d24d99e456691024612d81aa5d8aa28f5b9f4e7085a4b0\": rpc error: code = NotFound desc = could not find container \"7102a3196b6d2f9db4d24d99e456691024612d81aa5d8aa28f5b9f4e7085a4b0\": container with ID starting with 7102a3196b6d2f9db4d24d99e456691024612d81aa5d8aa28f5b9f4e7085a4b0 not found: ID does not exist" Sep 11 09:08:20.273053 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.273030 2562 scope.go:117] "RemoveContainer" containerID="1238aac031c1398f68fbe774511e93442c2db1a420b06a21442ee0f2ab4fbdc0" Sep 11 09:08:20.273260 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:08:20.273244 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1238aac031c1398f68fbe774511e93442c2db1a420b06a21442ee0f2ab4fbdc0\": container with ID starting with 1238aac031c1398f68fbe774511e93442c2db1a420b06a21442ee0f2ab4fbdc0 not found: ID does not exist" containerID="1238aac031c1398f68fbe774511e93442c2db1a420b06a21442ee0f2ab4fbdc0" Sep 11 09:08:20.273321 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.273269 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1238aac031c1398f68fbe774511e93442c2db1a420b06a21442ee0f2ab4fbdc0"} err="failed to get container status \"1238aac031c1398f68fbe774511e93442c2db1a420b06a21442ee0f2ab4fbdc0\": rpc error: code = NotFound desc = could not find container \"1238aac031c1398f68fbe774511e93442c2db1a420b06a21442ee0f2ab4fbdc0\": container with ID starting with 1238aac031c1398f68fbe774511e93442c2db1a420b06a21442ee0f2ab4fbdc0 not found: ID does not exist" Sep 11 09:08:20.273321 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.273286 2562 scope.go:117] "RemoveContainer" containerID="d23411544c8de44c1faf1db7a46d7dfd7d7b71ad2f85137d05f4de1ca62ea083" Sep 11 09:08:20.273570 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:08:20.273556 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d23411544c8de44c1faf1db7a46d7dfd7d7b71ad2f85137d05f4de1ca62ea083\": container with ID starting with d23411544c8de44c1faf1db7a46d7dfd7d7b71ad2f85137d05f4de1ca62ea083 not found: ID does not exist" containerID="d23411544c8de44c1faf1db7a46d7dfd7d7b71ad2f85137d05f4de1ca62ea083" Sep 11 09:08:20.273632 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.273572 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d23411544c8de44c1faf1db7a46d7dfd7d7b71ad2f85137d05f4de1ca62ea083"} err="failed to get container status \"d23411544c8de44c1faf1db7a46d7dfd7d7b71ad2f85137d05f4de1ca62ea083\": rpc error: code = NotFound desc = could not find container \"d23411544c8de44c1faf1db7a46d7dfd7d7b71ad2f85137d05f4de1ca62ea083\": container with ID starting with d23411544c8de44c1faf1db7a46d7dfd7d7b71ad2f85137d05f4de1ca62ea083 not found: ID does not exist" Sep 11 09:08:20.273632 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.273592 2562 scope.go:117] "RemoveContainer" containerID="3c7dfb87ef4c7f35437753fe715ffcae2c2fc4441db5fca6f72ccbd9404d48b5" Sep 11 09:08:20.273797 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:08:20.273782 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c7dfb87ef4c7f35437753fe715ffcae2c2fc4441db5fca6f72ccbd9404d48b5\": container with ID starting with 3c7dfb87ef4c7f35437753fe715ffcae2c2fc4441db5fca6f72ccbd9404d48b5 not found: ID does not exist" containerID="3c7dfb87ef4c7f35437753fe715ffcae2c2fc4441db5fca6f72ccbd9404d48b5" Sep 11 09:08:20.273836 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.273800 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c7dfb87ef4c7f35437753fe715ffcae2c2fc4441db5fca6f72ccbd9404d48b5"} err="failed to get container status \"3c7dfb87ef4c7f35437753fe715ffcae2c2fc4441db5fca6f72ccbd9404d48b5\": rpc error: code = NotFound desc = could not find container \"3c7dfb87ef4c7f35437753fe715ffcae2c2fc4441db5fca6f72ccbd9404d48b5\": container with ID starting with 3c7dfb87ef4c7f35437753fe715ffcae2c2fc4441db5fca6f72ccbd9404d48b5 not found: ID does not exist" Sep 11 09:08:20.273836 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.273811 2562 scope.go:117] "RemoveContainer" containerID="93f5d5ea81bb9e87f8479492eba5eba51996ea2c9f4255039576647f9392baec" Sep 11 09:08:20.274071 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:08:20.274055 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93f5d5ea81bb9e87f8479492eba5eba51996ea2c9f4255039576647f9392baec\": container with ID starting with 93f5d5ea81bb9e87f8479492eba5eba51996ea2c9f4255039576647f9392baec not found: ID does not exist" containerID="93f5d5ea81bb9e87f8479492eba5eba51996ea2c9f4255039576647f9392baec" Sep 11 09:08:20.274115 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.274072 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93f5d5ea81bb9e87f8479492eba5eba51996ea2c9f4255039576647f9392baec"} err="failed to get container status \"93f5d5ea81bb9e87f8479492eba5eba51996ea2c9f4255039576647f9392baec\": rpc error: code = NotFound desc = could not find container \"93f5d5ea81bb9e87f8479492eba5eba51996ea2c9f4255039576647f9392baec\": container with ID starting with 93f5d5ea81bb9e87f8479492eba5eba51996ea2c9f4255039576647f9392baec not found: ID does not exist" Sep 11 09:08:20.274115 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.274085 2562 scope.go:117] "RemoveContainer" containerID="df1729f940fbf378c45be5806f06e95a88288c0866d0b9ce746d398a6fa48682" Sep 11 09:08:20.274286 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.274269 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df1729f940fbf378c45be5806f06e95a88288c0866d0b9ce746d398a6fa48682"} err="failed to get container status \"df1729f940fbf378c45be5806f06e95a88288c0866d0b9ce746d398a6fa48682\": rpc error: code = NotFound desc = could not find container \"df1729f940fbf378c45be5806f06e95a88288c0866d0b9ce746d398a6fa48682\": container with ID starting with df1729f940fbf378c45be5806f06e95a88288c0866d0b9ce746d398a6fa48682 not found: ID does not exist" Sep 11 09:08:20.274348 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.274287 2562 scope.go:117] "RemoveContainer" containerID="7102a3196b6d2f9db4d24d99e456691024612d81aa5d8aa28f5b9f4e7085a4b0" Sep 11 09:08:20.274580 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.274563 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7102a3196b6d2f9db4d24d99e456691024612d81aa5d8aa28f5b9f4e7085a4b0"} err="failed to get container status \"7102a3196b6d2f9db4d24d99e456691024612d81aa5d8aa28f5b9f4e7085a4b0\": rpc error: code = NotFound desc = could not find container \"7102a3196b6d2f9db4d24d99e456691024612d81aa5d8aa28f5b9f4e7085a4b0\": container with ID starting with 7102a3196b6d2f9db4d24d99e456691024612d81aa5d8aa28f5b9f4e7085a4b0 not found: ID does not exist" Sep 11 09:08:20.274621 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.274580 2562 scope.go:117] "RemoveContainer" containerID="1238aac031c1398f68fbe774511e93442c2db1a420b06a21442ee0f2ab4fbdc0" Sep 11 09:08:20.274773 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.274756 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1238aac031c1398f68fbe774511e93442c2db1a420b06a21442ee0f2ab4fbdc0"} err="failed to get container status \"1238aac031c1398f68fbe774511e93442c2db1a420b06a21442ee0f2ab4fbdc0\": rpc error: code = NotFound desc = could not find container \"1238aac031c1398f68fbe774511e93442c2db1a420b06a21442ee0f2ab4fbdc0\": container with ID starting with 1238aac031c1398f68fbe774511e93442c2db1a420b06a21442ee0f2ab4fbdc0 not found: ID does not exist" Sep 11 09:08:20.274813 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.274774 2562 scope.go:117] "RemoveContainer" containerID="d23411544c8de44c1faf1db7a46d7dfd7d7b71ad2f85137d05f4de1ca62ea083" Sep 11 09:08:20.274953 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.274938 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d23411544c8de44c1faf1db7a46d7dfd7d7b71ad2f85137d05f4de1ca62ea083"} err="failed to get container status \"d23411544c8de44c1faf1db7a46d7dfd7d7b71ad2f85137d05f4de1ca62ea083\": rpc error: code = NotFound desc = could not find container \"d23411544c8de44c1faf1db7a46d7dfd7d7b71ad2f85137d05f4de1ca62ea083\": container with ID starting with d23411544c8de44c1faf1db7a46d7dfd7d7b71ad2f85137d05f4de1ca62ea083 not found: ID does not exist" Sep 11 09:08:20.274953 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.274953 2562 scope.go:117] "RemoveContainer" containerID="3c7dfb87ef4c7f35437753fe715ffcae2c2fc4441db5fca6f72ccbd9404d48b5" Sep 11 09:08:20.275148 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.275133 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c7dfb87ef4c7f35437753fe715ffcae2c2fc4441db5fca6f72ccbd9404d48b5"} err="failed to get container status \"3c7dfb87ef4c7f35437753fe715ffcae2c2fc4441db5fca6f72ccbd9404d48b5\": rpc error: code = NotFound desc = could not find container \"3c7dfb87ef4c7f35437753fe715ffcae2c2fc4441db5fca6f72ccbd9404d48b5\": container with ID starting with 3c7dfb87ef4c7f35437753fe715ffcae2c2fc4441db5fca6f72ccbd9404d48b5 not found: ID does not exist" Sep 11 09:08:20.275189 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.275149 2562 scope.go:117] "RemoveContainer" containerID="93f5d5ea81bb9e87f8479492eba5eba51996ea2c9f4255039576647f9392baec" Sep 11 09:08:20.275350 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:20.275335 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93f5d5ea81bb9e87f8479492eba5eba51996ea2c9f4255039576647f9392baec"} err="failed to get container status \"93f5d5ea81bb9e87f8479492eba5eba51996ea2c9f4255039576647f9392baec\": rpc error: code = NotFound desc = could not find container \"93f5d5ea81bb9e87f8479492eba5eba51996ea2c9f4255039576647f9392baec\": container with ID starting with 93f5d5ea81bb9e87f8479492eba5eba51996ea2c9f4255039576647f9392baec not found: ID does not exist" Sep 11 09:08:21.571518 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:21.571485 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" path="/var/lib/kubelet/pods/3ce0c87c-5b9c-4401-91b8-7204efafb8e1/volumes" Sep 11 09:08:59.067472 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:08:59.067393 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:09:00.482238 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:00.482210 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:09:14.144843 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.144805 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:09:14.241777 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.241747 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:09:14.578212 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.578178 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:09:14.675193 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.675161 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:09:14.678640 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.678617 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk"] Sep 11 09:09:14.678859 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.678848 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="feast-init" Sep 11 09:09:14.678907 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.678861 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="feast-init" Sep 11 09:09:14.678907 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.678870 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="feast-apply" Sep 11 09:09:14.678907 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.678876 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="feast-apply" Sep 11 09:09:14.678907 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.678883 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="ui" Sep 11 09:09:14.678907 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.678889 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="ui" Sep 11 09:09:14.678907 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.678898 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="offline" Sep 11 09:09:14.678907 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.678903 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="offline" Sep 11 09:09:14.678907 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.678910 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="registry" Sep 11 09:09:14.679140 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.678915 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="registry" Sep 11 09:09:14.679140 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.678926 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="online" Sep 11 09:09:14.679140 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.678931 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="online" Sep 11 09:09:14.679140 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.678969 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="ui" Sep 11 09:09:14.679140 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.678975 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="offline" Sep 11 09:09:14.679140 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.678981 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="online" Sep 11 09:09:14.679140 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.678989 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ce0c87c-5b9c-4401-91b8-7204efafb8e1" containerName="registry" Sep 11 09:09:14.682144 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.682129 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:14.685547 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.685529 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 11 09:09:14.685981 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.685962 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 11 09:09:14.686110 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.686094 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 11 09:09:14.686234 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.686217 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 11 09:09:14.686516 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.686500 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 11 09:09:14.686748 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.686733 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-slbmw\"" Sep 11 09:09:14.686832 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.686816 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 11 09:09:14.697319 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.697286 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk"] Sep 11 09:09:14.742923 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.742900 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:09:14.755751 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.755726 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c8534adb-97a3-45ba-ab15-a88ea93658c2-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-c7lxk\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:14.755840 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.755770 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c8534adb-97a3-45ba-ab15-a88ea93658c2-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-c7lxk\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:14.755840 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.755799 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c8534adb-97a3-45ba-ab15-a88ea93658c2-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-c7lxk\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:14.755926 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.755847 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c8534adb-97a3-45ba-ab15-a88ea93658c2-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-c7lxk\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:14.755926 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.755875 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/c8534adb-97a3-45ba-ab15-a88ea93658c2-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-c7lxk\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:14.755999 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.755931 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk4t5\" (UniqueName: \"kubernetes.io/projected/c8534adb-97a3-45ba-ab15-a88ea93658c2-kube-api-access-nk4t5\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-c7lxk\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:14.856617 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.856556 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk4t5\" (UniqueName: \"kubernetes.io/projected/c8534adb-97a3-45ba-ab15-a88ea93658c2-kube-api-access-nk4t5\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-c7lxk\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:14.856617 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.856608 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c8534adb-97a3-45ba-ab15-a88ea93658c2-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-c7lxk\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:14.856773 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.856644 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c8534adb-97a3-45ba-ab15-a88ea93658c2-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-c7lxk\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:14.856773 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.856672 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c8534adb-97a3-45ba-ab15-a88ea93658c2-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-c7lxk\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:14.856773 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.856702 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c8534adb-97a3-45ba-ab15-a88ea93658c2-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-c7lxk\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:14.856773 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.856728 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/c8534adb-97a3-45ba-ab15-a88ea93658c2-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-c7lxk\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:14.856773 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:09:14.856758 2562 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Sep 11 09:09:14.857011 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:09:14.856828 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8534adb-97a3-45ba-ab15-a88ea93658c2-online-tls podName:c8534adb-97a3-45ba-ab15-a88ea93658c2 nodeName:}" failed. No retries permitted until 2026-09-11 09:09:15.356810293 +0000 UTC m=+1362.256708589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/c8534adb-97a3-45ba-ab15-a88ea93658c2-online-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" (UID: "c8534adb-97a3-45ba-ab15-a88ea93658c2") : secret "feast-simple-feast-remote-setup-online-tls" not found Sep 11 09:09:14.857104 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.857081 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c8534adb-97a3-45ba-ab15-a88ea93658c2-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-c7lxk\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:14.857222 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:09:14.857206 2562 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 11 09:09:14.857370 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:09:14.857264 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8534adb-97a3-45ba-ab15-a88ea93658c2-ui-tls podName:c8534adb-97a3-45ba-ab15-a88ea93658c2 nodeName:}" failed. No retries permitted until 2026-09-11 09:09:15.357245162 +0000 UTC m=+1362.257143444 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/c8534adb-97a3-45ba-ab15-a88ea93658c2-ui-tls") pod "feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" (UID: "c8534adb-97a3-45ba-ab15-a88ea93658c2") : secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 11 09:09:14.857487 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.857468 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/c8534adb-97a3-45ba-ab15-a88ea93658c2-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-c7lxk\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:14.859108 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.859088 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c8534adb-97a3-45ba-ab15-a88ea93658c2-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-c7lxk\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:14.865989 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.865970 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk4t5\" (UniqueName: \"kubernetes.io/projected/c8534adb-97a3-45ba-ab15-a88ea93658c2-kube-api-access-nk4t5\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-c7lxk\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:14.991472 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:14.991450 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:09:15.060855 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:15.060830 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:09:15.360408 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:15.360384 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c8534adb-97a3-45ba-ab15-a88ea93658c2-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-c7lxk\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:15.360755 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:15.360424 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c8534adb-97a3-45ba-ab15-a88ea93658c2-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-c7lxk\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:15.362748 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:15.362722 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c8534adb-97a3-45ba-ab15-a88ea93658c2-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-c7lxk\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:15.362926 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:15.362901 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c8534adb-97a3-45ba-ab15-a88ea93658c2-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-c7lxk\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:15.592249 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:15.592218 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:15.862219 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:15.862197 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk"] Sep 11 09:09:15.864981 ip-10-0-141-169 kubenswrapper[2562]: W0911 09:09:15.864951 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc8534adb_97a3_45ba_ab15_a88ea93658c2.slice/crio-428aa4f16d498018a53f12ff1e92e9df2cd9b4eca80135e5120de1dd4e543168 WatchSource:0}: Error finding container 428aa4f16d498018a53f12ff1e92e9df2cd9b4eca80135e5120de1dd4e543168: Status 404 returned error can't find the container with id 428aa4f16d498018a53f12ff1e92e9df2cd9b4eca80135e5120de1dd4e543168 Sep 11 09:09:16.426731 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:16.426688 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" event={"ID":"c8534adb-97a3-45ba-ab15-a88ea93658c2","Type":"ContainerStarted","Data":"097f57d58fd9f7e237370e8239fcbf2bde41f678cf7c42811d8161b5a038f9ca"} Sep 11 09:09:16.427216 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:16.426739 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" event={"ID":"c8534adb-97a3-45ba-ab15-a88ea93658c2","Type":"ContainerStarted","Data":"428aa4f16d498018a53f12ff1e92e9df2cd9b4eca80135e5120de1dd4e543168"} Sep 11 09:09:21.446906 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:21.446869 2562 generic.go:356] "Generic (PLEG): container finished" podID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerID="097f57d58fd9f7e237370e8239fcbf2bde41f678cf7c42811d8161b5a038f9ca" exitCode=0 Sep 11 09:09:21.447346 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:21.446923 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" event={"ID":"c8534adb-97a3-45ba-ab15-a88ea93658c2","Type":"ContainerDied","Data":"097f57d58fd9f7e237370e8239fcbf2bde41f678cf7c42811d8161b5a038f9ca"} Sep 11 09:09:22.452370 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:22.452333 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" event={"ID":"c8534adb-97a3-45ba-ab15-a88ea93658c2","Type":"ContainerStarted","Data":"ea04370b041d0090610b5e2a1e11ad14995ca1c8eebe402727f146488778183d"} Sep 11 09:09:43.523053 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:43.522968 2562 generic.go:356] "Generic (PLEG): container finished" podID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerID="ea04370b041d0090610b5e2a1e11ad14995ca1c8eebe402727f146488778183d" exitCode=0 Sep 11 09:09:43.523053 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:43.523032 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" event={"ID":"c8534adb-97a3-45ba-ab15-a88ea93658c2","Type":"ContainerDied","Data":"ea04370b041d0090610b5e2a1e11ad14995ca1c8eebe402727f146488778183d"} Sep 11 09:09:44.531248 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:44.531211 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" event={"ID":"c8534adb-97a3-45ba-ab15-a88ea93658c2","Type":"ContainerStarted","Data":"4b27cb231eee282ad784f593907f2d0f01c54b0c6e4c861a0804b26d96e150e1"} Sep 11 09:09:44.531636 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:44.531269 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" event={"ID":"c8534adb-97a3-45ba-ab15-a88ea93658c2","Type":"ContainerStarted","Data":"a71406f78fe3abaced6860ec3a3b7d382ca625dfe1542348116024ea8e28fab8"} Sep 11 09:09:44.531636 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:44.531290 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" event={"ID":"c8534adb-97a3-45ba-ab15-a88ea93658c2","Type":"ContainerStarted","Data":"0ca1277fcdfeb3729ff5723171824b6a99547e2b125a735abe3c067447eee551"} Sep 11 09:09:44.555396 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:44.555345 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" podStartSLOduration=30.555324743 podStartE2EDuration="30.555324743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 09:09:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 09:09:44.553550022 +0000 UTC m=+1391.453448322" watchObservedRunningTime="2026-09-11 09:09:44.555324743 +0000 UTC m=+1391.455223052" Sep 11 09:09:45.592960 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:45.592799 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:45.592960 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:45.592856 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:45.592960 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:45.592882 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:45.592960 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:45.592900 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:45.592960 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:45.592916 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:45.592960 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:45.592931 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:45.594365 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:45.594334 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.18:8443: connect: connection refused" Sep 11 09:09:45.594501 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:45.594375 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="online" probeResult="failure" output="Get \"https://10.133.0.18:6567/health\": dial tcp 10.133.0.18:6567: connect: connection refused" Sep 11 09:09:45.594501 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:45.594383 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.18:8816: connect: connection refused" Sep 11 09:09:48.592807 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:48.592762 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.18:8443: connect: connection refused" Sep 11 09:09:48.592807 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:48.592784 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.18:8816: connect: connection refused" Sep 11 09:09:48.593265 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:48.592789 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="online" probeResult="failure" output="Get \"https://10.133.0.18:6567/health\": dial tcp 10.133.0.18:6567: connect: connection refused" Sep 11 09:09:51.592766 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:51.592719 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.18:8443: connect: connection refused" Sep 11 09:09:51.592766 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:51.592743 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="online" probeResult="failure" output="Get \"https://10.133.0.18:6567/health\": dial tcp 10.133.0.18:6567: connect: connection refused" Sep 11 09:09:51.593377 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:51.593350 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:51.594115 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:51.594093 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:54.593649 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:54.593617 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:54.594378 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:54.594357 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:55.311056 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:55.311025 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:56.270213 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:56.270183 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:09:56.597514 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:09:56.597429 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="online" probeResult="failure" output="Get \"https://10.133.0.18:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 11 09:10:16.597448 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:10:16.597406 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="online" probeResult="failure" output="Get \"https://10.133.0.18:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 11 09:10:36.597502 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:10:36.597413 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="online" probeResult="failure" output="Get \"https://10.133.0.18:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 11 09:10:56.596450 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:10:56.596403 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="online" probeResult="failure" output="Get \"https://10.133.0.18:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 11 09:10:59.134012 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:10:59.133984 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk"] Sep 11 09:10:59.134443 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:10:59.134192 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="online" containerID="cri-o://0ca1277fcdfeb3729ff5723171824b6a99547e2b125a735abe3c067447eee551" gracePeriod=30 Sep 11 09:10:59.134443 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:10:59.134267 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="offline" containerID="cri-o://a71406f78fe3abaced6860ec3a3b7d382ca625dfe1542348116024ea8e28fab8" gracePeriod=30 Sep 11 09:10:59.134443 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:10:59.134280 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="ui" containerID="cri-o://4b27cb231eee282ad784f593907f2d0f01c54b0c6e4c861a0804b26d96e150e1" gracePeriod=30 Sep 11 09:10:59.170916 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:10:59.170887 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:10:59.258623 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:10:59.258590 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:10:59.314217 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:10:59.314182 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682"] Sep 11 09:10:59.314535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:10:59.314502 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="registry" containerID="cri-o://49ed1824e563bb3b0bb2f6e423172b3ce578b0fb28de9204876effd5f95f33dc" gracePeriod=30 Sep 11 09:10:59.314640 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:10:59.314558 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="offline" containerID="cri-o://751b0efef9bc55ae19f4f4c410a487a76d99e27eb705847743ebc0cc496c2ef8" gracePeriod=30 Sep 11 09:10:59.314640 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:10:59.314582 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="online" containerID="cri-o://7051bcee3fe8e8ee6b73fc6067503a10502a18d832214d19eea4eefb1cd64eaf" gracePeriod=30 Sep 11 09:10:59.314755 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:10:59.314553 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="ui" containerID="cri-o://debdd6d352804c726a8243359fc189b74dcda5d2fcc149a5ad5c5c4e8838f522" gracePeriod=30 Sep 11 09:10:59.419704 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:10:59.419623 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:10:59.547694 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:10:59.547658 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:10:59.803438 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:10:59.803396 2562 generic.go:356] "Generic (PLEG): container finished" podID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerID="4b27cb231eee282ad784f593907f2d0f01c54b0c6e4c861a0804b26d96e150e1" exitCode=0 Sep 11 09:10:59.803617 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:10:59.803470 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" event={"ID":"c8534adb-97a3-45ba-ab15-a88ea93658c2","Type":"ContainerDied","Data":"4b27cb231eee282ad784f593907f2d0f01c54b0c6e4c861a0804b26d96e150e1"} Sep 11 09:11:00.810026 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:00.809995 2562 generic.go:356] "Generic (PLEG): container finished" podID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerID="0ca1277fcdfeb3729ff5723171824b6a99547e2b125a735abe3c067447eee551" exitCode=0 Sep 11 09:11:00.810449 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:00.810073 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" event={"ID":"c8534adb-97a3-45ba-ab15-a88ea93658c2","Type":"ContainerDied","Data":"0ca1277fcdfeb3729ff5723171824b6a99547e2b125a735abe3c067447eee551"} Sep 11 09:11:00.812881 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:00.812860 2562 generic.go:356] "Generic (PLEG): container finished" podID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerID="debdd6d352804c726a8243359fc189b74dcda5d2fcc149a5ad5c5c4e8838f522" exitCode=0 Sep 11 09:11:00.812881 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:00.812879 2562 generic.go:356] "Generic (PLEG): container finished" podID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerID="7051bcee3fe8e8ee6b73fc6067503a10502a18d832214d19eea4eefb1cd64eaf" exitCode=0 Sep 11 09:11:00.813022 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:00.812915 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" event={"ID":"e5dad503-5e52-4a1b-89fd-d7abec29726c","Type":"ContainerDied","Data":"debdd6d352804c726a8243359fc189b74dcda5d2fcc149a5ad5c5c4e8838f522"} Sep 11 09:11:00.813022 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:00.812943 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" event={"ID":"e5dad503-5e52-4a1b-89fd-d7abec29726c","Type":"ContainerDied","Data":"7051bcee3fe8e8ee6b73fc6067503a10502a18d832214d19eea4eefb1cd64eaf"} Sep 11 09:11:02.198412 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:02.198368 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.17:8443: connect: connection refused" Sep 11 09:11:02.198823 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:02.198384 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.17:6567/health\": dial tcp 10.133.0.17:6567: connect: connection refused" Sep 11 09:11:04.464485 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:04.464436 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:11:04.594861 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:04.594827 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.18:8443: connect: connection refused" Sep 11 09:11:05.582062 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:05.582023 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="online" probeResult="failure" output="Get \"https://10.133.0.18:6567/health\": dial tcp 10.133.0.18:6567: connect: connection refused" Sep 11 09:11:12.198886 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:12.198845 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.17:6567/health\": dial tcp 10.133.0.17:6567: connect: connection refused" Sep 11 09:11:12.199422 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:12.198845 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.17:8443: connect: connection refused" Sep 11 09:11:14.594800 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:14.594756 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.18:8443: connect: connection refused" Sep 11 09:11:15.581823 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:15.581779 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="online" probeResult="failure" output="Get \"https://10.133.0.18:6567/health\": dial tcp 10.133.0.18:6567: connect: connection refused" Sep 11 09:11:22.198177 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:22.198130 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.17:8443: connect: connection refused" Sep 11 09:11:22.198177 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:22.198156 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.17:6567/health\": dial tcp 10.133.0.17:6567: connect: connection refused" Sep 11 09:11:22.198726 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:22.198248 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:11:22.198726 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:22.198342 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:11:24.594240 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:24.594196 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.18:8443: connect: connection refused" Sep 11 09:11:24.594637 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:24.594333 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:11:25.582199 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:25.582159 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="online" probeResult="failure" output="Get \"https://10.133.0.18:6567/health\": dial tcp 10.133.0.18:6567: connect: connection refused" Sep 11 09:11:25.582402 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:25.582268 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:11:29.770433 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.770409 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:11:29.774801 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.774781 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c8534adb-97a3-45ba-ab15-a88ea93658c2-offline-tls\") pod \"c8534adb-97a3-45ba-ab15-a88ea93658c2\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " Sep 11 09:11:29.774911 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.774821 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/c8534adb-97a3-45ba-ab15-a88ea93658c2-registry-tls\") pod \"c8534adb-97a3-45ba-ab15-a88ea93658c2\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " Sep 11 09:11:29.774911 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.774852 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nk4t5\" (UniqueName: \"kubernetes.io/projected/c8534adb-97a3-45ba-ab15-a88ea93658c2-kube-api-access-nk4t5\") pod \"c8534adb-97a3-45ba-ab15-a88ea93658c2\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " Sep 11 09:11:29.774911 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.774884 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c8534adb-97a3-45ba-ab15-a88ea93658c2-online-tls\") pod \"c8534adb-97a3-45ba-ab15-a88ea93658c2\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " Sep 11 09:11:29.775073 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.774998 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c8534adb-97a3-45ba-ab15-a88ea93658c2-ui-tls\") pod \"c8534adb-97a3-45ba-ab15-a88ea93658c2\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " Sep 11 09:11:29.775073 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.775034 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c8534adb-97a3-45ba-ab15-a88ea93658c2-feast-data\") pod \"c8534adb-97a3-45ba-ab15-a88ea93658c2\" (UID: \"c8534adb-97a3-45ba-ab15-a88ea93658c2\") " Sep 11 09:11:29.775238 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.775211 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8534adb-97a3-45ba-ab15-a88ea93658c2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c8534adb-97a3-45ba-ab15-a88ea93658c2" (UID: "c8534adb-97a3-45ba-ab15-a88ea93658c2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 09:11:29.775625 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.775604 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8534adb-97a3-45ba-ab15-a88ea93658c2-feast-data" (OuterVolumeSpecName: "feast-data") pod "c8534adb-97a3-45ba-ab15-a88ea93658c2" (UID: "c8534adb-97a3-45ba-ab15-a88ea93658c2"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 09:11:29.776994 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.776969 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8534adb-97a3-45ba-ab15-a88ea93658c2-kube-api-access-nk4t5" (OuterVolumeSpecName: "kube-api-access-nk4t5") pod "c8534adb-97a3-45ba-ab15-a88ea93658c2" (UID: "c8534adb-97a3-45ba-ab15-a88ea93658c2"). InnerVolumeSpecName "kube-api-access-nk4t5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 09:11:29.777093 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.777006 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8534adb-97a3-45ba-ab15-a88ea93658c2-online-tls" (OuterVolumeSpecName: "online-tls") pod "c8534adb-97a3-45ba-ab15-a88ea93658c2" (UID: "c8534adb-97a3-45ba-ab15-a88ea93658c2"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 09:11:29.777401 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.777383 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8534adb-97a3-45ba-ab15-a88ea93658c2-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "c8534adb-97a3-45ba-ab15-a88ea93658c2" (UID: "c8534adb-97a3-45ba-ab15-a88ea93658c2"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 09:11:29.777447 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.777404 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8534adb-97a3-45ba-ab15-a88ea93658c2-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "c8534adb-97a3-45ba-ab15-a88ea93658c2" (UID: "c8534adb-97a3-45ba-ab15-a88ea93658c2"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 09:11:29.876137 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.876099 2562 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c8534adb-97a3-45ba-ab15-a88ea93658c2-ui-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:11:29.876137 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.876127 2562 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c8534adb-97a3-45ba-ab15-a88ea93658c2-feast-data\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:11:29.876137 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.876137 2562 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c8534adb-97a3-45ba-ab15-a88ea93658c2-offline-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:11:29.876137 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.876145 2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/c8534adb-97a3-45ba-ab15-a88ea93658c2-registry-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:11:29.876383 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.876154 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nk4t5\" (UniqueName: \"kubernetes.io/projected/c8534adb-97a3-45ba-ab15-a88ea93658c2-kube-api-access-nk4t5\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:11:29.876383 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.876162 2562 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c8534adb-97a3-45ba-ab15-a88ea93658c2-online-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:11:29.914672 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.914601 2562 generic.go:356] "Generic (PLEG): container finished" podID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerID="a71406f78fe3abaced6860ec3a3b7d382ca625dfe1542348116024ea8e28fab8" exitCode=137 Sep 11 09:11:29.914789 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.914682 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" Sep 11 09:11:29.914789 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.914683 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" event={"ID":"c8534adb-97a3-45ba-ab15-a88ea93658c2","Type":"ContainerDied","Data":"a71406f78fe3abaced6860ec3a3b7d382ca625dfe1542348116024ea8e28fab8"} Sep 11 09:11:29.914789 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.914728 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk" event={"ID":"c8534adb-97a3-45ba-ab15-a88ea93658c2","Type":"ContainerDied","Data":"428aa4f16d498018a53f12ff1e92e9df2cd9b4eca80135e5120de1dd4e543168"} Sep 11 09:11:29.914789 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.914750 2562 scope.go:117] "RemoveContainer" containerID="4b27cb231eee282ad784f593907f2d0f01c54b0c6e4c861a0804b26d96e150e1" Sep 11 09:11:29.918561 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.918537 2562 generic.go:356] "Generic (PLEG): container finished" podID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerID="751b0efef9bc55ae19f4f4c410a487a76d99e27eb705847743ebc0cc496c2ef8" exitCode=137 Sep 11 09:11:29.918561 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.918559 2562 generic.go:356] "Generic (PLEG): container finished" podID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerID="49ed1824e563bb3b0bb2f6e423172b3ce578b0fb28de9204876effd5f95f33dc" exitCode=137 Sep 11 09:11:29.918778 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.918617 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" event={"ID":"e5dad503-5e52-4a1b-89fd-d7abec29726c","Type":"ContainerDied","Data":"751b0efef9bc55ae19f4f4c410a487a76d99e27eb705847743ebc0cc496c2ef8"} Sep 11 09:11:29.918778 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.918665 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" event={"ID":"e5dad503-5e52-4a1b-89fd-d7abec29726c","Type":"ContainerDied","Data":"49ed1824e563bb3b0bb2f6e423172b3ce578b0fb28de9204876effd5f95f33dc"} Sep 11 09:11:29.924074 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.924035 2562 scope.go:117] "RemoveContainer" containerID="a71406f78fe3abaced6860ec3a3b7d382ca625dfe1542348116024ea8e28fab8" Sep 11 09:11:29.932068 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.932051 2562 scope.go:117] "RemoveContainer" containerID="0ca1277fcdfeb3729ff5723171824b6a99547e2b125a735abe3c067447eee551" Sep 11 09:11:29.938758 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.938734 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk"] Sep 11 09:11:29.940699 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.940677 2562 scope.go:117] "RemoveContainer" containerID="ea04370b041d0090610b5e2a1e11ad14995ca1c8eebe402727f146488778183d" Sep 11 09:11:29.944039 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.944010 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-c7lxk"] Sep 11 09:11:29.955411 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.955388 2562 scope.go:117] "RemoveContainer" containerID="097f57d58fd9f7e237370e8239fcbf2bde41f678cf7c42811d8161b5a038f9ca" Sep 11 09:11:29.963063 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.963043 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:11:29.964037 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.964018 2562 scope.go:117] "RemoveContainer" containerID="4b27cb231eee282ad784f593907f2d0f01c54b0c6e4c861a0804b26d96e150e1" Sep 11 09:11:29.964357 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:11:29.964333 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b27cb231eee282ad784f593907f2d0f01c54b0c6e4c861a0804b26d96e150e1\": container with ID starting with 4b27cb231eee282ad784f593907f2d0f01c54b0c6e4c861a0804b26d96e150e1 not found: ID does not exist" containerID="4b27cb231eee282ad784f593907f2d0f01c54b0c6e4c861a0804b26d96e150e1" Sep 11 09:11:29.964414 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.964363 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b27cb231eee282ad784f593907f2d0f01c54b0c6e4c861a0804b26d96e150e1"} err="failed to get container status \"4b27cb231eee282ad784f593907f2d0f01c54b0c6e4c861a0804b26d96e150e1\": rpc error: code = NotFound desc = could not find container \"4b27cb231eee282ad784f593907f2d0f01c54b0c6e4c861a0804b26d96e150e1\": container with ID starting with 4b27cb231eee282ad784f593907f2d0f01c54b0c6e4c861a0804b26d96e150e1 not found: ID does not exist" Sep 11 09:11:29.964414 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.964386 2562 scope.go:117] "RemoveContainer" containerID="a71406f78fe3abaced6860ec3a3b7d382ca625dfe1542348116024ea8e28fab8" Sep 11 09:11:29.964672 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:11:29.964652 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a71406f78fe3abaced6860ec3a3b7d382ca625dfe1542348116024ea8e28fab8\": container with ID starting with a71406f78fe3abaced6860ec3a3b7d382ca625dfe1542348116024ea8e28fab8 not found: ID does not exist" containerID="a71406f78fe3abaced6860ec3a3b7d382ca625dfe1542348116024ea8e28fab8" Sep 11 09:11:29.964737 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.964676 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a71406f78fe3abaced6860ec3a3b7d382ca625dfe1542348116024ea8e28fab8"} err="failed to get container status \"a71406f78fe3abaced6860ec3a3b7d382ca625dfe1542348116024ea8e28fab8\": rpc error: code = NotFound desc = could not find container \"a71406f78fe3abaced6860ec3a3b7d382ca625dfe1542348116024ea8e28fab8\": container with ID starting with a71406f78fe3abaced6860ec3a3b7d382ca625dfe1542348116024ea8e28fab8 not found: ID does not exist" Sep 11 09:11:29.964737 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.964692 2562 scope.go:117] "RemoveContainer" containerID="0ca1277fcdfeb3729ff5723171824b6a99547e2b125a735abe3c067447eee551" Sep 11 09:11:29.964949 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:11:29.964932 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ca1277fcdfeb3729ff5723171824b6a99547e2b125a735abe3c067447eee551\": container with ID starting with 0ca1277fcdfeb3729ff5723171824b6a99547e2b125a735abe3c067447eee551 not found: ID does not exist" containerID="0ca1277fcdfeb3729ff5723171824b6a99547e2b125a735abe3c067447eee551" Sep 11 09:11:29.964989 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.964954 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ca1277fcdfeb3729ff5723171824b6a99547e2b125a735abe3c067447eee551"} err="failed to get container status \"0ca1277fcdfeb3729ff5723171824b6a99547e2b125a735abe3c067447eee551\": rpc error: code = NotFound desc = could not find container \"0ca1277fcdfeb3729ff5723171824b6a99547e2b125a735abe3c067447eee551\": container with ID starting with 0ca1277fcdfeb3729ff5723171824b6a99547e2b125a735abe3c067447eee551 not found: ID does not exist" Sep 11 09:11:29.964989 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.964968 2562 scope.go:117] "RemoveContainer" containerID="ea04370b041d0090610b5e2a1e11ad14995ca1c8eebe402727f146488778183d" Sep 11 09:11:29.965201 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:11:29.965184 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea04370b041d0090610b5e2a1e11ad14995ca1c8eebe402727f146488778183d\": container with ID starting with ea04370b041d0090610b5e2a1e11ad14995ca1c8eebe402727f146488778183d not found: ID does not exist" containerID="ea04370b041d0090610b5e2a1e11ad14995ca1c8eebe402727f146488778183d" Sep 11 09:11:29.965240 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.965205 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea04370b041d0090610b5e2a1e11ad14995ca1c8eebe402727f146488778183d"} err="failed to get container status \"ea04370b041d0090610b5e2a1e11ad14995ca1c8eebe402727f146488778183d\": rpc error: code = NotFound desc = could not find container \"ea04370b041d0090610b5e2a1e11ad14995ca1c8eebe402727f146488778183d\": container with ID starting with ea04370b041d0090610b5e2a1e11ad14995ca1c8eebe402727f146488778183d not found: ID does not exist" Sep 11 09:11:29.965240 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.965224 2562 scope.go:117] "RemoveContainer" containerID="097f57d58fd9f7e237370e8239fcbf2bde41f678cf7c42811d8161b5a038f9ca" Sep 11 09:11:29.965458 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:11:29.965441 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"097f57d58fd9f7e237370e8239fcbf2bde41f678cf7c42811d8161b5a038f9ca\": container with ID starting with 097f57d58fd9f7e237370e8239fcbf2bde41f678cf7c42811d8161b5a038f9ca not found: ID does not exist" containerID="097f57d58fd9f7e237370e8239fcbf2bde41f678cf7c42811d8161b5a038f9ca" Sep 11 09:11:29.965506 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.965462 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"097f57d58fd9f7e237370e8239fcbf2bde41f678cf7c42811d8161b5a038f9ca"} err="failed to get container status \"097f57d58fd9f7e237370e8239fcbf2bde41f678cf7c42811d8161b5a038f9ca\": rpc error: code = NotFound desc = could not find container \"097f57d58fd9f7e237370e8239fcbf2bde41f678cf7c42811d8161b5a038f9ca\": container with ID starting with 097f57d58fd9f7e237370e8239fcbf2bde41f678cf7c42811d8161b5a038f9ca not found: ID does not exist" Sep 11 09:11:29.976887 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.976869 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-registry-tls\") pod \"e5dad503-5e52-4a1b-89fd-d7abec29726c\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " Sep 11 09:11:29.976954 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.976894 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-ui-tls\") pod \"e5dad503-5e52-4a1b-89fd-d7abec29726c\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " Sep 11 09:11:29.976954 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.976923 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-offline-tls\") pod \"e5dad503-5e52-4a1b-89fd-d7abec29726c\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " Sep 11 09:11:29.977035 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.976958 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-online-tls\") pod \"e5dad503-5e52-4a1b-89fd-d7abec29726c\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " Sep 11 09:11:29.977035 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.976974 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5xdq\" (UniqueName: \"kubernetes.io/projected/e5dad503-5e52-4a1b-89fd-d7abec29726c-kube-api-access-d5xdq\") pod \"e5dad503-5e52-4a1b-89fd-d7abec29726c\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " Sep 11 09:11:29.977035 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.977009 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e5dad503-5e52-4a1b-89fd-d7abec29726c-feast-data\") pod \"e5dad503-5e52-4a1b-89fd-d7abec29726c\" (UID: \"e5dad503-5e52-4a1b-89fd-d7abec29726c\") " Sep 11 09:11:29.977662 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.977628 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5dad503-5e52-4a1b-89fd-d7abec29726c-feast-data" (OuterVolumeSpecName: "feast-data") pod "e5dad503-5e52-4a1b-89fd-d7abec29726c" (UID: "e5dad503-5e52-4a1b-89fd-d7abec29726c"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 09:11:29.978879 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.978852 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "e5dad503-5e52-4a1b-89fd-d7abec29726c" (UID: "e5dad503-5e52-4a1b-89fd-d7abec29726c"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 09:11:29.978972 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.978912 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-online-tls" (OuterVolumeSpecName: "online-tls") pod "e5dad503-5e52-4a1b-89fd-d7abec29726c" (UID: "e5dad503-5e52-4a1b-89fd-d7abec29726c"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 09:11:29.979019 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.978983 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5dad503-5e52-4a1b-89fd-d7abec29726c-kube-api-access-d5xdq" (OuterVolumeSpecName: "kube-api-access-d5xdq") pod "e5dad503-5e52-4a1b-89fd-d7abec29726c" (UID: "e5dad503-5e52-4a1b-89fd-d7abec29726c"). InnerVolumeSpecName "kube-api-access-d5xdq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 09:11:29.979178 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.979160 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e5dad503-5e52-4a1b-89fd-d7abec29726c" (UID: "e5dad503-5e52-4a1b-89fd-d7abec29726c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 09:11:29.979262 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:29.979177 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "e5dad503-5e52-4a1b-89fd-d7abec29726c" (UID: "e5dad503-5e52-4a1b-89fd-d7abec29726c"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 09:11:30.077464 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:30.077436 2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-registry-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:11:30.077464 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:30.077457 2562 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-ui-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:11:30.077464 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:30.077466 2562 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-offline-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:11:30.077651 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:30.077473 2562 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e5dad503-5e52-4a1b-89fd-d7abec29726c-online-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:11:30.077651 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:30.077482 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5xdq\" (UniqueName: \"kubernetes.io/projected/e5dad503-5e52-4a1b-89fd-d7abec29726c-kube-api-access-d5xdq\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:11:30.077651 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:30.077490 2562 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e5dad503-5e52-4a1b-89fd-d7abec29726c-feast-data\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:11:30.925715 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:30.925681 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" event={"ID":"e5dad503-5e52-4a1b-89fd-d7abec29726c","Type":"ContainerDied","Data":"d26ef43ec55ae602ce8576c97da85576096f0840e05f3c310f78471e2080999a"} Sep 11 09:11:30.926107 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:30.925732 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682" Sep 11 09:11:30.926107 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:30.925736 2562 scope.go:117] "RemoveContainer" containerID="debdd6d352804c726a8243359fc189b74dcda5d2fcc149a5ad5c5c4e8838f522" Sep 11 09:11:30.934097 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:30.934076 2562 scope.go:117] "RemoveContainer" containerID="751b0efef9bc55ae19f4f4c410a487a76d99e27eb705847743ebc0cc496c2ef8" Sep 11 09:11:30.941233 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:30.941218 2562 scope.go:117] "RemoveContainer" containerID="7051bcee3fe8e8ee6b73fc6067503a10502a18d832214d19eea4eefb1cd64eaf" Sep 11 09:11:30.947778 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:30.947755 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682"] Sep 11 09:11:30.948336 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:30.948321 2562 scope.go:117] "RemoveContainer" containerID="49ed1824e563bb3b0bb2f6e423172b3ce578b0fb28de9204876effd5f95f33dc" Sep 11 09:11:30.951571 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:30.951549 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-mv682"] Sep 11 09:11:30.955429 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:30.955400 2562 scope.go:117] "RemoveContainer" containerID="b0256e8023ba7788be209eabfa65a95e11ba529542a211d8644b3692c9ff016f" Sep 11 09:11:30.964044 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:30.964016 2562 scope.go:117] "RemoveContainer" containerID="e8e02ea272d4114740215a06aca90baff7fcf2b4ffbf08d0880a07fbaa8421f3" Sep 11 09:11:31.571059 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:31.571026 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" path="/var/lib/kubelet/pods/c8534adb-97a3-45ba-ab15-a88ea93658c2/volumes" Sep 11 09:11:31.571584 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:31.571569 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" path="/var/lib/kubelet/pods/e5dad503-5e52-4a1b-89fd-d7abec29726c/volumes" Sep 11 09:11:33.559210 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:33.559182 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/ovn-acl-logging/0.log" Sep 11 09:11:33.560907 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:33.560885 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/ovn-acl-logging/0.log" Sep 11 09:11:43.105414 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:43.105377 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:11:43.166632 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:43.166606 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:11:48.059368 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:48.059336 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:11:48.119369 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:48.119336 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:11:48.176660 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:48.176635 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-znm7z"] Sep 11 09:11:48.176828 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:48.176810 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-znm7z" podUID="0749a41c-e0d9-4e10-9c2c-fc26a7ec41ba" containerName="manager" containerID="cri-o://90bd4f6fed5ee59e93cc5aa74ad7ef044104d4daaa0b0652bcb12306bf18b383" gracePeriod=10 Sep 11 09:11:48.414121 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:48.414098 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-znm7z" Sep 11 09:11:48.493363 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:48.493339 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbsmg\" (UniqueName: \"kubernetes.io/projected/0749a41c-e0d9-4e10-9c2c-fc26a7ec41ba-kube-api-access-sbsmg\") pod \"0749a41c-e0d9-4e10-9c2c-fc26a7ec41ba\" (UID: \"0749a41c-e0d9-4e10-9c2c-fc26a7ec41ba\") " Sep 11 09:11:48.495352 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:48.495326 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0749a41c-e0d9-4e10-9c2c-fc26a7ec41ba-kube-api-access-sbsmg" (OuterVolumeSpecName: "kube-api-access-sbsmg") pod "0749a41c-e0d9-4e10-9c2c-fc26a7ec41ba" (UID: "0749a41c-e0d9-4e10-9c2c-fc26a7ec41ba"). InnerVolumeSpecName "kube-api-access-sbsmg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 09:11:48.594571 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:48.594514 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbsmg\" (UniqueName: \"kubernetes.io/projected/0749a41c-e0d9-4e10-9c2c-fc26a7ec41ba-kube-api-access-sbsmg\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:11:48.988022 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:48.987989 2562 generic.go:356] "Generic (PLEG): container finished" podID="0749a41c-e0d9-4e10-9c2c-fc26a7ec41ba" containerID="90bd4f6fed5ee59e93cc5aa74ad7ef044104d4daaa0b0652bcb12306bf18b383" exitCode=0 Sep 11 09:11:48.988153 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:48.988034 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-znm7z" Sep 11 09:11:48.988153 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:48.988066 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-znm7z" event={"ID":"0749a41c-e0d9-4e10-9c2c-fc26a7ec41ba","Type":"ContainerDied","Data":"90bd4f6fed5ee59e93cc5aa74ad7ef044104d4daaa0b0652bcb12306bf18b383"} Sep 11 09:11:48.988153 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:48.988106 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-znm7z" event={"ID":"0749a41c-e0d9-4e10-9c2c-fc26a7ec41ba","Type":"ContainerDied","Data":"7218a3171a9b84818e007bb85aaf47fc6f35175e4881e6fc49ba0f5b5c495d54"} Sep 11 09:11:48.988153 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:48.988128 2562 scope.go:117] "RemoveContainer" containerID="90bd4f6fed5ee59e93cc5aa74ad7ef044104d4daaa0b0652bcb12306bf18b383" Sep 11 09:11:48.996093 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:48.996071 2562 scope.go:117] "RemoveContainer" containerID="90bd4f6fed5ee59e93cc5aa74ad7ef044104d4daaa0b0652bcb12306bf18b383" Sep 11 09:11:48.996391 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:11:48.996366 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90bd4f6fed5ee59e93cc5aa74ad7ef044104d4daaa0b0652bcb12306bf18b383\": container with ID starting with 90bd4f6fed5ee59e93cc5aa74ad7ef044104d4daaa0b0652bcb12306bf18b383 not found: ID does not exist" containerID="90bd4f6fed5ee59e93cc5aa74ad7ef044104d4daaa0b0652bcb12306bf18b383" Sep 11 09:11:48.996500 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:48.996395 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90bd4f6fed5ee59e93cc5aa74ad7ef044104d4daaa0b0652bcb12306bf18b383"} err="failed to get container status \"90bd4f6fed5ee59e93cc5aa74ad7ef044104d4daaa0b0652bcb12306bf18b383\": rpc error: code = NotFound desc = could not find container \"90bd4f6fed5ee59e93cc5aa74ad7ef044104d4daaa0b0652bcb12306bf18b383\": container with ID starting with 90bd4f6fed5ee59e93cc5aa74ad7ef044104d4daaa0b0652bcb12306bf18b383 not found: ID does not exist" Sep 11 09:11:49.007693 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:49.007670 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-znm7z"] Sep 11 09:11:49.015857 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:49.015839 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-znm7z"] Sep 11 09:11:49.571406 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:49.571370 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0749a41c-e0d9-4e10-9c2c-fc26a7ec41ba" path="/var/lib/kubelet/pods/0749a41c-e0d9-4e10-9c2c-fc26a7ec41ba/volumes" Sep 11 09:11:53.127348 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127297 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-kwfhv"] Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127520 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="feast-apply" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127530 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="feast-apply" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127539 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="offline" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127544 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="offline" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127551 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="online" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127556 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="online" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127562 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="feast-apply" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127569 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="feast-apply" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127574 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="feast-init" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127579 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="feast-init" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127590 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0749a41c-e0d9-4e10-9c2c-fc26a7ec41ba" containerName="manager" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127595 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0749a41c-e0d9-4e10-9c2c-fc26a7ec41ba" containerName="manager" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127600 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="feast-init" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127605 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="feast-init" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127612 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="offline" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127617 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="offline" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127623 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="online" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127627 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="online" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127638 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="registry" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127643 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="registry" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127648 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="ui" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127653 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="ui" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127661 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="ui" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127665 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="ui" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127701 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="ui" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127708 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="registry" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127716 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="offline" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127721 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="online" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127726 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="ui" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127730 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8534adb-97a3-45ba-ab15-a88ea93658c2" containerName="online" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127735 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5dad503-5e52-4a1b-89fd-d7abec29726c" containerName="offline" Sep 11 09:11:53.127734 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.127741 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0749a41c-e0d9-4e10-9c2c-fc26a7ec41ba" containerName="manager" Sep 11 09:11:53.131882 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.131865 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kwfhv" Sep 11 09:11:53.134332 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.134298 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 11 09:11:53.134447 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.134417 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 11 09:11:53.135043 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.135024 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-bttrw\"" Sep 11 09:11:53.140876 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.140852 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-kwfhv"] Sep 11 09:11:53.221842 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.221821 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8mjp\" (UniqueName: \"kubernetes.io/projected/9d284e65-018e-4338-b2a9-763a5fdc41ac-kube-api-access-l8mjp\") pod \"feast-operator-controller-manager-689b877b4b-kwfhv\" (UID: \"9d284e65-018e-4338-b2a9-763a5fdc41ac\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kwfhv" Sep 11 09:11:53.322727 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.322704 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8mjp\" (UniqueName: \"kubernetes.io/projected/9d284e65-018e-4338-b2a9-763a5fdc41ac-kube-api-access-l8mjp\") pod \"feast-operator-controller-manager-689b877b4b-kwfhv\" (UID: \"9d284e65-018e-4338-b2a9-763a5fdc41ac\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kwfhv" Sep 11 09:11:53.337752 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.337728 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8mjp\" (UniqueName: \"kubernetes.io/projected/9d284e65-018e-4338-b2a9-763a5fdc41ac-kube-api-access-l8mjp\") pod \"feast-operator-controller-manager-689b877b4b-kwfhv\" (UID: \"9d284e65-018e-4338-b2a9-763a5fdc41ac\") " pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kwfhv" Sep 11 09:11:53.442647 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.442592 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kwfhv" Sep 11 09:11:53.711976 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:53.711909 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-689b877b4b-kwfhv"] Sep 11 09:11:53.715111 ip-10-0-141-169 kubenswrapper[2562]: W0911 09:11:53.715086 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d284e65_018e_4338_b2a9_763a5fdc41ac.slice/crio-56c5e9745aa96866df2c0ebaed4a67acb309d1051d5fcd77dce90656cef8baa3 WatchSource:0}: Error finding container 56c5e9745aa96866df2c0ebaed4a67acb309d1051d5fcd77dce90656cef8baa3: Status 404 returned error can't find the container with id 56c5e9745aa96866df2c0ebaed4a67acb309d1051d5fcd77dce90656cef8baa3 Sep 11 09:11:54.012036 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:54.011997 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kwfhv" event={"ID":"9d284e65-018e-4338-b2a9-763a5fdc41ac","Type":"ContainerStarted","Data":"21655912da78217fad908da81fb28d7d497e109367bf19cd8101c91b2662c510"} Sep 11 09:11:54.012229 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:54.012045 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kwfhv" event={"ID":"9d284e65-018e-4338-b2a9-763a5fdc41ac","Type":"ContainerStarted","Data":"56c5e9745aa96866df2c0ebaed4a67acb309d1051d5fcd77dce90656cef8baa3"} Sep 11 09:11:54.012229 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:54.012122 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kwfhv" Sep 11 09:11:54.028827 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:11:54.028787 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kwfhv" podStartSLOduration=1.028775347 podStartE2EDuration="1.028775347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 09:11:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 09:11:54.028126234 +0000 UTC m=+1520.928024532" watchObservedRunningTime="2026-09-11 09:11:54.028775347 +0000 UTC m=+1520.928673656" Sep 11 09:12:05.017534 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:05.017459 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-689b877b4b-kwfhv" Sep 11 09:12:07.461315 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:07.461261 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:12:07.561721 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:07.561690 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:12:12.161622 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.161570 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:12:12.264706 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.264672 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:12:12.274242 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.274191 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq"] Sep 11 09:12:12.277800 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.277782 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:12.280003 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.279983 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 11 09:12:12.280130 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.280110 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 11 09:12:12.280226 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.280205 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-hlhnb\"" Sep 11 09:12:12.280312 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.280005 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 11 09:12:12.280389 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.279998 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 11 09:12:12.280493 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.280477 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 11 09:12:12.280903 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.280556 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 11 09:12:12.290140 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.290073 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq"] Sep 11 09:12:12.347359 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.347325 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-94dwq\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:12.347535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.347370 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-94dwq\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:12.347535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.347403 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-94dwq\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:12.347535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.347460 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-94dwq\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:12.347535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.347486 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p55kw\" (UniqueName: \"kubernetes.io/projected/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-kube-api-access-p55kw\") pod \"feast-simple-feast-setup-57576b8fcf-94dwq\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:12.347535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.347522 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-94dwq\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:12.448591 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.448509 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-94dwq\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:12.448591 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.448547 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-94dwq\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:12.448591 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.448576 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-94dwq\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:12.448850 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.448606 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-94dwq\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:12.448850 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.448629 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p55kw\" (UniqueName: \"kubernetes.io/projected/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-kube-api-access-p55kw\") pod \"feast-simple-feast-setup-57576b8fcf-94dwq\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:12.448850 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.448660 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-94dwq\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:12.448850 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:12:12.448706 2562 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 11 09:12:12.448850 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:12:12.448756 2562 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Sep 11 09:12:12.448850 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:12:12.448769 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-offline-tls podName:e2fadcf8-72ea-44f1-9740-3a0ea775b17d nodeName:}" failed. No retries permitted until 2026-09-11 09:12:12.948752807 +0000 UTC m=+1539.848651088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-offline-tls") pod "feast-simple-feast-setup-57576b8fcf-94dwq" (UID: "e2fadcf8-72ea-44f1-9740-3a0ea775b17d") : secret "feast-simple-feast-setup-offline-tls" not found Sep 11 09:12:12.448850 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:12:12.448808 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-online-tls podName:e2fadcf8-72ea-44f1-9740-3a0ea775b17d nodeName:}" failed. No retries permitted until 2026-09-11 09:12:12.948790721 +0000 UTC m=+1539.848689012 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-online-tls") pod "feast-simple-feast-setup-57576b8fcf-94dwq" (UID: "e2fadcf8-72ea-44f1-9740-3a0ea775b17d") : secret "feast-simple-feast-setup-online-tls" not found Sep 11 09:12:12.448850 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:12:12.448708 2562 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 11 09:12:12.449149 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:12:12.448889 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-registry-tls podName:e2fadcf8-72ea-44f1-9740-3a0ea775b17d nodeName:}" failed. No retries permitted until 2026-09-11 09:12:12.948873943 +0000 UTC m=+1539.848772234 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-registry-tls") pod "feast-simple-feast-setup-57576b8fcf-94dwq" (UID: "e2fadcf8-72ea-44f1-9740-3a0ea775b17d") : secret "feast-simple-feast-setup-registry-tls" not found Sep 11 09:12:12.449149 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.448955 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-94dwq\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:12.451388 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.451360 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-94dwq\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:12.457650 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.457614 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p55kw\" (UniqueName: \"kubernetes.io/projected/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-kube-api-access-p55kw\") pod \"feast-simple-feast-setup-57576b8fcf-94dwq\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:12.487972 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.487949 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:12:12.588822 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.588784 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:12:12.688195 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.688141 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:12:12.952617 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.952587 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-94dwq\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:12.952813 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.952630 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-94dwq\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:12.952813 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.952666 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-94dwq\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:12.955074 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.955044 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-94dwq\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:12.955188 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.955082 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-94dwq\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:12.955188 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:12.955097 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-94dwq\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:13.188296 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:13.188258 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:13.465657 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:13.465625 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq"] Sep 11 09:12:13.468362 ip-10-0-141-169 kubenswrapper[2562]: W0911 09:12:13.468340 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2fadcf8_72ea_44f1_9740_3a0ea775b17d.slice/crio-645371831d1ab51c493277800bc94955cd4244ba067206537dbfdef1421b01df WatchSource:0}: Error finding container 645371831d1ab51c493277800bc94955cd4244ba067206537dbfdef1421b01df: Status 404 returned error can't find the container with id 645371831d1ab51c493277800bc94955cd4244ba067206537dbfdef1421b01df Sep 11 09:12:14.078162 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:14.078122 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" event={"ID":"e2fadcf8-72ea-44f1-9740-3a0ea775b17d","Type":"ContainerStarted","Data":"ed060cbb79663881ef352921b24419b6adcc046943fffc33234d357d70487f8c"} Sep 11 09:12:14.078333 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:14.078187 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" event={"ID":"e2fadcf8-72ea-44f1-9740-3a0ea775b17d","Type":"ContainerStarted","Data":"645371831d1ab51c493277800bc94955cd4244ba067206537dbfdef1421b01df"} Sep 11 09:12:19.095430 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:19.095395 2562 generic.go:356] "Generic (PLEG): container finished" podID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerID="ed060cbb79663881ef352921b24419b6adcc046943fffc33234d357d70487f8c" exitCode=0 Sep 11 09:12:19.095985 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:19.095471 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" event={"ID":"e2fadcf8-72ea-44f1-9740-3a0ea775b17d","Type":"ContainerDied","Data":"ed060cbb79663881ef352921b24419b6adcc046943fffc33234d357d70487f8c"} Sep 11 09:12:20.101858 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:20.101821 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" event={"ID":"e2fadcf8-72ea-44f1-9740-3a0ea775b17d","Type":"ContainerStarted","Data":"81302003c8436219856c5e0fa95a06dc7e9c6503fab069a4deb63c2445113689"} Sep 11 09:12:25.119078 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:25.119002 2562 generic.go:356] "Generic (PLEG): container finished" podID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerID="81302003c8436219856c5e0fa95a06dc7e9c6503fab069a4deb63c2445113689" exitCode=0 Sep 11 09:12:25.119078 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:25.119048 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" event={"ID":"e2fadcf8-72ea-44f1-9740-3a0ea775b17d","Type":"ContainerDied","Data":"81302003c8436219856c5e0fa95a06dc7e9c6503fab069a4deb63c2445113689"} Sep 11 09:12:26.127856 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:26.127810 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" event={"ID":"e2fadcf8-72ea-44f1-9740-3a0ea775b17d","Type":"ContainerStarted","Data":"ea04366f3643414d85a5999e939dfd40c5b95ce909017eab9edb8758257856ff"} Sep 11 09:12:26.128339 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:26.127879 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" event={"ID":"e2fadcf8-72ea-44f1-9740-3a0ea775b17d","Type":"ContainerStarted","Data":"c430effe7493f234eb089938c0c68232eb3da9f11c62640e74f1e08a452f53eb"} Sep 11 09:12:26.128339 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:26.127901 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" event={"ID":"e2fadcf8-72ea-44f1-9740-3a0ea775b17d","Type":"ContainerStarted","Data":"90bf0ab9146c23fb7f5a0421a72ee1f964371566ef890e8f282907f1200fcaf3"} Sep 11 09:12:26.128339 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:26.127920 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" event={"ID":"e2fadcf8-72ea-44f1-9740-3a0ea775b17d","Type":"ContainerStarted","Data":"905e299cbcb1bad0fbcf9bc44b258eb85b25ece280054afe990a5318e4081ad5"} Sep 11 09:12:26.154267 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:26.154208 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" podStartSLOduration=14.154185465 podStartE2EDuration="14.154185465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 09:12:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 09:12:26.151728728 +0000 UTC m=+1553.051627028" watchObservedRunningTime="2026-09-11 09:12:26.154185465 +0000 UTC m=+1553.054083763" Sep 11 09:12:28.189142 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:28.188996 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:28.189142 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:28.189066 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:28.189142 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:28.189086 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:28.190209 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:28.189902 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:28.191053 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:28.191020 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused" Sep 11 09:12:28.191162 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:28.191048 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.20:8816: connect: connection refused" Sep 11 09:12:28.191162 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:28.191062 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused" Sep 11 09:12:28.191162 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:28.191131 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.20:6571: connect: connection refused" Sep 11 09:12:31.188905 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:31.188850 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.20:6571: connect: connection refused" Sep 11 09:12:31.188905 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:31.188897 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused" Sep 11 09:12:31.189480 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:31.188896 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused" Sep 11 09:12:31.189480 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:31.189332 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.20:8816: connect: connection refused" Sep 11 09:12:33.188966 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:33.188925 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:33.188966 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:33.188973 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:33.189586 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:33.188987 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:33.189586 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:33.188998 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:34.189618 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:34.189586 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:34.189973 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:34.189719 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:34.190097 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:34.190073 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:34.190421 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:34.190401 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:34.190533 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:34.190445 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:34.194844 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:34.194822 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:35.167408 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:35.167382 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:12:35.170260 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:12:35.170231 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:13:36.641703 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:36.641617 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:13:36.646207 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:36.646181 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq"] Sep 11 09:13:36.646497 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:36.646466 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="registry" containerID="cri-o://905e299cbcb1bad0fbcf9bc44b258eb85b25ece280054afe990a5318e4081ad5" gracePeriod=30 Sep 11 09:13:36.646587 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:36.646503 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="offline" containerID="cri-o://c430effe7493f234eb089938c0c68232eb3da9f11c62640e74f1e08a452f53eb" gracePeriod=30 Sep 11 09:13:36.646587 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:36.646519 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="online" containerID="cri-o://90bf0ab9146c23fb7f5a0421a72ee1f964371566ef890e8f282907f1200fcaf3" gracePeriod=30 Sep 11 09:13:36.646701 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:36.646550 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="ui" containerID="cri-o://ea04366f3643414d85a5999e939dfd40c5b95ce909017eab9edb8758257856ff" gracePeriod=30 Sep 11 09:13:36.742165 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:36.742134 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:13:37.153453 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.153421 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:13:37.322314 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.322258 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:13:37.340184 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.340151 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn"] Sep 11 09:13:37.343926 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.343902 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:37.346628 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.346605 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-vts79\"" Sep 11 09:13:37.362520 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.362448 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn"] Sep 11 09:13:37.380634 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.380607 2562 generic.go:356] "Generic (PLEG): container finished" podID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerID="ea04366f3643414d85a5999e939dfd40c5b95ce909017eab9edb8758257856ff" exitCode=0 Sep 11 09:13:37.380764 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.380665 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" event={"ID":"e2fadcf8-72ea-44f1-9740-3a0ea775b17d","Type":"ContainerDied","Data":"ea04366f3643414d85a5999e939dfd40c5b95ce909017eab9edb8758257856ff"} Sep 11 09:13:37.437166 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.437090 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhtdc\" (UniqueName: \"kubernetes.io/projected/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-kube-api-access-hhtdc\") pod \"feast-simple-feast-setup-57576b8fcf-zx5bn\" (UID: \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:37.437166 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.437158 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zx5bn\" (UID: \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:37.437369 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.437209 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zx5bn\" (UID: \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:37.437369 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.437235 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-zx5bn\" (UID: \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:37.437369 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.437270 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zx5bn\" (UID: \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:37.437369 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.437289 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zx5bn\" (UID: \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:37.489692 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.489651 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:13:37.538287 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.538263 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-zx5bn\" (UID: \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:37.538420 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.538297 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zx5bn\" (UID: \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:37.538420 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.538332 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zx5bn\" (UID: \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:37.538646 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.538631 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhtdc\" (UniqueName: \"kubernetes.io/projected/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-kube-api-access-hhtdc\") pod \"feast-simple-feast-setup-57576b8fcf-zx5bn\" (UID: \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:37.538724 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.538676 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zx5bn\" (UID: \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:37.538790 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.538723 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zx5bn\" (UID: \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:37.538849 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.538834 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-feast-data\") pod \"feast-simple-feast-setup-57576b8fcf-zx5bn\" (UID: \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:37.540914 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.540884 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-registry-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zx5bn\" (UID: \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:37.541004 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.540955 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-ui-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zx5bn\" (UID: \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:37.541317 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.541285 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-offline-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zx5bn\" (UID: \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:37.541487 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.541465 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-online-tls\") pod \"feast-simple-feast-setup-57576b8fcf-zx5bn\" (UID: \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:37.546151 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.546131 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhtdc\" (UniqueName: \"kubernetes.io/projected/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-kube-api-access-hhtdc\") pod \"feast-simple-feast-setup-57576b8fcf-zx5bn\" (UID: \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\") " pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:37.589509 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.589471 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:13:37.655710 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.655691 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:37.689185 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.689122 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:13:37.752678 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.752653 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:13:37.928210 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:37.928185 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn"] Sep 11 09:13:37.930956 ip-10-0-141-169 kubenswrapper[2562]: W0911 09:13:37.930931 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee47b745_ab7c_4fc3_b2a3_de338a7d7af5.slice/crio-3c4964e339abe5e93092f9598a2e4921320fde423bfc19e60ac8fd037863e364 WatchSource:0}: Error finding container 3c4964e339abe5e93092f9598a2e4921320fde423bfc19e60ac8fd037863e364: Status 404 returned error can't find the container with id 3c4964e339abe5e93092f9598a2e4921320fde423bfc19e60ac8fd037863e364 Sep 11 09:13:38.387915 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:38.387885 2562 generic.go:356] "Generic (PLEG): container finished" podID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerID="90bf0ab9146c23fb7f5a0421a72ee1f964371566ef890e8f282907f1200fcaf3" exitCode=0 Sep 11 09:13:38.388078 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:38.387959 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" event={"ID":"e2fadcf8-72ea-44f1-9740-3a0ea775b17d","Type":"ContainerDied","Data":"90bf0ab9146c23fb7f5a0421a72ee1f964371566ef890e8f282907f1200fcaf3"} Sep 11 09:13:38.389468 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:38.389440 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" event={"ID":"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5","Type":"ContainerStarted","Data":"c14dc0506f2ba1ec098cf74d0ade97ed78b3929b31762f6432fd818db98705fe"} Sep 11 09:13:38.389573 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:38.389488 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" event={"ID":"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5","Type":"ContainerStarted","Data":"3c4964e339abe5e93092f9598a2e4921320fde423bfc19e60ac8fd037863e364"} Sep 11 09:13:40.882823 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:40.882789 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:13:43.413728 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:43.413652 2562 generic.go:356] "Generic (PLEG): container finished" podID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerID="c14dc0506f2ba1ec098cf74d0ade97ed78b3929b31762f6432fd818db98705fe" exitCode=0 Sep 11 09:13:43.414039 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:43.413726 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" event={"ID":"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5","Type":"ContainerDied","Data":"c14dc0506f2ba1ec098cf74d0ade97ed78b3929b31762f6432fd818db98705fe"} Sep 11 09:13:44.190984 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:44.190940 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused" Sep 11 09:13:44.418861 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:44.418826 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" event={"ID":"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5","Type":"ContainerStarted","Data":"6530531a4de80a93e18dd2d8b679bad4bacfb5b21660d077691a2b340e4c5616"} Sep 11 09:13:45.167275 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:45.167232 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused" Sep 11 09:13:49.438875 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:49.438845 2562 generic.go:356] "Generic (PLEG): container finished" podID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerID="6530531a4de80a93e18dd2d8b679bad4bacfb5b21660d077691a2b340e4c5616" exitCode=0 Sep 11 09:13:49.439160 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:49.438903 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" event={"ID":"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5","Type":"ContainerDied","Data":"6530531a4de80a93e18dd2d8b679bad4bacfb5b21660d077691a2b340e4c5616"} Sep 11 09:13:50.446824 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:50.446752 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" event={"ID":"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5","Type":"ContainerStarted","Data":"9ae1790a6d24b5495605cc1547048832e6540a3ae029fd606d03a6c1aff78cef"} Sep 11 09:13:50.446824 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:50.446806 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" event={"ID":"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5","Type":"ContainerStarted","Data":"ae1a29ce0cc8dad658dee7d1d1560f4c63448a54b70084efd228fdab1da2732f"} Sep 11 09:13:50.446824 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:50.446826 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" event={"ID":"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5","Type":"ContainerStarted","Data":"58bccef913ae43fcdfe54f2f032b8271cd5d935ed8b741b8289e1b076d4c9612"} Sep 11 09:13:50.447440 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:50.446838 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" event={"ID":"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5","Type":"ContainerStarted","Data":"b01e3ba000ce4caedcdc2f0063129c719ff2484a99be228e111aae4e231f6472"} Sep 11 09:13:50.470977 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:50.470659 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" podStartSLOduration=13.470641244 podStartE2EDuration="13.470641244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 09:13:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 09:13:50.469698933 +0000 UTC m=+1637.369597235" watchObservedRunningTime="2026-09-11 09:13:50.470641244 +0000 UTC m=+1637.370539542" Sep 11 09:13:52.656231 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:52.656133 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:52.656231 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:52.656194 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:52.656231 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:52.656215 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:52.656231 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:52.656234 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:52.658081 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:52.658040 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.21:8816: connect: connection refused" Sep 11 09:13:52.658220 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:52.658066 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.21:6571: connect: connection refused" Sep 11 09:13:52.658220 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:52.658072 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.21:8443: connect: connection refused" Sep 11 09:13:52.658220 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:52.658128 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="online" probeResult="failure" output="Get \"https://10.133.0.21:6567/health\": dial tcp 10.133.0.21:6567: connect: connection refused" Sep 11 09:13:54.190517 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:54.190463 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused" Sep 11 09:13:55.167352 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:55.167283 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused" Sep 11 09:13:55.656868 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:55.656812 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.21:8443: connect: connection refused" Sep 11 09:13:55.656868 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:55.656856 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="online" probeResult="failure" output="Get \"https://10.133.0.21:6567/health\": dial tcp 10.133.0.21:6567: connect: connection refused" Sep 11 09:13:55.657418 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:55.656814 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.21:6571: connect: connection refused" Sep 11 09:13:55.657418 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:55.657008 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.21:8816: connect: connection refused" Sep 11 09:13:57.656627 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:57.656591 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:57.657062 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:57.656642 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:57.657062 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:57.656661 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:57.657062 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:57.656684 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:58.657076 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:58.657047 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:58.657496 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:58.657367 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:58.657496 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:58.657482 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:58.662243 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:58.662220 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:59.488871 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:59.488844 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:59.489046 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:59.488925 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:59.489130 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:59.489111 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:13:59.491438 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:13:59.491416 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:14:04.191032 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:04.190986 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.20:8443: connect: connection refused" Sep 11 09:14:04.191463 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:04.191110 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:14:05.167557 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:05.167507 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.20:6567/health\": dial tcp 10.133.0.20:6567: connect: connection refused" Sep 11 09:14:05.167721 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:05.167663 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:14:07.286602 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.286577 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:14:07.358752 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.358722 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p55kw\" (UniqueName: \"kubernetes.io/projected/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-kube-api-access-p55kw\") pod \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " Sep 11 09:14:07.358894 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.358765 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-ui-tls\") pod \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " Sep 11 09:14:07.358894 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.358795 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-feast-data\") pod \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " Sep 11 09:14:07.359022 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.358915 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-offline-tls\") pod \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " Sep 11 09:14:07.359022 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.358974 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-registry-tls\") pod \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " Sep 11 09:14:07.359130 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.359033 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-online-tls\") pod \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\" (UID: \"e2fadcf8-72ea-44f1-9740-3a0ea775b17d\") " Sep 11 09:14:07.359338 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.359295 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-feast-data" (OuterVolumeSpecName: "feast-data") pod "e2fadcf8-72ea-44f1-9740-3a0ea775b17d" (UID: "e2fadcf8-72ea-44f1-9740-3a0ea775b17d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 09:14:07.361074 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.361046 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-online-tls" (OuterVolumeSpecName: "online-tls") pod "e2fadcf8-72ea-44f1-9740-3a0ea775b17d" (UID: "e2fadcf8-72ea-44f1-9740-3a0ea775b17d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 09:14:07.361189 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.361102 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "e2fadcf8-72ea-44f1-9740-3a0ea775b17d" (UID: "e2fadcf8-72ea-44f1-9740-3a0ea775b17d"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 09:14:07.361189 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.361165 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-kube-api-access-p55kw" (OuterVolumeSpecName: "kube-api-access-p55kw") pod "e2fadcf8-72ea-44f1-9740-3a0ea775b17d" (UID: "e2fadcf8-72ea-44f1-9740-3a0ea775b17d"). InnerVolumeSpecName "kube-api-access-p55kw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 09:14:07.361282 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.361264 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e2fadcf8-72ea-44f1-9740-3a0ea775b17d" (UID: "e2fadcf8-72ea-44f1-9740-3a0ea775b17d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 09:14:07.361342 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.361291 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "e2fadcf8-72ea-44f1-9740-3a0ea775b17d" (UID: "e2fadcf8-72ea-44f1-9740-3a0ea775b17d"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 09:14:07.460101 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.460035 2562 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-ui-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:14:07.460101 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.460066 2562 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-feast-data\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:14:07.460101 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.460076 2562 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-offline-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:14:07.460101 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.460084 2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-registry-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:14:07.460101 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.460091 2562 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-online-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:14:07.460101 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.460099 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p55kw\" (UniqueName: \"kubernetes.io/projected/e2fadcf8-72ea-44f1-9740-3a0ea775b17d-kube-api-access-p55kw\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:14:07.518614 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.518588 2562 generic.go:356] "Generic (PLEG): container finished" podID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerID="c430effe7493f234eb089938c0c68232eb3da9f11c62640e74f1e08a452f53eb" exitCode=137 Sep 11 09:14:07.518614 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.518610 2562 generic.go:356] "Generic (PLEG): container finished" podID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerID="905e299cbcb1bad0fbcf9bc44b258eb85b25ece280054afe990a5318e4081ad5" exitCode=137 Sep 11 09:14:07.518756 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.518649 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" Sep 11 09:14:07.518756 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.518676 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" event={"ID":"e2fadcf8-72ea-44f1-9740-3a0ea775b17d","Type":"ContainerDied","Data":"c430effe7493f234eb089938c0c68232eb3da9f11c62640e74f1e08a452f53eb"} Sep 11 09:14:07.518756 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.518725 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" event={"ID":"e2fadcf8-72ea-44f1-9740-3a0ea775b17d","Type":"ContainerDied","Data":"905e299cbcb1bad0fbcf9bc44b258eb85b25ece280054afe990a5318e4081ad5"} Sep 11 09:14:07.518756 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.518740 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq" event={"ID":"e2fadcf8-72ea-44f1-9740-3a0ea775b17d","Type":"ContainerDied","Data":"645371831d1ab51c493277800bc94955cd4244ba067206537dbfdef1421b01df"} Sep 11 09:14:07.518888 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.518758 2562 scope.go:117] "RemoveContainer" containerID="ea04366f3643414d85a5999e939dfd40c5b95ce909017eab9edb8758257856ff" Sep 11 09:14:07.527096 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.527078 2562 scope.go:117] "RemoveContainer" containerID="c430effe7493f234eb089938c0c68232eb3da9f11c62640e74f1e08a452f53eb" Sep 11 09:14:07.534218 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.534201 2562 scope.go:117] "RemoveContainer" containerID="90bf0ab9146c23fb7f5a0421a72ee1f964371566ef890e8f282907f1200fcaf3" Sep 11 09:14:07.539691 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.539673 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq"] Sep 11 09:14:07.542148 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.541549 2562 scope.go:117] "RemoveContainer" containerID="905e299cbcb1bad0fbcf9bc44b258eb85b25ece280054afe990a5318e4081ad5" Sep 11 09:14:07.543285 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.543267 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-94dwq"] Sep 11 09:14:07.548769 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.548755 2562 scope.go:117] "RemoveContainer" containerID="81302003c8436219856c5e0fa95a06dc7e9c6503fab069a4deb63c2445113689" Sep 11 09:14:07.555657 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.555643 2562 scope.go:117] "RemoveContainer" containerID="ed060cbb79663881ef352921b24419b6adcc046943fffc33234d357d70487f8c" Sep 11 09:14:07.562212 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.562194 2562 scope.go:117] "RemoveContainer" containerID="ea04366f3643414d85a5999e939dfd40c5b95ce909017eab9edb8758257856ff" Sep 11 09:14:07.562485 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:14:07.562467 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea04366f3643414d85a5999e939dfd40c5b95ce909017eab9edb8758257856ff\": container with ID starting with ea04366f3643414d85a5999e939dfd40c5b95ce909017eab9edb8758257856ff not found: ID does not exist" containerID="ea04366f3643414d85a5999e939dfd40c5b95ce909017eab9edb8758257856ff" Sep 11 09:14:07.562533 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.562493 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea04366f3643414d85a5999e939dfd40c5b95ce909017eab9edb8758257856ff"} err="failed to get container status \"ea04366f3643414d85a5999e939dfd40c5b95ce909017eab9edb8758257856ff\": rpc error: code = NotFound desc = could not find container \"ea04366f3643414d85a5999e939dfd40c5b95ce909017eab9edb8758257856ff\": container with ID starting with ea04366f3643414d85a5999e939dfd40c5b95ce909017eab9edb8758257856ff not found: ID does not exist" Sep 11 09:14:07.562533 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.562510 2562 scope.go:117] "RemoveContainer" containerID="c430effe7493f234eb089938c0c68232eb3da9f11c62640e74f1e08a452f53eb" Sep 11 09:14:07.562735 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:14:07.562717 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c430effe7493f234eb089938c0c68232eb3da9f11c62640e74f1e08a452f53eb\": container with ID starting with c430effe7493f234eb089938c0c68232eb3da9f11c62640e74f1e08a452f53eb not found: ID does not exist" containerID="c430effe7493f234eb089938c0c68232eb3da9f11c62640e74f1e08a452f53eb" Sep 11 09:14:07.562824 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.562741 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c430effe7493f234eb089938c0c68232eb3da9f11c62640e74f1e08a452f53eb"} err="failed to get container status \"c430effe7493f234eb089938c0c68232eb3da9f11c62640e74f1e08a452f53eb\": rpc error: code = NotFound desc = could not find container \"c430effe7493f234eb089938c0c68232eb3da9f11c62640e74f1e08a452f53eb\": container with ID starting with c430effe7493f234eb089938c0c68232eb3da9f11c62640e74f1e08a452f53eb not found: ID does not exist" Sep 11 09:14:07.562824 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.562758 2562 scope.go:117] "RemoveContainer" containerID="90bf0ab9146c23fb7f5a0421a72ee1f964371566ef890e8f282907f1200fcaf3" Sep 11 09:14:07.563033 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:14:07.563013 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90bf0ab9146c23fb7f5a0421a72ee1f964371566ef890e8f282907f1200fcaf3\": container with ID starting with 90bf0ab9146c23fb7f5a0421a72ee1f964371566ef890e8f282907f1200fcaf3 not found: ID does not exist" containerID="90bf0ab9146c23fb7f5a0421a72ee1f964371566ef890e8f282907f1200fcaf3" Sep 11 09:14:07.563079 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.563038 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90bf0ab9146c23fb7f5a0421a72ee1f964371566ef890e8f282907f1200fcaf3"} err="failed to get container status \"90bf0ab9146c23fb7f5a0421a72ee1f964371566ef890e8f282907f1200fcaf3\": rpc error: code = NotFound desc = could not find container \"90bf0ab9146c23fb7f5a0421a72ee1f964371566ef890e8f282907f1200fcaf3\": container with ID starting with 90bf0ab9146c23fb7f5a0421a72ee1f964371566ef890e8f282907f1200fcaf3 not found: ID does not exist" Sep 11 09:14:07.563079 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.563053 2562 scope.go:117] "RemoveContainer" containerID="905e299cbcb1bad0fbcf9bc44b258eb85b25ece280054afe990a5318e4081ad5" Sep 11 09:14:07.563263 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:14:07.563246 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"905e299cbcb1bad0fbcf9bc44b258eb85b25ece280054afe990a5318e4081ad5\": container with ID starting with 905e299cbcb1bad0fbcf9bc44b258eb85b25ece280054afe990a5318e4081ad5 not found: ID does not exist" containerID="905e299cbcb1bad0fbcf9bc44b258eb85b25ece280054afe990a5318e4081ad5" Sep 11 09:14:07.563435 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.563270 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"905e299cbcb1bad0fbcf9bc44b258eb85b25ece280054afe990a5318e4081ad5"} err="failed to get container status \"905e299cbcb1bad0fbcf9bc44b258eb85b25ece280054afe990a5318e4081ad5\": rpc error: code = NotFound desc = could not find container \"905e299cbcb1bad0fbcf9bc44b258eb85b25ece280054afe990a5318e4081ad5\": container with ID starting with 905e299cbcb1bad0fbcf9bc44b258eb85b25ece280054afe990a5318e4081ad5 not found: ID does not exist" Sep 11 09:14:07.563435 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.563290 2562 scope.go:117] "RemoveContainer" containerID="81302003c8436219856c5e0fa95a06dc7e9c6503fab069a4deb63c2445113689" Sep 11 09:14:07.563608 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:14:07.563590 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81302003c8436219856c5e0fa95a06dc7e9c6503fab069a4deb63c2445113689\": container with ID starting with 81302003c8436219856c5e0fa95a06dc7e9c6503fab069a4deb63c2445113689 not found: ID does not exist" containerID="81302003c8436219856c5e0fa95a06dc7e9c6503fab069a4deb63c2445113689" Sep 11 09:14:07.563662 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.563612 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81302003c8436219856c5e0fa95a06dc7e9c6503fab069a4deb63c2445113689"} err="failed to get container status \"81302003c8436219856c5e0fa95a06dc7e9c6503fab069a4deb63c2445113689\": rpc error: code = NotFound desc = could not find container \"81302003c8436219856c5e0fa95a06dc7e9c6503fab069a4deb63c2445113689\": container with ID starting with 81302003c8436219856c5e0fa95a06dc7e9c6503fab069a4deb63c2445113689 not found: ID does not exist" Sep 11 09:14:07.563662 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.563625 2562 scope.go:117] "RemoveContainer" containerID="ed060cbb79663881ef352921b24419b6adcc046943fffc33234d357d70487f8c" Sep 11 09:14:07.563896 ip-10-0-141-169 kubenswrapper[2562]: E0911 09:14:07.563880 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed060cbb79663881ef352921b24419b6adcc046943fffc33234d357d70487f8c\": container with ID starting with ed060cbb79663881ef352921b24419b6adcc046943fffc33234d357d70487f8c not found: ID does not exist" containerID="ed060cbb79663881ef352921b24419b6adcc046943fffc33234d357d70487f8c" Sep 11 09:14:07.563936 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.563898 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed060cbb79663881ef352921b24419b6adcc046943fffc33234d357d70487f8c"} err="failed to get container status \"ed060cbb79663881ef352921b24419b6adcc046943fffc33234d357d70487f8c\": rpc error: code = NotFound desc = could not find container \"ed060cbb79663881ef352921b24419b6adcc046943fffc33234d357d70487f8c\": container with ID starting with ed060cbb79663881ef352921b24419b6adcc046943fffc33234d357d70487f8c not found: ID does not exist" Sep 11 09:14:07.563936 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.563910 2562 scope.go:117] "RemoveContainer" containerID="ea04366f3643414d85a5999e939dfd40c5b95ce909017eab9edb8758257856ff" Sep 11 09:14:07.564132 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.564111 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea04366f3643414d85a5999e939dfd40c5b95ce909017eab9edb8758257856ff"} err="failed to get container status \"ea04366f3643414d85a5999e939dfd40c5b95ce909017eab9edb8758257856ff\": rpc error: code = NotFound desc = could not find container \"ea04366f3643414d85a5999e939dfd40c5b95ce909017eab9edb8758257856ff\": container with ID starting with ea04366f3643414d85a5999e939dfd40c5b95ce909017eab9edb8758257856ff not found: ID does not exist" Sep 11 09:14:07.564169 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.564134 2562 scope.go:117] "RemoveContainer" containerID="c430effe7493f234eb089938c0c68232eb3da9f11c62640e74f1e08a452f53eb" Sep 11 09:14:07.564404 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.564385 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c430effe7493f234eb089938c0c68232eb3da9f11c62640e74f1e08a452f53eb"} err="failed to get container status \"c430effe7493f234eb089938c0c68232eb3da9f11c62640e74f1e08a452f53eb\": rpc error: code = NotFound desc = could not find container \"c430effe7493f234eb089938c0c68232eb3da9f11c62640e74f1e08a452f53eb\": container with ID starting with c430effe7493f234eb089938c0c68232eb3da9f11c62640e74f1e08a452f53eb not found: ID does not exist" Sep 11 09:14:07.564404 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.564405 2562 scope.go:117] "RemoveContainer" containerID="90bf0ab9146c23fb7f5a0421a72ee1f964371566ef890e8f282907f1200fcaf3" Sep 11 09:14:07.564637 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.564617 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90bf0ab9146c23fb7f5a0421a72ee1f964371566ef890e8f282907f1200fcaf3"} err="failed to get container status \"90bf0ab9146c23fb7f5a0421a72ee1f964371566ef890e8f282907f1200fcaf3\": rpc error: code = NotFound desc = could not find container \"90bf0ab9146c23fb7f5a0421a72ee1f964371566ef890e8f282907f1200fcaf3\": container with ID starting with 90bf0ab9146c23fb7f5a0421a72ee1f964371566ef890e8f282907f1200fcaf3 not found: ID does not exist" Sep 11 09:14:07.564637 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.564637 2562 scope.go:117] "RemoveContainer" containerID="905e299cbcb1bad0fbcf9bc44b258eb85b25ece280054afe990a5318e4081ad5" Sep 11 09:14:07.564873 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.564857 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"905e299cbcb1bad0fbcf9bc44b258eb85b25ece280054afe990a5318e4081ad5"} err="failed to get container status \"905e299cbcb1bad0fbcf9bc44b258eb85b25ece280054afe990a5318e4081ad5\": rpc error: code = NotFound desc = could not find container \"905e299cbcb1bad0fbcf9bc44b258eb85b25ece280054afe990a5318e4081ad5\": container with ID starting with 905e299cbcb1bad0fbcf9bc44b258eb85b25ece280054afe990a5318e4081ad5 not found: ID does not exist" Sep 11 09:14:07.564919 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.564874 2562 scope.go:117] "RemoveContainer" containerID="81302003c8436219856c5e0fa95a06dc7e9c6503fab069a4deb63c2445113689" Sep 11 09:14:07.565093 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.565075 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81302003c8436219856c5e0fa95a06dc7e9c6503fab069a4deb63c2445113689"} err="failed to get container status \"81302003c8436219856c5e0fa95a06dc7e9c6503fab069a4deb63c2445113689\": rpc error: code = NotFound desc = could not find container \"81302003c8436219856c5e0fa95a06dc7e9c6503fab069a4deb63c2445113689\": container with ID starting with 81302003c8436219856c5e0fa95a06dc7e9c6503fab069a4deb63c2445113689 not found: ID does not exist" Sep 11 09:14:07.565093 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.565092 2562 scope.go:117] "RemoveContainer" containerID="ed060cbb79663881ef352921b24419b6adcc046943fffc33234d357d70487f8c" Sep 11 09:14:07.565320 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.565288 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed060cbb79663881ef352921b24419b6adcc046943fffc33234d357d70487f8c"} err="failed to get container status \"ed060cbb79663881ef352921b24419b6adcc046943fffc33234d357d70487f8c\": rpc error: code = NotFound desc = could not find container \"ed060cbb79663881ef352921b24419b6adcc046943fffc33234d357d70487f8c\": container with ID starting with ed060cbb79663881ef352921b24419b6adcc046943fffc33234d357d70487f8c not found: ID does not exist" Sep 11 09:14:07.571022 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:07.571003 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" path="/var/lib/kubelet/pods/e2fadcf8-72ea-44f1-9740-3a0ea775b17d/volumes" Sep 11 09:14:10.176237 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:14:10.176204 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:15:01.236993 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.236910 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:15:01.337421 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.337376 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:15:01.687562 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.687476 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:15:01.783952 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.783912 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:15:01.787376 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.787331 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf"] Sep 11 09:15:01.787588 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.787574 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="feast-apply" Sep 11 09:15:01.787640 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.787589 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="feast-apply" Sep 11 09:15:01.787640 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.787598 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="online" Sep 11 09:15:01.787640 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.787604 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="online" Sep 11 09:15:01.787640 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.787613 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="offline" Sep 11 09:15:01.787640 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.787618 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="offline" Sep 11 09:15:01.787640 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.787627 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="registry" Sep 11 09:15:01.787640 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.787636 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="registry" Sep 11 09:15:01.787640 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.787643 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="feast-init" Sep 11 09:15:01.787917 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.787648 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="feast-init" Sep 11 09:15:01.787917 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.787658 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="ui" Sep 11 09:15:01.787917 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.787663 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="ui" Sep 11 09:15:01.787917 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.787709 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="registry" Sep 11 09:15:01.787917 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.787716 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="ui" Sep 11 09:15:01.787917 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.787723 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="online" Sep 11 09:15:01.787917 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.787729 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2fadcf8-72ea-44f1-9740-3a0ea775b17d" containerName="offline" Sep 11 09:15:01.790920 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.790902 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:01.793234 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.793214 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 11 09:15:01.793367 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.793222 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 11 09:15:01.793367 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.793243 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-hgxmd\"" Sep 11 09:15:01.793589 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.793553 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 11 09:15:01.793589 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.793576 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 11 09:15:01.793761 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.793610 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 11 09:15:01.793761 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.793577 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 11 09:15:01.802942 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.802922 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf"] Sep 11 09:15:01.860905 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.860873 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:15:01.907332 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.907287 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/cc69f352-568a-4d85-bad5-b9f441d62409-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8xcxf\" (UID: \"cc69f352-568a-4d85-bad5-b9f441d62409\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:01.907332 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.907334 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/cc69f352-568a-4d85-bad5-b9f441d62409-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8xcxf\" (UID: \"cc69f352-568a-4d85-bad5-b9f441d62409\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:01.907554 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.907357 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/cc69f352-568a-4d85-bad5-b9f441d62409-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8xcxf\" (UID: \"cc69f352-568a-4d85-bad5-b9f441d62409\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:01.907554 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.907451 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/cc69f352-568a-4d85-bad5-b9f441d62409-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8xcxf\" (UID: \"cc69f352-568a-4d85-bad5-b9f441d62409\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:01.907554 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.907492 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/cc69f352-568a-4d85-bad5-b9f441d62409-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8xcxf\" (UID: \"cc69f352-568a-4d85-bad5-b9f441d62409\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:01.907672 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.907548 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9ct8\" (UniqueName: \"kubernetes.io/projected/cc69f352-568a-4d85-bad5-b9f441d62409-kube-api-access-t9ct8\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8xcxf\" (UID: \"cc69f352-568a-4d85-bad5-b9f441d62409\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:01.959602 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:01.959535 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:15:02.008669 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:02.008642 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/cc69f352-568a-4d85-bad5-b9f441d62409-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8xcxf\" (UID: \"cc69f352-568a-4d85-bad5-b9f441d62409\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:02.008776 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:02.008681 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/cc69f352-568a-4d85-bad5-b9f441d62409-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8xcxf\" (UID: \"cc69f352-568a-4d85-bad5-b9f441d62409\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:02.008776 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:02.008704 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9ct8\" (UniqueName: \"kubernetes.io/projected/cc69f352-568a-4d85-bad5-b9f441d62409-kube-api-access-t9ct8\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8xcxf\" (UID: \"cc69f352-568a-4d85-bad5-b9f441d62409\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:02.008776 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:02.008729 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/cc69f352-568a-4d85-bad5-b9f441d62409-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8xcxf\" (UID: \"cc69f352-568a-4d85-bad5-b9f441d62409\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:02.008776 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:02.008752 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/cc69f352-568a-4d85-bad5-b9f441d62409-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8xcxf\" (UID: \"cc69f352-568a-4d85-bad5-b9f441d62409\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:02.008776 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:02.008775 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/cc69f352-568a-4d85-bad5-b9f441d62409-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8xcxf\" (UID: \"cc69f352-568a-4d85-bad5-b9f441d62409\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:02.009105 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:02.009087 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/cc69f352-568a-4d85-bad5-b9f441d62409-feast-data\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8xcxf\" (UID: \"cc69f352-568a-4d85-bad5-b9f441d62409\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:02.009430 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:02.009409 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/cc69f352-568a-4d85-bad5-b9f441d62409-registry-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8xcxf\" (UID: \"cc69f352-568a-4d85-bad5-b9f441d62409\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:02.011231 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:02.011213 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/cc69f352-568a-4d85-bad5-b9f441d62409-online-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8xcxf\" (UID: \"cc69f352-568a-4d85-bad5-b9f441d62409\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:02.011416 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:02.011396 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/cc69f352-568a-4d85-bad5-b9f441d62409-offline-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8xcxf\" (UID: \"cc69f352-568a-4d85-bad5-b9f441d62409\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:02.011459 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:02.011397 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/cc69f352-568a-4d85-bad5-b9f441d62409-ui-tls\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8xcxf\" (UID: \"cc69f352-568a-4d85-bad5-b9f441d62409\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:02.016623 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:02.016602 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9ct8\" (UniqueName: \"kubernetes.io/projected/cc69f352-568a-4d85-bad5-b9f441d62409-kube-api-access-t9ct8\") pod \"feast-simple-feast-remote-setup-c5d49d8c6-8xcxf\" (UID: \"cc69f352-568a-4d85-bad5-b9f441d62409\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:02.099653 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:02.099628 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:02.374086 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:02.374064 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf"] Sep 11 09:15:02.376884 ip-10-0-141-169 kubenswrapper[2562]: W0911 09:15:02.376852 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc69f352_568a_4d85_bad5_b9f441d62409.slice/crio-500db512909a0855c348072402a51b15dfdedf73da97ec1f41b12386f264305a WatchSource:0}: Error finding container 500db512909a0855c348072402a51b15dfdedf73da97ec1f41b12386f264305a: Status 404 returned error can't find the container with id 500db512909a0855c348072402a51b15dfdedf73da97ec1f41b12386f264305a Sep 11 09:15:02.702145 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:02.701380 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" event={"ID":"cc69f352-568a-4d85-bad5-b9f441d62409","Type":"ContainerStarted","Data":"879e39af55e04409e9ed23f70d628a9aa4e584555983121c60ea71f8ed4177eb"} Sep 11 09:15:02.702145 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:02.701433 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" event={"ID":"cc69f352-568a-4d85-bad5-b9f441d62409","Type":"ContainerStarted","Data":"500db512909a0855c348072402a51b15dfdedf73da97ec1f41b12386f264305a"} Sep 11 09:15:07.721011 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:07.720978 2562 generic.go:356] "Generic (PLEG): container finished" podID="cc69f352-568a-4d85-bad5-b9f441d62409" containerID="879e39af55e04409e9ed23f70d628a9aa4e584555983121c60ea71f8ed4177eb" exitCode=0 Sep 11 09:15:07.721499 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:07.721055 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" event={"ID":"cc69f352-568a-4d85-bad5-b9f441d62409","Type":"ContainerDied","Data":"879e39af55e04409e9ed23f70d628a9aa4e584555983121c60ea71f8ed4177eb"} Sep 11 09:15:08.726802 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:08.726767 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" event={"ID":"cc69f352-568a-4d85-bad5-b9f441d62409","Type":"ContainerStarted","Data":"bf3243ede25380abade4410436560c7089594dfb8b677db16775fbcf1056b737"} Sep 11 09:15:30.807036 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:30.807003 2562 generic.go:356] "Generic (PLEG): container finished" podID="cc69f352-568a-4d85-bad5-b9f441d62409" containerID="bf3243ede25380abade4410436560c7089594dfb8b677db16775fbcf1056b737" exitCode=0 Sep 11 09:15:30.807426 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:30.807074 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" event={"ID":"cc69f352-568a-4d85-bad5-b9f441d62409","Type":"ContainerDied","Data":"bf3243ede25380abade4410436560c7089594dfb8b677db16775fbcf1056b737"} Sep 11 09:15:31.815942 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:31.815897 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" event={"ID":"cc69f352-568a-4d85-bad5-b9f441d62409","Type":"ContainerStarted","Data":"e24caa558373a012184d710e0c2073e324adae5d001d5625689dcf269ced7a40"} Sep 11 09:15:31.816455 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:31.815963 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" event={"ID":"cc69f352-568a-4d85-bad5-b9f441d62409","Type":"ContainerStarted","Data":"74d011df2a481e76ceb6ad682518ad4d20b6f413593b7e5a8ac087684df9b98e"} Sep 11 09:15:31.816455 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:31.815987 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" event={"ID":"cc69f352-568a-4d85-bad5-b9f441d62409","Type":"ContainerStarted","Data":"869a1fec8125529418ebc216db36de20a65b4de4ba34a1ce171c9f002bb5daa3"} Sep 11 09:15:31.842056 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:31.841997 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" podStartSLOduration=30.841976799 podStartE2EDuration="30.841976799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 09:15:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 09:15:31.840755503 +0000 UTC m=+1738.740653802" watchObservedRunningTime="2026-09-11 09:15:31.841976799 +0000 UTC m=+1738.741875097" Sep 11 09:15:32.100634 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:32.100555 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:32.100634 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:32.100633 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:32.100851 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:32.100655 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:32.100851 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:32.100670 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:32.100851 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:32.100684 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:32.100851 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:32.100698 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:32.102572 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:32.102532 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused" Sep 11 09:15:32.102572 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:32.102548 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.22:8816: connect: connection refused" Sep 11 09:15:32.102572 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:32.102564 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": dial tcp 10.133.0.22:6567: connect: connection refused" Sep 11 09:15:35.100806 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:35.100756 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused" Sep 11 09:15:35.101181 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:35.100814 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": dial tcp 10.133.0.22:6567: connect: connection refused" Sep 11 09:15:35.101181 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:35.100766 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.22:8816: connect: connection refused" Sep 11 09:15:38.099987 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:38.099937 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused" Sep 11 09:15:38.100463 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:38.099937 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.22:8816: connect: connection refused" Sep 11 09:15:38.100463 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:38.099950 2562 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": dial tcp 10.133.0.22:6567: connect: connection refused" Sep 11 09:15:41.100898 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:41.100864 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:41.101359 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:41.101340 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:41.101745 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:41.101720 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:41.857470 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:41.857441 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:41.877053 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:41.877027 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:42.821040 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:42.821007 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:15:43.104480 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:15:43.104394 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 11 09:16:03.141427 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:03.141386 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 11 09:16:23.141444 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:23.141399 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 11 09:16:33.581106 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:33.581073 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/ovn-acl-logging/0.log" Sep 11 09:16:33.582817 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:33.582800 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/ovn-acl-logging/0.log" Sep 11 09:16:43.104475 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:43.104430 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 11 09:16:43.108103 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:43.108071 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Sep 11 09:16:43.865031 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:43.864997 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf"] Sep 11 09:16:43.865268 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:43.865239 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="online" containerID="cri-o://869a1fec8125529418ebc216db36de20a65b4de4ba34a1ce171c9f002bb5daa3" gracePeriod=30 Sep 11 09:16:43.865390 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:43.865287 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="offline" containerID="cri-o://74d011df2a481e76ceb6ad682518ad4d20b6f413593b7e5a8ac087684df9b98e" gracePeriod=30 Sep 11 09:16:43.865390 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:43.865285 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="ui" containerID="cri-o://e24caa558373a012184d710e0c2073e324adae5d001d5625689dcf269ced7a40" gracePeriod=30 Sep 11 09:16:43.898789 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:43.898766 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:16:43.985457 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:43.985410 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:16:44.008485 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:44.008450 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn"] Sep 11 09:16:44.008755 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:44.008731 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="registry" containerID="cri-o://b01e3ba000ce4caedcdc2f0063129c719ff2484a99be228e111aae4e231f6472" gracePeriod=30 Sep 11 09:16:44.008967 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:44.008841 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="ui" containerID="cri-o://9ae1790a6d24b5495605cc1547048832e6540a3ae029fd606d03a6c1aff78cef" gracePeriod=30 Sep 11 09:16:44.008967 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:44.008841 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="offline" containerID="cri-o://ae1a29ce0cc8dad658dee7d1d1560f4c63448a54b70084efd228fdab1da2732f" gracePeriod=30 Sep 11 09:16:44.008967 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:44.008867 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="online" containerID="cri-o://58bccef913ae43fcdfe54f2f032b8271cd5d935ed8b741b8289e1b076d4c9612" gracePeriod=30 Sep 11 09:16:44.085531 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:44.085490 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:16:44.156936 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:44.156829 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:16:45.078980 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:45.078904 2562 generic.go:356] "Generic (PLEG): container finished" podID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerID="9ae1790a6d24b5495605cc1547048832e6540a3ae029fd606d03a6c1aff78cef" exitCode=0 Sep 11 09:16:45.078980 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:45.078927 2562 generic.go:356] "Generic (PLEG): container finished" podID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerID="58bccef913ae43fcdfe54f2f032b8271cd5d935ed8b741b8289e1b076d4c9612" exitCode=0 Sep 11 09:16:45.079161 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:45.078971 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" event={"ID":"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5","Type":"ContainerDied","Data":"9ae1790a6d24b5495605cc1547048832e6540a3ae029fd606d03a6c1aff78cef"} Sep 11 09:16:45.079161 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:45.079017 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" event={"ID":"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5","Type":"ContainerDied","Data":"58bccef913ae43fcdfe54f2f032b8271cd5d935ed8b741b8289e1b076d4c9612"} Sep 11 09:16:45.081577 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:45.081560 2562 generic.go:356] "Generic (PLEG): container finished" podID="cc69f352-568a-4d85-bad5-b9f441d62409" containerID="e24caa558373a012184d710e0c2073e324adae5d001d5625689dcf269ced7a40" exitCode=0 Sep 11 09:16:45.081632 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:45.081586 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" event={"ID":"cc69f352-568a-4d85-bad5-b9f441d62409","Type":"ContainerDied","Data":"e24caa558373a012184d710e0c2073e324adae5d001d5625689dcf269ced7a40"} Sep 11 09:16:49.165474 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:49.165438 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:16:49.488767 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:49.488726 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.21:8443: connect: connection refused" Sep 11 09:16:49.488954 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:49.488915 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="online" probeResult="failure" output="Get \"https://10.133.0.21:6567/health\": dial tcp 10.133.0.21:6567: connect: connection refused" Sep 11 09:16:51.101615 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:51.101580 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused" Sep 11 09:16:53.101711 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:53.101667 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Sep 11 09:16:59.488603 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:59.488555 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.21:8443: connect: connection refused" Sep 11 09:16:59.489062 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:16:59.488624 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="online" probeResult="failure" output="Get \"https://10.133.0.21:6567/health\": dial tcp 10.133.0.21:6567: connect: connection refused" Sep 11 09:17:01.102219 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:01.102179 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused" Sep 11 09:17:03.101431 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:03.101391 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": context deadline exceeded" Sep 11 09:17:03.101785 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:03.101520 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:17:09.488594 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:09.488547 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.21:8443: connect: connection refused" Sep 11 09:17:09.489090 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:09.488601 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="online" probeResult="failure" output="Get \"https://10.133.0.21:6567/health\": dial tcp 10.133.0.21:6567: connect: connection refused" Sep 11 09:17:09.489090 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:09.488705 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:17:09.489090 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:09.488777 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:17:11.101930 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:11.101885 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.22:8443: connect: connection refused" Sep 11 09:17:11.102408 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:11.102005 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:17:13.101572 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:13.101524 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="online" probeResult="failure" output="Get \"https://10.133.0.22:6567/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Sep 11 09:17:14.182912 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.182839 2562 generic.go:356] "Generic (PLEG): container finished" podID="cc69f352-568a-4d85-bad5-b9f441d62409" containerID="74d011df2a481e76ceb6ad682518ad4d20b6f413593b7e5a8ac087684df9b98e" exitCode=137 Sep 11 09:17:14.182912 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.182865 2562 generic.go:356] "Generic (PLEG): container finished" podID="cc69f352-568a-4d85-bad5-b9f441d62409" containerID="869a1fec8125529418ebc216db36de20a65b4de4ba34a1ce171c9f002bb5daa3" exitCode=137 Sep 11 09:17:14.183290 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.182910 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" event={"ID":"cc69f352-568a-4d85-bad5-b9f441d62409","Type":"ContainerDied","Data":"74d011df2a481e76ceb6ad682518ad4d20b6f413593b7e5a8ac087684df9b98e"} Sep 11 09:17:14.183290 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.182951 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" event={"ID":"cc69f352-568a-4d85-bad5-b9f441d62409","Type":"ContainerDied","Data":"869a1fec8125529418ebc216db36de20a65b4de4ba34a1ce171c9f002bb5daa3"} Sep 11 09:17:14.185863 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.185842 2562 generic.go:356] "Generic (PLEG): container finished" podID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerID="ae1a29ce0cc8dad658dee7d1d1560f4c63448a54b70084efd228fdab1da2732f" exitCode=137 Sep 11 09:17:14.185863 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.185861 2562 generic.go:356] "Generic (PLEG): container finished" podID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerID="b01e3ba000ce4caedcdc2f0063129c719ff2484a99be228e111aae4e231f6472" exitCode=137 Sep 11 09:17:14.186046 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.185878 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" event={"ID":"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5","Type":"ContainerDied","Data":"ae1a29ce0cc8dad658dee7d1d1560f4c63448a54b70084efd228fdab1da2732f"} Sep 11 09:17:14.186046 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.185904 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" event={"ID":"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5","Type":"ContainerDied","Data":"b01e3ba000ce4caedcdc2f0063129c719ff2484a99be228e111aae4e231f6472"} Sep 11 09:17:14.504448 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.504426 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:17:14.607442 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.607408 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/cc69f352-568a-4d85-bad5-b9f441d62409-online-tls\") pod \"cc69f352-568a-4d85-bad5-b9f441d62409\" (UID: \"cc69f352-568a-4d85-bad5-b9f441d62409\") " Sep 11 09:17:14.607583 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.607486 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/cc69f352-568a-4d85-bad5-b9f441d62409-ui-tls\") pod \"cc69f352-568a-4d85-bad5-b9f441d62409\" (UID: \"cc69f352-568a-4d85-bad5-b9f441d62409\") " Sep 11 09:17:14.607583 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.607520 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/cc69f352-568a-4d85-bad5-b9f441d62409-feast-data\") pod \"cc69f352-568a-4d85-bad5-b9f441d62409\" (UID: \"cc69f352-568a-4d85-bad5-b9f441d62409\") " Sep 11 09:17:14.607583 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.607574 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/cc69f352-568a-4d85-bad5-b9f441d62409-registry-tls\") pod \"cc69f352-568a-4d85-bad5-b9f441d62409\" (UID: \"cc69f352-568a-4d85-bad5-b9f441d62409\") " Sep 11 09:17:14.607722 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.607609 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9ct8\" (UniqueName: \"kubernetes.io/projected/cc69f352-568a-4d85-bad5-b9f441d62409-kube-api-access-t9ct8\") pod \"cc69f352-568a-4d85-bad5-b9f441d62409\" (UID: \"cc69f352-568a-4d85-bad5-b9f441d62409\") " Sep 11 09:17:14.607722 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.607636 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/cc69f352-568a-4d85-bad5-b9f441d62409-offline-tls\") pod \"cc69f352-568a-4d85-bad5-b9f441d62409\" (UID: \"cc69f352-568a-4d85-bad5-b9f441d62409\") " Sep 11 09:17:14.608007 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.607979 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc69f352-568a-4d85-bad5-b9f441d62409-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "cc69f352-568a-4d85-bad5-b9f441d62409" (UID: "cc69f352-568a-4d85-bad5-b9f441d62409"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 09:17:14.608154 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.608127 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc69f352-568a-4d85-bad5-b9f441d62409-feast-data" (OuterVolumeSpecName: "feast-data") pod "cc69f352-568a-4d85-bad5-b9f441d62409" (UID: "cc69f352-568a-4d85-bad5-b9f441d62409"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 09:17:14.609959 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.609931 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc69f352-568a-4d85-bad5-b9f441d62409-online-tls" (OuterVolumeSpecName: "online-tls") pod "cc69f352-568a-4d85-bad5-b9f441d62409" (UID: "cc69f352-568a-4d85-bad5-b9f441d62409"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 09:17:14.610073 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.610018 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc69f352-568a-4d85-bad5-b9f441d62409-kube-api-access-t9ct8" (OuterVolumeSpecName: "kube-api-access-t9ct8") pod "cc69f352-568a-4d85-bad5-b9f441d62409" (UID: "cc69f352-568a-4d85-bad5-b9f441d62409"). InnerVolumeSpecName "kube-api-access-t9ct8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 09:17:14.610123 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.610088 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc69f352-568a-4d85-bad5-b9f441d62409-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "cc69f352-568a-4d85-bad5-b9f441d62409" (UID: "cc69f352-568a-4d85-bad5-b9f441d62409"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 09:17:14.610123 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.610100 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc69f352-568a-4d85-bad5-b9f441d62409-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "cc69f352-568a-4d85-bad5-b9f441d62409" (UID: "cc69f352-568a-4d85-bad5-b9f441d62409"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 09:17:14.648104 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.648084 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:17:14.708452 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.708431 2562 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/cc69f352-568a-4d85-bad5-b9f441d62409-feast-data\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:17:14.708452 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.708452 2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/cc69f352-568a-4d85-bad5-b9f441d62409-registry-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:17:14.708594 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.708468 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9ct8\" (UniqueName: \"kubernetes.io/projected/cc69f352-568a-4d85-bad5-b9f441d62409-kube-api-access-t9ct8\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:17:14.708594 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.708478 2562 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/cc69f352-568a-4d85-bad5-b9f441d62409-offline-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:17:14.708594 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.708486 2562 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/cc69f352-568a-4d85-bad5-b9f441d62409-online-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:17:14.708594 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.708493 2562 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/cc69f352-568a-4d85-bad5-b9f441d62409-ui-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:17:14.809708 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.809648 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-feast-data\") pod \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\" (UID: \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\") " Sep 11 09:17:14.809708 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.809686 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-offline-tls\") pod \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\" (UID: \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\") " Sep 11 09:17:14.809862 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.809731 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-ui-tls\") pod \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\" (UID: \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\") " Sep 11 09:17:14.809862 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.809762 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhtdc\" (UniqueName: \"kubernetes.io/projected/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-kube-api-access-hhtdc\") pod \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\" (UID: \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\") " Sep 11 09:17:14.809862 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.809798 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-registry-tls\") pod \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\" (UID: \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\") " Sep 11 09:17:14.809862 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.809837 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-online-tls\") pod \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\" (UID: \"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5\") " Sep 11 09:17:14.810218 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.810188 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-feast-data" (OuterVolumeSpecName: "feast-data") pod "ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" (UID: "ee47b745-ab7c-4fc3-b2a3-de338a7d7af5"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 09:17:14.811864 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.811842 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" (UID: "ee47b745-ab7c-4fc3-b2a3-de338a7d7af5"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 09:17:14.811955 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.811858 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-kube-api-access-hhtdc" (OuterVolumeSpecName: "kube-api-access-hhtdc") pod "ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" (UID: "ee47b745-ab7c-4fc3-b2a3-de338a7d7af5"). InnerVolumeSpecName "kube-api-access-hhtdc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 09:17:14.811955 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.811899 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" (UID: "ee47b745-ab7c-4fc3-b2a3-de338a7d7af5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 09:17:14.812205 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.812175 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" (UID: "ee47b745-ab7c-4fc3-b2a3-de338a7d7af5"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 09:17:14.812205 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.812186 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-online-tls" (OuterVolumeSpecName: "online-tls") pod "ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" (UID: "ee47b745-ab7c-4fc3-b2a3-de338a7d7af5"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 09:17:14.910840 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.910817 2562 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-online-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:17:14.910840 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.910838 2562 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-feast-data\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:17:14.910953 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.910846 2562 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-offline-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:17:14.910953 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.910855 2562 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-ui-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:17:14.910953 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.910864 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhtdc\" (UniqueName: \"kubernetes.io/projected/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-kube-api-access-hhtdc\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:17:14.910953 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:14.910872 2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5-registry-tls\") on node \"ip-10-0-141-169.ec2.internal\" DevicePath \"\"" Sep 11 09:17:15.192472 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:15.192387 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" event={"ID":"ee47b745-ab7c-4fc3-b2a3-de338a7d7af5","Type":"ContainerDied","Data":"3c4964e339abe5e93092f9598a2e4921320fde423bfc19e60ac8fd037863e364"} Sep 11 09:17:15.192472 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:15.192439 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn" Sep 11 09:17:15.192472 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:15.192457 2562 scope.go:117] "RemoveContainer" containerID="9ae1790a6d24b5495605cc1547048832e6540a3ae029fd606d03a6c1aff78cef" Sep 11 09:17:15.195139 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:15.195118 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" Sep 11 09:17:15.195243 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:15.195141 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf" event={"ID":"cc69f352-568a-4d85-bad5-b9f441d62409","Type":"ContainerDied","Data":"500db512909a0855c348072402a51b15dfdedf73da97ec1f41b12386f264305a"} Sep 11 09:17:15.201664 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:15.201643 2562 scope.go:117] "RemoveContainer" containerID="ae1a29ce0cc8dad658dee7d1d1560f4c63448a54b70084efd228fdab1da2732f" Sep 11 09:17:15.212615 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:15.212586 2562 scope.go:117] "RemoveContainer" containerID="58bccef913ae43fcdfe54f2f032b8271cd5d935ed8b741b8289e1b076d4c9612" Sep 11 09:17:15.219521 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:15.219503 2562 scope.go:117] "RemoveContainer" containerID="b01e3ba000ce4caedcdc2f0063129c719ff2484a99be228e111aae4e231f6472" Sep 11 09:17:15.220441 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:15.220394 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf"] Sep 11 09:17:15.224087 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:15.224066 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-c5d49d8c6-8xcxf"] Sep 11 09:17:15.226855 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:15.226841 2562 scope.go:117] "RemoveContainer" containerID="6530531a4de80a93e18dd2d8b679bad4bacfb5b21660d077691a2b340e4c5616" Sep 11 09:17:15.233683 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:15.233661 2562 scope.go:117] "RemoveContainer" containerID="c14dc0506f2ba1ec098cf74d0ade97ed78b3929b31762f6432fd818db98705fe" Sep 11 09:17:15.234839 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:15.234821 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn"] Sep 11 09:17:15.238432 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:15.238409 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-57576b8fcf-zx5bn"] Sep 11 09:17:15.240533 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:15.240515 2562 scope.go:117] "RemoveContainer" containerID="e24caa558373a012184d710e0c2073e324adae5d001d5625689dcf269ced7a40" Sep 11 09:17:15.246740 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:15.246724 2562 scope.go:117] "RemoveContainer" containerID="74d011df2a481e76ceb6ad682518ad4d20b6f413593b7e5a8ac087684df9b98e" Sep 11 09:17:15.253257 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:15.253241 2562 scope.go:117] "RemoveContainer" containerID="869a1fec8125529418ebc216db36de20a65b4de4ba34a1ce171c9f002bb5daa3" Sep 11 09:17:15.259814 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:15.259800 2562 scope.go:117] "RemoveContainer" containerID="bf3243ede25380abade4410436560c7089594dfb8b677db16775fbcf1056b737" Sep 11 09:17:15.266060 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:15.266045 2562 scope.go:117] "RemoveContainer" containerID="879e39af55e04409e9ed23f70d628a9aa4e584555983121c60ea71f8ed4177eb" Sep 11 09:17:15.570543 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:15.570514 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" path="/var/lib/kubelet/pods/cc69f352-568a-4d85-bad5-b9f441d62409/volumes" Sep 11 09:17:15.571019 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:15.571006 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" path="/var/lib/kubelet/pods/ee47b745-ab7c-4fc3-b2a3-de338a7d7af5/volumes" Sep 11 09:17:26.772254 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:26.772218 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:17:26.887782 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:26.887750 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:17:32.406987 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:32.406950 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:17:32.543490 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:32.543446 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:17:33.121567 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:33.121530 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:17:33.218191 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:33.218165 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:17:50.877447 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:50.877413 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:17:51.014698 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.014669 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qk7h7/must-gather-fxzvt"] Sep 11 09:17:51.014976 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.014960 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="feast-apply" Sep 11 09:17:51.015057 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.014979 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="feast-apply" Sep 11 09:17:51.015057 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.014992 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="feast-apply" Sep 11 09:17:51.015057 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015000 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="feast-apply" Sep 11 09:17:51.015057 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015010 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="ui" Sep 11 09:17:51.015057 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015019 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="ui" Sep 11 09:17:51.015057 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015032 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="feast-init" Sep 11 09:17:51.015057 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015040 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="feast-init" Sep 11 09:17:51.015057 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015051 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="registry" Sep 11 09:17:51.015057 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015059 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="registry" Sep 11 09:17:51.015535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015083 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="offline" Sep 11 09:17:51.015535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015093 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="offline" Sep 11 09:17:51.015535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015103 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="ui" Sep 11 09:17:51.015535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015111 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="ui" Sep 11 09:17:51.015535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015122 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="online" Sep 11 09:17:51.015535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015130 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="online" Sep 11 09:17:51.015535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015147 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="feast-init" Sep 11 09:17:51.015535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015155 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="feast-init" Sep 11 09:17:51.015535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015164 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="online" Sep 11 09:17:51.015535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015171 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="online" Sep 11 09:17:51.015535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015181 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="offline" Sep 11 09:17:51.015535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015188 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="offline" Sep 11 09:17:51.015535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015247 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="ui" Sep 11 09:17:51.015535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015257 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="registry" Sep 11 09:17:51.015535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015265 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="online" Sep 11 09:17:51.015535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015274 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="offline" Sep 11 09:17:51.015535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015283 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="ui" Sep 11 09:17:51.015535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015292 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee47b745-ab7c-4fc3-b2a3-de338a7d7af5" containerName="online" Sep 11 09:17:51.015535 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.015316 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc69f352-568a-4d85-bad5-b9f441d62409" containerName="offline" Sep 11 09:17:51.018471 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.018451 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qk7h7/must-gather-fxzvt" Sep 11 09:17:51.020628 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.020606 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qk7h7\"/\"default-dockercfg-2zmbc\"" Sep 11 09:17:51.020839 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.020819 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qk7h7\"/\"openshift-service-ca.crt\"" Sep 11 09:17:51.021126 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.021112 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qk7h7\"/\"kube-root-ca.crt\"" Sep 11 09:17:51.025411 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.025387 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qk7h7/must-gather-fxzvt"] Sep 11 09:17:51.029067 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.029038 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:17:51.045859 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.045835 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rntd7\" (UniqueName: \"kubernetes.io/projected/35691cce-7cbe-4d03-8979-9162be78ec75-kube-api-access-rntd7\") pod \"must-gather-fxzvt\" (UID: \"35691cce-7cbe-4d03-8979-9162be78ec75\") " pod="openshift-must-gather-qk7h7/must-gather-fxzvt" Sep 11 09:17:51.045945 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.045869 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/35691cce-7cbe-4d03-8979-9162be78ec75-must-gather-output\") pod \"must-gather-fxzvt\" (UID: \"35691cce-7cbe-4d03-8979-9162be78ec75\") " pod="openshift-must-gather-qk7h7/must-gather-fxzvt" Sep 11 09:17:51.091715 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.091693 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:17:51.146367 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.146298 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rntd7\" (UniqueName: \"kubernetes.io/projected/35691cce-7cbe-4d03-8979-9162be78ec75-kube-api-access-rntd7\") pod \"must-gather-fxzvt\" (UID: \"35691cce-7cbe-4d03-8979-9162be78ec75\") " pod="openshift-must-gather-qk7h7/must-gather-fxzvt" Sep 11 09:17:51.146367 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.146345 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/35691cce-7cbe-4d03-8979-9162be78ec75-must-gather-output\") pod \"must-gather-fxzvt\" (UID: \"35691cce-7cbe-4d03-8979-9162be78ec75\") " pod="openshift-must-gather-qk7h7/must-gather-fxzvt" Sep 11 09:17:51.146587 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.146573 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/35691cce-7cbe-4d03-8979-9162be78ec75-must-gather-output\") pod \"must-gather-fxzvt\" (UID: \"35691cce-7cbe-4d03-8979-9162be78ec75\") " pod="openshift-must-gather-qk7h7/must-gather-fxzvt" Sep 11 09:17:51.155621 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.155598 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rntd7\" (UniqueName: \"kubernetes.io/projected/35691cce-7cbe-4d03-8979-9162be78ec75-kube-api-access-rntd7\") pod \"must-gather-fxzvt\" (UID: \"35691cce-7cbe-4d03-8979-9162be78ec75\") " pod="openshift-must-gather-qk7h7/must-gather-fxzvt" Sep 11 09:17:51.327707 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.327685 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qk7h7/must-gather-fxzvt" Sep 11 09:17:51.594002 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.593877 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qk7h7/must-gather-fxzvt"] Sep 11 09:17:51.596373 ip-10-0-141-169 kubenswrapper[2562]: W0911 09:17:51.596348 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35691cce_7cbe_4d03_8979_9162be78ec75.slice/crio-97e6c408a3daff902649f5390b135a2905a396412acfeb04ee897950755806fd WatchSource:0}: Error finding container 97e6c408a3daff902649f5390b135a2905a396412acfeb04ee897950755806fd: Status 404 returned error can't find the container with id 97e6c408a3daff902649f5390b135a2905a396412acfeb04ee897950755806fd Sep 11 09:17:51.598226 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:51.598205 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 11 09:17:52.312770 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:52.312671 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qk7h7/must-gather-fxzvt" event={"ID":"35691cce-7cbe-4d03-8979-9162be78ec75","Type":"ContainerStarted","Data":"97e6c408a3daff902649f5390b135a2905a396412acfeb04ee897950755806fd"} Sep 11 09:17:53.319537 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:53.319494 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qk7h7/must-gather-fxzvt" event={"ID":"35691cce-7cbe-4d03-8979-9162be78ec75","Type":"ContainerStarted","Data":"c2b696debf224736096874c761112ae5f233d7bdd6404d49ae2f427ee73dbb9b"} Sep 11 09:17:53.319927 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:53.319549 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qk7h7/must-gather-fxzvt" event={"ID":"35691cce-7cbe-4d03-8979-9162be78ec75","Type":"ContainerStarted","Data":"d9b4bce150f0b75bddc14616b26af969c33783c70f0f6a42b5ca9781f3dfae45"} Sep 11 09:17:53.336586 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:53.336531 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qk7h7/must-gather-fxzvt" podStartSLOduration=1.6153432680000002 podStartE2EDuration="2.336510967s" totalImagesPullingTime="721.167699ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 09:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 09:17:53.334683046 +0000 UTC m=+1880.234581344" watchObservedRunningTime="2026-09-11 09:17:53.336510967 +0000 UTC m=+1880.236409265" Sep 11 09:17:53.678393 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:53.678288 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tldcl_e53c87e2-2721-4540-87c6-214593d333a4/global-pull-secret-syncer/0.log" Sep 11 09:17:53.772128 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:53.772090 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" Sep 11 09:17:53.899554 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:53.899514 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-169.ec2.internal_458a6d26c1086ff6914f03deab3d15f2/haproxy/0.log" Sep 11 09:17:56.443640 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:56.443530 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:17:56.527925 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:56.527888 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4smrv_20f65aa9-c978-41d0-9ca1-9e1ddd4ef5bf/konnectivity-agent/0.log" limit=1024 Sep 11 09:17:58.127802 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:58.127745 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w6llw_3bf195fe-d614-4941-b271-d8821fc90554/node-exporter/0.log" Sep 11 09:17:58.154586 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:58.154556 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w6llw_3bf195fe-d614-4941-b271-d8821fc90554/kube-rbac-proxy/0.log" Sep 11 09:17:58.182018 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:17:58.181983 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-w6llw_3bf195fe-d614-4941-b271-d8821fc90554/init-textfile/0.log" Sep 11 09:18:00.908751 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:00.908713 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln"] Sep 11 09:18:00.914593 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:00.914566 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln" Sep 11 09:18:00.927432 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:00.927403 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln"] Sep 11 09:18:00.939277 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:00.939245 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7f86f7c1-6df0-4913-b459-90c871898868-lib-modules\") pod \"perf-node-gather-daemonset-x5wln\" (UID: \"7f86f7c1-6df0-4913-b459-90c871898868\") " pod="openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln" Sep 11 09:18:00.939417 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:00.939326 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhwh5\" (UniqueName: \"kubernetes.io/projected/7f86f7c1-6df0-4913-b459-90c871898868-kube-api-access-jhwh5\") pod \"perf-node-gather-daemonset-x5wln\" (UID: \"7f86f7c1-6df0-4913-b459-90c871898868\") " pod="openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln" Sep 11 09:18:00.939417 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:00.939399 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7f86f7c1-6df0-4913-b459-90c871898868-podres\") pod \"perf-node-gather-daemonset-x5wln\" (UID: \"7f86f7c1-6df0-4913-b459-90c871898868\") " pod="openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln" Sep 11 09:18:00.939529 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:00.939429 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f86f7c1-6df0-4913-b459-90c871898868-sys\") pod \"perf-node-gather-daemonset-x5wln\" (UID: \"7f86f7c1-6df0-4913-b459-90c871898868\") " pod="openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln" Sep 11 09:18:00.939529 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:00.939454 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7f86f7c1-6df0-4913-b459-90c871898868-proc\") pod \"perf-node-gather-daemonset-x5wln\" (UID: \"7f86f7c1-6df0-4913-b459-90c871898868\") " pod="openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln" Sep 11 09:18:01.040474 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:01.040427 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7f86f7c1-6df0-4913-b459-90c871898868-podres\") pod \"perf-node-gather-daemonset-x5wln\" (UID: \"7f86f7c1-6df0-4913-b459-90c871898868\") " pod="openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln" Sep 11 09:18:01.040654 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:01.040487 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f86f7c1-6df0-4913-b459-90c871898868-sys\") pod \"perf-node-gather-daemonset-x5wln\" (UID: \"7f86f7c1-6df0-4913-b459-90c871898868\") " pod="openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln" Sep 11 09:18:01.040654 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:01.040513 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7f86f7c1-6df0-4913-b459-90c871898868-proc\") pod \"perf-node-gather-daemonset-x5wln\" (UID: \"7f86f7c1-6df0-4913-b459-90c871898868\") " pod="openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln" Sep 11 09:18:01.040654 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:01.040537 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7f86f7c1-6df0-4913-b459-90c871898868-lib-modules\") pod \"perf-node-gather-daemonset-x5wln\" (UID: \"7f86f7c1-6df0-4913-b459-90c871898868\") " pod="openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln" Sep 11 09:18:01.040654 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:01.040575 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7f86f7c1-6df0-4913-b459-90c871898868-podres\") pod \"perf-node-gather-daemonset-x5wln\" (UID: \"7f86f7c1-6df0-4913-b459-90c871898868\") " pod="openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln" Sep 11 09:18:01.040654 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:01.040592 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhwh5\" (UniqueName: \"kubernetes.io/projected/7f86f7c1-6df0-4913-b459-90c871898868-kube-api-access-jhwh5\") pod \"perf-node-gather-daemonset-x5wln\" (UID: \"7f86f7c1-6df0-4913-b459-90c871898868\") " pod="openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln" Sep 11 09:18:01.040654 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:01.040602 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f86f7c1-6df0-4913-b459-90c871898868-sys\") pod \"perf-node-gather-daemonset-x5wln\" (UID: \"7f86f7c1-6df0-4913-b459-90c871898868\") " pod="openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln" Sep 11 09:18:01.040654 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:01.040650 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7f86f7c1-6df0-4913-b459-90c871898868-proc\") pod \"perf-node-gather-daemonset-x5wln\" (UID: \"7f86f7c1-6df0-4913-b459-90c871898868\") " pod="openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln" Sep 11 09:18:01.040974 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:01.040689 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7f86f7c1-6df0-4913-b459-90c871898868-lib-modules\") pod \"perf-node-gather-daemonset-x5wln\" (UID: \"7f86f7c1-6df0-4913-b459-90c871898868\") " pod="openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln" Sep 11 09:18:01.057824 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:01.057796 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhwh5\" (UniqueName: \"kubernetes.io/projected/7f86f7c1-6df0-4913-b459-90c871898868-kube-api-access-jhwh5\") pod \"perf-node-gather-daemonset-x5wln\" (UID: \"7f86f7c1-6df0-4913-b459-90c871898868\") " pod="openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln" Sep 11 09:18:01.225809 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:01.225725 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln" Sep 11 09:18:01.509324 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:01.509235 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln"] Sep 11 09:18:01.513348 ip-10-0-141-169 kubenswrapper[2562]: W0911 09:18:01.513317 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7f86f7c1_6df0_4913_b459_90c871898868.slice/crio-116f14da30331411909866c371b83ccf7b1a63ecdf80dd90c2d3d20927e57420 WatchSource:0}: Error finding container 116f14da30331411909866c371b83ccf7b1a63ecdf80dd90c2d3d20927e57420: Status 404 returned error can't find the container with id 116f14da30331411909866c371b83ccf7b1a63ecdf80dd90c2d3d20927e57420 Sep 11 09:18:02.359012 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:02.358973 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln" event={"ID":"7f86f7c1-6df0-4913-b459-90c871898868","Type":"ContainerStarted","Data":"8b969ae88b3f9a823c48cee2e4f786cc17971c9a2cd6f87091647ce2e6795473"} Sep 11 09:18:02.359422 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:02.359030 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln" event={"ID":"7f86f7c1-6df0-4913-b459-90c871898868","Type":"ContainerStarted","Data":"116f14da30331411909866c371b83ccf7b1a63ecdf80dd90c2d3d20927e57420"} Sep 11 09:18:02.359422 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:02.359113 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln" Sep 11 09:18:02.721400 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:02.721321 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7c6vh_0cf62b5d-a365-4686-a79d-9301c5f5297b/dns/0.log" Sep 11 09:18:02.773839 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:02.773813 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7c6vh_0cf62b5d-a365-4686-a79d-9301c5f5297b/kube-rbac-proxy/0.log" Sep 11 09:18:03.166876 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:03.166842 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cbhgc_4cd3f396-137e-4748-83b5-582ddc810965/dns-node-resolver/0.log" Sep 11 09:18:03.803492 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:03.803455 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9gfk6_395401b2-9748-42b5-b600-4fa1f7369b1e/node-ca/0.log" Sep 11 09:18:05.122798 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:05.122765 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-js7h2_cb966255-7c76-47bb-aed7-bd2671e8013a/serve-healthcheck-canary/0.log" Sep 11 09:18:05.575683 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:05.575657 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jwpjd_79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7/kube-rbac-proxy/0.log" Sep 11 09:18:05.597922 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:05.597897 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jwpjd_79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7/exporter/0.log" Sep 11 09:18:05.646094 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:05.646071 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jwpjd_79c653d6-bad2-4ddf-ad33-97c2b4cd2ef7/extractor/0.log" Sep 11 09:18:08.371600 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:08.371575 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln" Sep 11 09:18:08.389147 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:08.389105 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qk7h7/perf-node-gather-daemonset-x5wln" podStartSLOduration=8.389089314 podStartE2EDuration="8.389089314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 09:18:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 09:18:02.380046357 +0000 UTC m=+1889.279944659" watchObservedRunningTime="2026-09-11 09:18:08.389089314 +0000 UTC m=+1895.288987611" Sep 11 09:18:12.130663 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:12.130635 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wpkr7_c62c56b1-a68f-4a3f-a41b-23a2a08e148d/kube-multus-additional-cni-plugins/0.log" Sep 11 09:18:12.159836 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:12.159808 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wpkr7_c62c56b1-a68f-4a3f-a41b-23a2a08e148d/egress-router-binary-copy/0.log" Sep 11 09:18:12.188472 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:12.188434 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wpkr7_c62c56b1-a68f-4a3f-a41b-23a2a08e148d/cni-plugins/0.log" Sep 11 09:18:12.218733 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:12.218666 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wpkr7_c62c56b1-a68f-4a3f-a41b-23a2a08e148d/bond-cni-plugin/0.log" Sep 11 09:18:12.244408 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:12.244381 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wpkr7_c62c56b1-a68f-4a3f-a41b-23a2a08e148d/routeoverride-cni/0.log" Sep 11 09:18:12.268368 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:12.268334 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wpkr7_c62c56b1-a68f-4a3f-a41b-23a2a08e148d/whereabouts-cni-bincopy/0.log" Sep 11 09:18:12.294075 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:12.294051 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wpkr7_c62c56b1-a68f-4a3f-a41b-23a2a08e148d/whereabouts-cni/0.log" Sep 11 09:18:12.335975 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:12.335952 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-x9gk7_731bdf3b-9131-4284-a232-93b9b73f3a89/kube-multus/0.log" Sep 11 09:18:12.453932 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:12.453901 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-97r66_faa7ad4f-7140-43cd-a195-d659bd3bee15/network-metrics-daemon/0.log" Sep 11 09:18:12.504591 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:12.504567 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-97r66_faa7ad4f-7140-43cd-a195-d659bd3bee15/kube-rbac-proxy/0.log" Sep 11 09:18:14.075216 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:14.075171 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/ovn-controller/0.log" Sep 11 09:18:14.120219 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:14.120189 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/ovn-acl-logging/1.log" Sep 11 09:18:14.120384 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:14.120227 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/ovn-acl-logging/0.log" Sep 11 09:18:14.160036 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:14.160015 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/kube-rbac-proxy-node/0.log" Sep 11 09:18:14.192120 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:14.192093 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/kube-rbac-proxy-ovn-metrics/0.log" Sep 11 09:18:14.236694 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:14.236670 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/northd/0.log" Sep 11 09:18:14.270033 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:14.270016 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/nbdb/0.log" Sep 11 09:18:14.299408 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:14.299390 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/sbdb/0.log" Sep 11 09:18:14.356620 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:14.356553 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n6jlz_1e30076a-a074-40b2-8c43-2b2757da92d2/ovnkube-controller/0.log" Sep 11 09:18:15.421410 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:15.421362 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-q7bqv_f24c0bea-46aa-432e-a51f-f25451143d07/network-check-target-container/0.log" Sep 11 09:18:16.382556 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:16.382525 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-tgx4q_caf9e2dc-28c5-4aa3-bfd0-b09bbfd49b0b/iptables-alerter/0.log" Sep 11 09:18:17.184072 ip-10-0-141-169 kubenswrapper[2562]: I0911 09:18:17.184027 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-nxhhd_8901cf6c-2530-49cc-971f-431e24a0b88d/tuned/0.log"