Sep 17 07:27:58.332178 ip-10-0-141-54 systemd[1]: Starting Kubernetes Kubelet... Sep 17 07:27:58.713868 ip-10-0-141-54 kubenswrapper[2553]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 07:27:58.713868 ip-10-0-141-54 kubenswrapper[2553]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 17 07:27:58.713868 ip-10-0-141-54 kubenswrapper[2553]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 07:27:58.713868 ip-10-0-141-54 kubenswrapper[2553]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 17 07:27:58.713868 ip-10-0-141-54 kubenswrapper[2553]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 07:27:58.716032 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.715944 2553 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 17 07:27:58.720048 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720027 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 07:27:58.720048 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720044 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 07:27:58.720161 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720073 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 07:27:58.720161 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720078 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 07:27:58.720161 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720081 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 07:27:58.720161 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720085 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 07:27:58.720161 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720088 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 07:27:58.720161 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720091 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 07:27:58.720161 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720094 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 07:27:58.720161 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720097 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 07:27:58.720161 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720100 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 07:27:58.720161 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720102 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 07:27:58.720161 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720105 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 07:27:58.720161 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720107 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 07:27:58.720161 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720110 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 07:27:58.720161 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720112 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 07:27:58.720161 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720115 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 07:27:58.720161 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720118 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 07:27:58.720161 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720120 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 07:27:58.720161 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720134 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 07:27:58.720161 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720137 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 07:27:58.720161 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720140 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 07:27:58.720635 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720142 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 07:27:58.720635 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720145 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 07:27:58.720635 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720148 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 07:27:58.720635 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720150 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 07:27:58.720635 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720153 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 07:27:58.720635 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720156 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 07:27:58.720635 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720158 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 07:27:58.720635 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720161 2553 feature_gate.go:328] unrecognized feature gate: Example Sep 17 07:27:58.720635 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720163 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 07:27:58.720635 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720166 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 07:27:58.720635 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720169 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 07:27:58.720635 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720172 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 07:27:58.720635 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720175 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 07:27:58.720635 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720177 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 07:27:58.720635 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720180 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 07:27:58.720635 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720182 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 07:27:58.720635 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720185 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 07:27:58.720635 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720187 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 07:27:58.720635 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720190 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 07:27:58.720635 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720192 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 07:27:58.721223 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720195 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 07:27:58.721223 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720197 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 07:27:58.721223 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720200 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 07:27:58.721223 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720202 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 07:27:58.721223 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720204 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 07:27:58.721223 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720207 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 07:27:58.721223 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720210 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 07:27:58.721223 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720212 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 07:27:58.721223 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720215 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 07:27:58.721223 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720217 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 07:27:58.721223 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720220 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 07:27:58.721223 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720222 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 07:27:58.721223 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720226 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 07:27:58.721223 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720229 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 07:27:58.721223 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720232 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 07:27:58.721223 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720236 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 07:27:58.721223 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720239 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 07:27:58.721223 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720242 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 07:27:58.721223 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720245 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 07:27:58.721683 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720247 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 07:27:58.721683 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720250 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 07:27:58.721683 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720252 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 07:27:58.721683 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720255 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 07:27:58.721683 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720257 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 07:27:58.721683 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720261 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 07:27:58.721683 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720266 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 07:27:58.721683 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720269 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 07:27:58.721683 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720272 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 07:27:58.721683 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720275 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 07:27:58.721683 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720277 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 07:27:58.721683 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720281 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 07:27:58.721683 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720286 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 07:27:58.721683 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720289 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 07:27:58.721683 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720292 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 07:27:58.721683 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720295 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 07:27:58.721683 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720298 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 07:27:58.721683 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720301 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 07:27:58.721683 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720303 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 07:27:58.721683 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720306 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 07:27:58.722186 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720309 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 07:27:58.722186 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720311 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 07:27:58.722186 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720314 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 07:27:58.722186 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720317 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 07:27:58.722186 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720319 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 07:27:58.722186 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720322 2553 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 07:27:58.722186 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720326 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 07:27:58.722186 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720328 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 07:27:58.722186 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720331 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 07:27:58.722186 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720334 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 07:27:58.722186 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720336 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 07:27:58.722186 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720339 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 07:27:58.722186 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720342 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 07:27:58.722186 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720344 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 07:27:58.722186 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720347 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 07:27:58.722186 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720349 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 07:27:58.722186 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720352 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 07:27:58.722186 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.720354 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 07:27:58.722186 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721602 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 07:27:58.722186 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721609 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 07:27:58.722658 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721614 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 07:27:58.722658 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721618 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 07:27:58.722658 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721621 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 07:27:58.722658 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721624 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 07:27:58.722658 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721627 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 07:27:58.722658 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721630 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 07:27:58.722658 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721633 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 07:27:58.722658 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721636 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 07:27:58.722658 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721638 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 07:27:58.722658 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721641 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 07:27:58.722658 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721643 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 07:27:58.722658 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721646 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 07:27:58.722658 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721648 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 07:27:58.722658 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721651 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 07:27:58.722658 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721653 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 07:27:58.722658 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721656 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 07:27:58.722658 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721659 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 07:27:58.722658 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721662 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 07:27:58.722658 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721665 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 07:27:58.723193 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721668 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 07:27:58.723193 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721671 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 07:27:58.723193 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721673 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 07:27:58.723193 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721676 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 07:27:58.723193 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721679 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 07:27:58.723193 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721682 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 07:27:58.723193 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721684 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 07:27:58.723193 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721687 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 07:27:58.723193 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721690 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 07:27:58.723193 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721692 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 07:27:58.723193 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721695 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 07:27:58.723193 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721697 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 07:27:58.723193 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721700 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 07:27:58.723193 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721703 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 07:27:58.723193 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721706 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 07:27:58.723193 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721708 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 07:27:58.723193 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721711 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 07:27:58.723193 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721713 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 07:27:58.723193 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721716 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 07:27:58.723193 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721718 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 07:27:58.723681 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721721 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 07:27:58.723681 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721723 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 07:27:58.723681 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721726 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 07:27:58.723681 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721728 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 07:27:58.723681 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721731 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 07:27:58.723681 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721734 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 07:27:58.723681 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721736 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 07:27:58.723681 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721739 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 07:27:58.723681 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721741 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 07:27:58.723681 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721744 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 07:27:58.723681 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721747 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 07:27:58.723681 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721750 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 07:27:58.723681 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721753 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 07:27:58.723681 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721755 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 07:27:58.723681 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721758 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 07:27:58.723681 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721761 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 07:27:58.723681 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721763 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 07:27:58.723681 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721767 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 07:27:58.723681 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721771 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 07:27:58.724155 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721774 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 07:27:58.724155 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721776 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 07:27:58.724155 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721779 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 07:27:58.724155 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721781 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 07:27:58.724155 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721784 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 07:27:58.724155 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721787 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 07:27:58.724155 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721789 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 07:27:58.724155 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721793 2553 feature_gate.go:328] unrecognized feature gate: Example Sep 17 07:27:58.724155 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721795 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 07:27:58.724155 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721798 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 07:27:58.724155 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721801 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 07:27:58.724155 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721803 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 07:27:58.724155 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721806 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 07:27:58.724155 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721808 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 07:27:58.724155 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721811 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 07:27:58.724155 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721813 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 07:27:58.724155 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721815 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 07:27:58.724155 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721818 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 07:27:58.724155 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721820 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 07:27:58.724155 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721822 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 07:27:58.724636 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721825 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 07:27:58.724636 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721830 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 07:27:58.724636 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721833 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 07:27:58.724636 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721836 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 07:27:58.724636 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721839 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 07:27:58.724636 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721841 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 07:27:58.724636 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721844 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 07:27:58.724636 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721847 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 07:27:58.724636 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721850 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 07:27:58.724636 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721852 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 07:27:58.724636 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721855 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 07:27:58.724636 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721857 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 07:27:58.724636 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721860 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 07:27:58.724636 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721862 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 07:27:58.724636 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721864 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 07:27:58.724636 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721867 2553 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 07:27:58.724636 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721869 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 07:27:58.724636 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721872 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 07:27:58.724636 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.721892 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 07:27:58.725091 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722280 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 07:27:58.725091 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722285 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 07:27:58.725091 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722287 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 07:27:58.725091 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722290 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 07:27:58.725091 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722293 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 07:27:58.725091 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722296 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 07:27:58.725091 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722299 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 07:27:58.725091 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722301 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 07:27:58.725091 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722306 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 07:27:58.725091 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722309 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 07:27:58.725091 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722312 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 07:27:58.725091 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722315 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 07:27:58.725091 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722318 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 07:27:58.725091 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722322 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 07:27:58.725091 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722324 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 07:27:58.725091 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722327 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 07:27:58.725091 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722330 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 07:27:58.725091 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722333 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 07:27:58.725091 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722335 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 07:27:58.725550 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722338 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 07:27:58.725550 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722340 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 07:27:58.725550 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722343 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 07:27:58.725550 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722345 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 07:27:58.725550 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722347 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 07:27:58.725550 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722350 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 07:27:58.725550 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722353 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 07:27:58.725550 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722355 2553 feature_gate.go:328] unrecognized feature gate: Example Sep 17 07:27:58.725550 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722358 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 07:27:58.725550 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722361 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 07:27:58.725550 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722364 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 07:27:58.725550 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722368 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 07:27:58.725550 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722371 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 07:27:58.725550 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722375 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 07:27:58.725550 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722378 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 07:27:58.725550 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722381 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 07:27:58.725550 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722384 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 07:27:58.725550 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722386 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 07:27:58.725550 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722389 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 07:27:58.726033 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722391 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 07:27:58.726033 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722394 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 07:27:58.726033 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722396 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 07:27:58.726033 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722398 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 07:27:58.726033 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722401 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 07:27:58.726033 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722403 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 07:27:58.726033 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722406 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 07:27:58.726033 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722408 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 07:27:58.726033 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722411 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 07:27:58.726033 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722413 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 07:27:58.726033 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722415 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 07:27:58.726033 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722418 2553 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 07:27:58.726033 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722421 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 07:27:58.726033 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722423 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 07:27:58.726033 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722426 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 07:27:58.726033 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722428 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 07:27:58.726033 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722430 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 07:27:58.726033 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722433 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 07:27:58.726033 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722435 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 07:27:58.726033 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722438 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 07:27:58.726512 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722440 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 07:27:58.726512 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722442 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 07:27:58.726512 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722445 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 07:27:58.726512 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722449 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 07:27:58.726512 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722452 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 07:27:58.726512 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722454 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 07:27:58.726512 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722457 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 07:27:58.726512 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722460 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 07:27:58.726512 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722462 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 07:27:58.726512 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722465 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 07:27:58.726512 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722468 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 07:27:58.726512 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722472 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 07:27:58.726512 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722474 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 07:27:58.726512 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722477 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 07:27:58.726512 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722479 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 07:27:58.726512 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722482 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 07:27:58.726512 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722484 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 07:27:58.726512 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722486 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 07:27:58.726512 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722489 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 07:27:58.726512 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722492 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 07:27:58.727014 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722494 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 07:27:58.727014 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722496 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 07:27:58.727014 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722499 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 07:27:58.727014 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722501 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 07:27:58.727014 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722504 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 07:27:58.727014 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722506 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 07:27:58.727014 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722509 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 07:27:58.727014 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722511 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 07:27:58.727014 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722514 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 07:27:58.727014 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722516 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 07:27:58.727014 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722519 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 07:27:58.727014 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722521 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 07:27:58.727014 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722524 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 07:27:58.727014 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722526 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 07:27:58.727014 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722528 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 07:27:58.727014 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722532 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 07:27:58.727014 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722535 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 07:27:58.727014 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722537 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 07:27:58.727014 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722540 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 07:27:58.727014 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722543 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 07:27:58.727485 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.722545 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 07:27:58.727485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722618 2553 flags.go:64] FLAG: --address="0.0.0.0" Sep 17 07:27:58.727485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722651 2553 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 17 07:27:58.727485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722657 2553 flags.go:64] FLAG: --anonymous-auth="true" Sep 17 07:27:58.727485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722662 2553 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 17 07:27:58.727485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722666 2553 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 17 07:27:58.727485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722670 2553 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 17 07:27:58.727485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722674 2553 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 17 07:27:58.727485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722678 2553 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 17 07:27:58.727485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722681 2553 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 17 07:27:58.727485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722684 2553 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 17 07:27:58.727485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722688 2553 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 17 07:27:58.727485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722691 2553 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 17 07:27:58.727485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722694 2553 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 17 07:27:58.727485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722697 2553 flags.go:64] FLAG: --cgroup-root="" Sep 17 07:27:58.727485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722700 2553 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 17 07:27:58.727485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722703 2553 flags.go:64] FLAG: --client-ca-file="" Sep 17 07:27:58.727485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722706 2553 flags.go:64] FLAG: --cloud-provider="external" Sep 17 07:27:58.727485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722709 2553 flags.go:64] FLAG: --cluster-dns="[]" Sep 17 07:27:58.727485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722713 2553 flags.go:64] FLAG: --cluster-domain="" Sep 17 07:27:58.727485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722716 2553 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 17 07:27:58.727485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722719 2553 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 17 07:27:58.727485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722722 2553 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 17 07:27:58.727485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722725 2553 flags.go:64] FLAG: --container-log-max-files="5" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722729 2553 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722732 2553 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722736 2553 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722740 2553 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722743 2553 flags.go:64] FLAG: --contention-profiling="false" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722746 2553 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722749 2553 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722752 2553 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722755 2553 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722758 2553 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722761 2553 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722764 2553 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722767 2553 flags.go:64] FLAG: --enable-load-reader="false" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722770 2553 flags.go:64] FLAG: --enable-server="true" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722772 2553 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722776 2553 flags.go:64] FLAG: --event-burst="100" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722779 2553 flags.go:64] FLAG: --event-qps="50" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722782 2553 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722785 2553 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722787 2553 flags.go:64] FLAG: --eviction-hard="" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722791 2553 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722794 2553 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722805 2553 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722808 2553 flags.go:64] FLAG: --eviction-soft="" Sep 17 07:27:58.728098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722811 2553 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 17 07:27:58.728696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722814 2553 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 17 07:27:58.728696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722818 2553 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 17 07:27:58.728696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722821 2553 flags.go:64] FLAG: --experimental-mounter-path="" Sep 17 07:27:58.728696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722824 2553 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 17 07:27:58.728696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722826 2553 flags.go:64] FLAG: --fail-swap-on="true" Sep 17 07:27:58.728696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722829 2553 flags.go:64] FLAG: --feature-gates="" Sep 17 07:27:58.728696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722833 2553 flags.go:64] FLAG: --file-check-frequency="20s" Sep 17 07:27:58.728696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722835 2553 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 17 07:27:58.728696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722838 2553 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 17 07:27:58.728696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722841 2553 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 17 07:27:58.728696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722846 2553 flags.go:64] FLAG: --healthz-port="10248" Sep 17 07:27:58.728696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722849 2553 flags.go:64] FLAG: --help="false" Sep 17 07:27:58.728696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722851 2553 flags.go:64] FLAG: --hostname-override="ip-10-0-141-54.ec2.internal" Sep 17 07:27:58.728696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722856 2553 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 17 07:27:58.728696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722859 2553 flags.go:64] FLAG: --http-check-frequency="20s" Sep 17 07:27:58.728696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722862 2553 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 17 07:27:58.728696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722865 2553 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 17 07:27:58.728696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722868 2553 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 17 07:27:58.728696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722871 2553 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 17 07:27:58.728696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722887 2553 flags.go:64] FLAG: --image-service-endpoint="" Sep 17 07:27:58.728696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722890 2553 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 17 07:27:58.728696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722893 2553 flags.go:64] FLAG: --kube-api-burst="100" Sep 17 07:27:58.728696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722896 2553 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722900 2553 flags.go:64] FLAG: --kube-api-qps="50" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722903 2553 flags.go:64] FLAG: --kube-reserved="" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722905 2553 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722908 2553 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722911 2553 flags.go:64] FLAG: --kubelet-cgroups="" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722914 2553 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722917 2553 flags.go:64] FLAG: --lock-file="" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722920 2553 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722923 2553 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722926 2553 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722931 2553 flags.go:64] FLAG: --log-json-split-stream="false" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722935 2553 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722938 2553 flags.go:64] FLAG: --log-text-split-stream="false" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722941 2553 flags.go:64] FLAG: --logging-format="text" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722943 2553 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722947 2553 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722949 2553 flags.go:64] FLAG: --manifest-url="" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722952 2553 flags.go:64] FLAG: --manifest-url-header="" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722956 2553 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722960 2553 flags.go:64] FLAG: --max-open-files="1000000" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722964 2553 flags.go:64] FLAG: --max-pods="110" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722967 2553 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722971 2553 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722974 2553 flags.go:64] FLAG: --memory-manager-policy="None" Sep 17 07:27:58.729288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722977 2553 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722980 2553 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722983 2553 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722986 2553 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722993 2553 flags.go:64] FLAG: --node-status-max-images="50" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722996 2553 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.722999 2553 flags.go:64] FLAG: --oom-score-adj="-999" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723002 2553 flags.go:64] FLAG: --pod-cidr="" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723005 2553 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723010 2553 flags.go:64] FLAG: --pod-manifest-path="" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723013 2553 flags.go:64] FLAG: --pod-max-pids="-1" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723016 2553 flags.go:64] FLAG: --pods-per-core="0" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723019 2553 flags.go:64] FLAG: --port="10250" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723021 2553 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723024 2553 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0240934760616a9f4" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723027 2553 flags.go:64] FLAG: --qos-reserved="" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723030 2553 flags.go:64] FLAG: --read-only-port="10255" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723033 2553 flags.go:64] FLAG: --register-node="true" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723036 2553 flags.go:64] FLAG: --register-with-taints="" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723040 2553 flags.go:64] FLAG: --registry-burst="10" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723043 2553 flags.go:64] FLAG: --registry-qps="5" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723046 2553 flags.go:64] FLAG: --reserved-cpus="" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723049 2553 flags.go:64] FLAG: --reserved-memory="" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723053 2553 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723056 2553 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 17 07:27:58.729905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723059 2553 flags.go:64] FLAG: --rotate-certificates="false" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723062 2553 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723066 2553 flags.go:64] FLAG: --runonce="false" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723069 2553 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723071 2553 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723074 2553 flags.go:64] FLAG: --seccomp-default="false" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723078 2553 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723081 2553 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723084 2553 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723087 2553 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723090 2553 flags.go:64] FLAG: --storage-driver-password="root" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723093 2553 flags.go:64] FLAG: --storage-driver-secure="false" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723096 2553 flags.go:64] FLAG: --storage-driver-table="stats" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723099 2553 flags.go:64] FLAG: --storage-driver-user="root" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723102 2553 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723105 2553 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723108 2553 flags.go:64] FLAG: --system-cgroups="" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723111 2553 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723115 2553 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723118 2553 flags.go:64] FLAG: --tls-cert-file="" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723121 2553 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723125 2553 flags.go:64] FLAG: --tls-min-version="" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723128 2553 flags.go:64] FLAG: --tls-private-key-file="" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723131 2553 flags.go:64] FLAG: --topology-manager-policy="none" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723134 2553 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 17 07:27:58.730538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723136 2553 flags.go:64] FLAG: --topology-manager-scope="container" Sep 17 07:27:58.731197 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723139 2553 flags.go:64] FLAG: --v="2" Sep 17 07:27:58.731197 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723143 2553 flags.go:64] FLAG: --version="false" Sep 17 07:27:58.731197 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723147 2553 flags.go:64] FLAG: --vmodule="" Sep 17 07:27:58.731197 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723151 2553 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 17 07:27:58.731197 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.723154 2553 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 17 07:27:58.731197 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723268 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 07:27:58.731197 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723272 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 07:27:58.731197 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723275 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 07:27:58.731197 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723280 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 07:27:58.731197 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723284 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 07:27:58.731197 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723287 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 07:27:58.731197 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723289 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 07:27:58.731197 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723293 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 07:27:58.731197 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723297 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 07:27:58.731197 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723300 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 07:27:58.731197 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723303 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 07:27:58.731197 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723306 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 07:27:58.731197 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723309 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 07:27:58.731197 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723311 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 07:27:58.731197 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723314 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 07:27:58.731701 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723317 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 07:27:58.731701 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723319 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 07:27:58.731701 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723322 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 07:27:58.731701 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723325 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 07:27:58.731701 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723327 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 07:27:58.731701 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723330 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 07:27:58.731701 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723332 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 07:27:58.731701 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723335 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 07:27:58.731701 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723338 2553 feature_gate.go:328] unrecognized feature gate: Example Sep 17 07:27:58.731701 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723341 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 07:27:58.731701 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723343 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 07:27:58.731701 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723346 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 07:27:58.731701 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723349 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 07:27:58.731701 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723351 2553 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 07:27:58.731701 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723354 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 07:27:58.731701 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723357 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 07:27:58.731701 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723360 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 07:27:58.731701 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723363 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 07:27:58.731701 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723365 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 07:27:58.731701 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723368 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 07:27:58.732202 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723373 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 07:27:58.732202 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723376 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 07:27:58.732202 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723378 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 07:27:58.732202 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723381 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 07:27:58.732202 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723384 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 07:27:58.732202 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723387 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 07:27:58.732202 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723390 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 07:27:58.732202 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723392 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 07:27:58.732202 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723395 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 07:27:58.732202 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723397 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 07:27:58.732202 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723400 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 07:27:58.732202 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723403 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 07:27:58.732202 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723405 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 07:27:58.732202 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723409 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 07:27:58.732202 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723411 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 07:27:58.732202 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723414 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 07:27:58.732202 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723416 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 07:27:58.732202 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723419 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 07:27:58.732202 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723422 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 07:27:58.732202 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723424 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 07:27:58.732687 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723427 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 07:27:58.732687 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723430 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 07:27:58.732687 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723432 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 07:27:58.732687 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723435 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 07:27:58.732687 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723437 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 07:27:58.732687 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723440 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 07:27:58.732687 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723442 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 07:27:58.732687 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723445 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 07:27:58.732687 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723448 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 07:27:58.732687 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723450 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 07:27:58.732687 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723453 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 07:27:58.732687 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723456 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 07:27:58.732687 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723459 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 07:27:58.732687 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723462 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 07:27:58.732687 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723464 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 07:27:58.732687 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723467 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 07:27:58.732687 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723472 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 07:27:58.732687 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723476 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 07:27:58.732687 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723479 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 07:27:58.732687 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723482 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 07:27:58.733182 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723485 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 07:27:58.733182 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723487 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 07:27:58.733182 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723490 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 07:27:58.733182 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723493 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 07:27:58.733182 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723495 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 07:27:58.733182 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723498 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 07:27:58.733182 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723501 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 07:27:58.733182 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723504 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 07:27:58.733182 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723508 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 07:27:58.733182 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723511 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 07:27:58.733182 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723514 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 07:27:58.733182 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723517 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 07:27:58.733182 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723520 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 07:27:58.733182 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723523 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 07:27:58.733182 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723526 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 07:27:58.733182 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723528 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 07:27:58.733182 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723530 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 07:27:58.733182 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723533 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 07:27:58.733182 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723536 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 07:27:58.733182 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723538 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 07:27:58.733664 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723541 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 07:27:58.733664 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723543 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 07:27:58.733664 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723546 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 07:27:58.733664 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.723549 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 07:27:58.733664 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.724042 2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 07:27:58.733664 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.733159 2553 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 17 07:27:58.733664 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.733175 2553 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 17 07:27:58.734052 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.734041 2553 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 17 07:27:58.734094 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.734051 2553 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 17 07:27:58.734125 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734108 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 07:27:58.734125 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734113 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 07:27:58.734125 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734117 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 07:27:58.734125 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734119 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 07:27:58.734125 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734123 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 07:27:58.734125 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734126 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 07:27:58.734125 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734128 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 07:27:58.734299 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734132 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 07:27:58.734299 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734135 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 07:27:58.734299 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734138 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 07:27:58.734299 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734141 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 07:27:58.734299 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734145 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 07:27:58.734299 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734148 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 07:27:58.734299 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734151 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 07:27:58.734299 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734154 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 07:27:58.734299 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734157 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 07:27:58.734299 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734161 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 07:27:58.734299 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734165 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 07:27:58.734299 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734168 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 07:27:58.734299 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734171 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 07:27:58.734299 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734174 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 07:27:58.734299 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734176 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 07:27:58.734299 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734179 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 07:27:58.734299 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734181 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 07:27:58.734299 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734184 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 07:27:58.734299 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734187 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 07:27:58.734793 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734190 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 07:27:58.734793 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734192 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 07:27:58.734793 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734195 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 07:27:58.734793 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734197 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 07:27:58.734793 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734200 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 07:27:58.734793 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734202 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 07:27:58.734793 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734204 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 07:27:58.734793 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734207 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 07:27:58.734793 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734209 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 07:27:58.734793 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734212 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 07:27:58.734793 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734215 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 07:27:58.734793 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734217 2553 feature_gate.go:328] unrecognized feature gate: Example Sep 17 07:27:58.734793 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734219 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 07:27:58.734793 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734222 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 07:27:58.734793 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734226 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 07:27:58.734793 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734228 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 07:27:58.734793 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734231 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 07:27:58.734793 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734233 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 07:27:58.734793 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734238 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 07:27:58.735332 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734241 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 07:27:58.735332 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734244 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 07:27:58.735332 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734247 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 07:27:58.735332 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734250 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 07:27:58.735332 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734252 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 07:27:58.735332 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734255 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 07:27:58.735332 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734258 2553 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 07:27:58.735332 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734260 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 07:27:58.735332 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734263 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 07:27:58.735332 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734266 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 07:27:58.735332 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734269 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 07:27:58.735332 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734271 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 07:27:58.735332 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734274 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 07:27:58.735332 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734277 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 07:27:58.735332 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734280 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 07:27:58.735332 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734282 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 07:27:58.735332 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734285 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 07:27:58.735332 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734287 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 07:27:58.735332 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734290 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 07:27:58.735332 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734292 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 07:27:58.735818 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734295 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 07:27:58.735818 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734298 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 07:27:58.735818 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734300 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 07:27:58.735818 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734303 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 07:27:58.735818 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734306 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 07:27:58.735818 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734308 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 07:27:58.735818 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734311 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 07:27:58.735818 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734314 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 07:27:58.735818 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734317 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 07:27:58.735818 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734320 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 07:27:58.735818 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734322 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 07:27:58.735818 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734325 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 07:27:58.735818 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734328 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 07:27:58.735818 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734330 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 07:27:58.735818 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734333 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 07:27:58.735818 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734335 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 07:27:58.735818 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734338 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 07:27:58.735818 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734340 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 07:27:58.735818 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734343 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 07:27:58.736287 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734345 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 07:27:58.736287 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734348 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 07:27:58.736287 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734351 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 07:27:58.736287 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734353 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 07:27:58.736287 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734356 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 07:27:58.736287 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734358 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 07:27:58.736287 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734362 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 07:27:58.736287 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734365 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 07:27:58.736287 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734367 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 07:27:58.736287 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734370 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 07:27:58.736287 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734372 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 07:27:58.736287 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734375 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 07:27:58.736287 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734378 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 07:27:58.736287 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734380 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 07:27:58.736287 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734382 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 07:27:58.736648 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.734387 2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 07:27:58.736648 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734495 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 07:27:58.736648 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734503 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 07:27:58.736648 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734507 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 07:27:58.736648 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734511 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 07:27:58.736648 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734516 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 07:27:58.736648 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734520 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 07:27:58.736648 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734523 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 07:27:58.736648 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734525 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 07:27:58.736648 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734528 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 07:27:58.736648 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734530 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 07:27:58.736648 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734533 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 07:27:58.736648 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734535 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 07:27:58.736648 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734538 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 07:27:58.737001 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734541 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 07:27:58.737001 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734544 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 07:27:58.737001 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734547 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 07:27:58.737001 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734550 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 07:27:58.737001 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734553 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 07:27:58.737001 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734555 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 07:27:58.737001 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734558 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 07:27:58.737001 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734560 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 07:27:58.737001 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734563 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 07:27:58.737001 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734566 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 07:27:58.737001 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734569 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 07:27:58.737001 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734573 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 07:27:58.737001 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734576 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 07:27:58.737001 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734579 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 07:27:58.737001 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734582 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 07:27:58.737001 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734585 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 07:27:58.737001 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734588 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 07:27:58.737001 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734590 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 07:27:58.737001 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734593 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 07:27:58.737457 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734596 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 07:27:58.737457 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734599 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 07:27:58.737457 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734602 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 07:27:58.737457 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734604 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 07:27:58.737457 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734607 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 07:27:58.737457 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734611 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 07:27:58.737457 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734614 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 07:27:58.737457 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734616 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 07:27:58.737457 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734619 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 07:27:58.737457 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734622 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 07:27:58.737457 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734624 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 07:27:58.737457 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734627 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 07:27:58.737457 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734630 2553 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 07:27:58.737457 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734632 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 07:27:58.737457 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734635 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 07:27:58.737457 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734637 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 07:27:58.737457 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734640 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 07:27:58.737457 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734642 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 07:27:58.737457 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734645 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 07:27:58.737457 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734647 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 07:27:58.737981 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734650 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 07:27:58.737981 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734653 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 07:27:58.737981 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734655 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 07:27:58.737981 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734658 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 07:27:58.737981 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734661 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 07:27:58.737981 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734663 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 07:27:58.737981 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734666 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 07:27:58.737981 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734668 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 07:27:58.737981 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734670 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 07:27:58.737981 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734673 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 07:27:58.737981 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734675 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 07:27:58.737981 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734678 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 07:27:58.737981 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734680 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 07:27:58.737981 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734683 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 07:27:58.737981 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734685 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 07:27:58.737981 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734688 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 07:27:58.737981 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734691 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 07:27:58.737981 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734693 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 07:27:58.737981 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734696 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 07:27:58.737981 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734698 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 07:27:58.738459 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734701 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 07:27:58.738459 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734703 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 07:27:58.738459 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734706 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 07:27:58.738459 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734708 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 07:27:58.738459 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734711 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 07:27:58.738459 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734713 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 07:27:58.738459 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734717 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 07:27:58.738459 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734720 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 07:27:58.738459 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734722 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 07:27:58.738459 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734725 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 07:27:58.738459 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734727 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 07:27:58.738459 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734730 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 07:27:58.738459 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734734 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 07:27:58.738459 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734737 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 07:27:58.738459 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734740 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 07:27:58.738459 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734742 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 07:27:58.738459 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734745 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 07:27:58.738459 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734747 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 07:27:58.738459 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734750 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 07:27:58.738459 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734753 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 07:27:58.739026 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734755 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 07:27:58.739026 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734757 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 07:27:58.739026 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734760 2553 feature_gate.go:328] unrecognized feature gate: Example Sep 17 07:27:58.739026 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734763 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 07:27:58.739026 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734765 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 07:27:58.739026 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734768 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 07:27:58.739026 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:27:58.734770 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 07:27:58.739026 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.734775 2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 07:27:58.739026 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.734870 2553 server.go:962] "Client rotation is on, will bootstrap in background" Sep 17 07:27:58.739026 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.736742 2553 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 17 07:27:58.739026 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.738189 2553 server.go:1019] "Starting client certificate rotation" Sep 17 07:27:58.739026 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.738288 2553 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 17 07:27:58.739026 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.738341 2553 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 17 07:27:58.740432 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.740419 2553 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 17 07:27:58.741520 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.741503 2553 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 17 07:27:58.756945 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.756928 2553 log.go:25] "Validated CRI v1 runtime API" Sep 17 07:27:58.761329 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.761313 2553 log.go:25] "Validated CRI v1 image API" Sep 17 07:27:58.762154 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.762138 2553 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 17 07:27:58.765034 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.765014 2553 fs.go:135] Filesystem UUIDs: map[03bfde04-f886-4b43-936d-a822c2cff945:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 909f8aad-5b03-44cd-965e-f4a9c98255b3:/dev/nvme0n1p4] Sep 17 07:27:58.765098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.765033 2553 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 17 07:27:58.769767 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.769666 2553 manager.go:216] Machine: {Timestamp:2026-09-17 07:27:58.768456013 +0000 UTC m=+0.326640812 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102103 MemoryCapacity:33164472320 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23b0b1fdf40def3df57446b57e1892 SystemUUID:ec23b0b1-fdf4-0def-3df5-7446b57e1892 BootID:8a0c3edc-3912-4ec5-87d2-d33d884e1987 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c6:86:21:3f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c6:86:21:3f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7a:1e:73:1c:aa:94 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164472320 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 17 07:27:58.769767 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.769766 2553 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 17 07:27:58.769872 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.769838 2553 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 17 07:27:58.772682 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.772662 2553 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 17 07:27:58.772822 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.772685 2553 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-54.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 17 07:27:58.773556 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.773546 2553 topology_manager.go:138] "Creating topology manager with none policy" Sep 17 07:27:58.773588 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.773559 2553 container_manager_linux.go:309] "Creating device plugin manager" Sep 17 07:27:58.773588 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.773570 2553 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 17 07:27:58.774425 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.774414 2553 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 17 07:27:58.775292 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.775282 2553 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 17 07:27:58.779293 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.779281 2553 state_mem.go:36] "Initialized new in-memory state store" Sep 17 07:27:58.779405 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.779396 2553 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 17 07:27:58.779434 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.779430 2553 kubelet.go:486] "Attempting to sync node with API server" Sep 17 07:27:58.779467 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.779445 2553 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 17 07:27:58.779467 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.779458 2553 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 17 07:27:58.779467 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.779466 2553 kubelet.go:398] "Adding apiserver pod source" Sep 17 07:27:58.779552 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.779475 2553 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 17 07:27:58.779825 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.779812 2553 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 17 07:27:58.779866 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.779833 2553 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 17 07:27:58.781692 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.781676 2553 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 17 07:27:58.783972 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.783959 2553 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 17 07:27:58.784024 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.783989 2553 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 17 07:27:58.785523 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.785512 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 17 07:27:58.785599 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.785528 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 17 07:27:58.785599 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.785535 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 17 07:27:58.785599 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.785540 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 17 07:27:58.785599 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.785549 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 17 07:27:58.785599 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.785556 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 17 07:27:58.785599 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.785579 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 17 07:27:58.785599 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.785589 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 17 07:27:58.785599 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.785596 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 17 07:27:58.785599 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.785602 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 17 07:27:58.785815 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.785614 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 17 07:27:58.785815 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.785620 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 17 07:27:58.786270 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.786260 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 17 07:27:58.786306 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.786271 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 17 07:27:58.789746 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.789723 2553 server.go:1268] "Started kubelet" Sep 17 07:27:58.789850 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.789804 2553 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 17 07:27:58.789921 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.789866 2553 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 17 07:27:58.789921 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.789809 2553 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 17 07:27:58.790643 ip-10-0-141-54 systemd[1]: Started Kubernetes Kubelet. Sep 17 07:27:58.790940 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.790926 2553 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 17 07:27:58.796440 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:58.796410 2553 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 17 07:27:58.796825 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.796810 2553 server.go:310] "Adding debug handlers to kubelet server" Sep 17 07:27:58.798423 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.798405 2553 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-54.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 17 07:27:58.798549 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:58.798533 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-54.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 17 07:27:58.798649 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.798634 2553 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 17 07:27:58.798778 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.798759 2553 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 17 07:27:58.798864 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:58.798779 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 17 07:27:58.800277 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.800250 2553 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 17 07:27:58.800691 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:58.800671 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-54.ec2.internal\" not found" Sep 17 07:27:58.801259 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.801243 2553 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 17 07:27:58.801259 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.801259 2553 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 17 07:27:58.801362 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.801243 2553 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 17 07:27:58.801362 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.801332 2553 reconstruct.go:97] "Volume reconstruction finished" Sep 17 07:27:58.801362 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.801340 2553 reconciler.go:29] "Reconciler: start to sync state" Sep 17 07:27:58.802451 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.802434 2553 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 17 07:27:58.802451 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.802449 2553 factory.go:55] Registering systemd factory Sep 17 07:27:58.802451 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.802455 2553 factory.go:223] Registration of the systemd container factory successfully Sep 17 07:27:58.802646 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:58.802531 2553 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-54.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 17 07:27:58.802695 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:58.801804 2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-54.ec2.internal.18d60b13259d4288 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-54.ec2.internal,UID:ip-10-0-141-54.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-54.ec2.internal,},FirstTimestamp:2026-09-17 07:27:58.789698184 +0000 UTC m=+0.347882983,LastTimestamp:2026-09-17 07:27:58.789698184 +0000 UTC m=+0.347882983,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-54.ec2.internal,}" Sep 17 07:27:58.802739 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.802692 2553 factory.go:153] Registering CRI-O factory Sep 17 07:27:58.802739 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.802706 2553 factory.go:223] Registration of the crio container factory successfully Sep 17 07:27:58.802739 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.802721 2553 factory.go:103] Registering Raw factory Sep 17 07:27:58.802739 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.802734 2553 manager.go:1196] Started watching for new ooms in manager Sep 17 07:27:58.802959 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:58.802821 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 17 07:27:58.803468 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.803457 2553 manager.go:349] Starting recovery of all containers Sep 17 07:27:58.813393 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.813287 2553 manager.go:354] Recovery completed Sep 17 07:27:58.814541 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:58.814490 2553 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Sep 17 07:27:58.817466 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.817454 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:27:58.819131 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.819114 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-54.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:27:58.819215 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.819148 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-54.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:27:58.819215 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.819161 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-54.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:27:58.819692 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.819678 2553 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 17 07:27:58.819692 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.819689 2553 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 17 07:27:58.819810 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.819703 2553 state_mem.go:36] "Initialized new in-memory state store" Sep 17 07:27:58.819810 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.819757 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:27:58.820437 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.820423 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-54.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:27:58.820494 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.820453 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-54.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:27:58.820494 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.820467 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-54.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:27:58.820870 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.820860 2553 policy_none.go:49] "None policy: Start" Sep 17 07:27:58.820940 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.820914 2553 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 17 07:27:58.820940 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.820928 2553 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 17 07:27:58.821515 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:58.821441 2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-54.ec2.internal.18d60b13275e5c0e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-54.ec2.internal,UID:ip-10-0-141-54.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-54.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-54.ec2.internal,},FirstTimestamp:2026-09-17 07:27:58.819130382 +0000 UTC m=+0.377315188,LastTimestamp:2026-09-17 07:27:58.819130382 +0000 UTC m=+0.377315188,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-54.ec2.internal,}" Sep 17 07:27:58.821956 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.821944 2553 policy_none.go:47] "Start" Sep 17 07:27:58.833522 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:58.833436 2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-54.ec2.internal.18d60b13275ebb17 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-54.ec2.internal,UID:ip-10-0-141-54.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-54.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-54.ec2.internal,},FirstTimestamp:2026-09-17 07:27:58.819154711 +0000 UTC m=+0.377339509,LastTimestamp:2026-09-17 07:27:58.819154711 +0000 UTC m=+0.377339509,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-54.ec2.internal,}" Sep 17 07:27:58.834254 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.834237 2553 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sj9rb" Sep 17 07:27:58.852419 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.852402 2553 manager.go:337] "Starting Device Plugin manager" Sep 17 07:27:58.861366 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:58.852441 2553 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 17 07:27:58.861366 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.852454 2553 server.go:87] "Starting device plugin registration server" Sep 17 07:27:58.861366 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.852712 2553 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 17 07:27:58.861366 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.852721 2553 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 17 07:27:58.861366 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.852838 2553 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 17 07:27:58.861366 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.852917 2553 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 17 07:27:58.861366 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.852926 2553 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 17 07:27:58.861366 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:58.853391 2553 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 17 07:27:58.861366 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:58.853439 2553 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-54.ec2.internal\" not found" Sep 17 07:27:58.861366 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:58.857327 2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-54.ec2.internal.18d60b13275ee18b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-54.ec2.internal,UID:ip-10-0-141-54.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-54.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-54.ec2.internal,},FirstTimestamp:2026-09-17 07:27:58.819164555 +0000 UTC m=+0.377349353,LastTimestamp:2026-09-17 07:27:58.819164555 +0000 UTC m=+0.377349353,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-54.ec2.internal,}" Sep 17 07:27:58.861366 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.859003 2553 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sj9rb" Sep 17 07:27:58.926949 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.926909 2553 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 17 07:27:58.928080 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.928054 2553 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 17 07:27:58.928167 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.928113 2553 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 17 07:27:58.928167 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.928137 2553 kubelet.go:2448] "Starting kubelet main sync loop" Sep 17 07:27:58.928261 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:58.928230 2553 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 17 07:27:58.935046 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.935022 2553 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 07:27:58.953232 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.953213 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:27:58.954165 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.954150 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-54.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:27:58.954259 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.954180 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-54.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:27:58.954259 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.954191 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-54.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:27:58.954259 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.954219 2553 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-54.ec2.internal" Sep 17 07:27:58.968736 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:58.968674 2553 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-54.ec2.internal" Sep 17 07:27:58.968736 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:58.968699 2553 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-54.ec2.internal\": node \"ip-10-0-141-54.ec2.internal\" not found" Sep 17 07:27:58.989304 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:58.989286 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-54.ec2.internal\" not found" Sep 17 07:27:59.028465 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.028436 2553 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-54.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-54.ec2.internal"] Sep 17 07:27:59.028575 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.028506 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:27:59.030180 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.030159 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-54.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:27:59.030312 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.030190 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-54.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:27:59.030312 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.030200 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-54.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:27:59.031508 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.031494 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:27:59.031640 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.031624 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-54.ec2.internal" Sep 17 07:27:59.031732 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.031660 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:27:59.032241 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.032223 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-54.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:27:59.032354 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.032251 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-54.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:27:59.032354 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.032261 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-54.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:27:59.032354 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.032223 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-54.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:27:59.032354 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.032302 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-54.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:27:59.032354 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.032318 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-54.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:27:59.033425 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.033410 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-54.ec2.internal" Sep 17 07:27:59.033505 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.033438 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:27:59.034080 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.034065 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-54.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:27:59.034144 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.034090 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-54.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:27:59.034144 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.034103 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-54.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:27:59.055507 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:59.055481 2553 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-54.ec2.internal\" not found" node="ip-10-0-141-54.ec2.internal" Sep 17 07:27:59.059813 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:59.059799 2553 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-54.ec2.internal\" not found" node="ip-10-0-141-54.ec2.internal" Sep 17 07:27:59.090100 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:59.090077 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-54.ec2.internal\" not found" Sep 17 07:27:59.103434 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.103410 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bf34cff4e5d7631f28563807f0551e5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-54.ec2.internal\" (UID: \"7bf34cff4e5d7631f28563807f0551e5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-54.ec2.internal" Sep 17 07:27:59.103523 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.103438 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ea85235c507e0a22eb3b6e21763019df-config\") pod \"kube-apiserver-proxy-ip-10-0-141-54.ec2.internal\" (UID: \"ea85235c507e0a22eb3b6e21763019df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-54.ec2.internal" Sep 17 07:27:59.103523 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.103455 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7bf34cff4e5d7631f28563807f0551e5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-54.ec2.internal\" (UID: \"7bf34cff4e5d7631f28563807f0551e5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-54.ec2.internal" Sep 17 07:27:59.190388 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:59.190356 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-54.ec2.internal\" not found" Sep 17 07:27:59.203759 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.203733 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7bf34cff4e5d7631f28563807f0551e5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-54.ec2.internal\" (UID: \"7bf34cff4e5d7631f28563807f0551e5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-54.ec2.internal" Sep 17 07:27:59.203905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.203767 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bf34cff4e5d7631f28563807f0551e5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-54.ec2.internal\" (UID: \"7bf34cff4e5d7631f28563807f0551e5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-54.ec2.internal" Sep 17 07:27:59.203905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.203791 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ea85235c507e0a22eb3b6e21763019df-config\") pod \"kube-apiserver-proxy-ip-10-0-141-54.ec2.internal\" (UID: \"ea85235c507e0a22eb3b6e21763019df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-54.ec2.internal" Sep 17 07:27:59.203905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.203858 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ea85235c507e0a22eb3b6e21763019df-config\") pod \"kube-apiserver-proxy-ip-10-0-141-54.ec2.internal\" (UID: \"ea85235c507e0a22eb3b6e21763019df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-54.ec2.internal" Sep 17 07:27:59.203905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.203862 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7bf34cff4e5d7631f28563807f0551e5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-54.ec2.internal\" (UID: \"7bf34cff4e5d7631f28563807f0551e5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-54.ec2.internal" Sep 17 07:27:59.204039 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.203923 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bf34cff4e5d7631f28563807f0551e5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-54.ec2.internal\" (UID: \"7bf34cff4e5d7631f28563807f0551e5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-54.ec2.internal" Sep 17 07:27:59.291005 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:59.290931 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-54.ec2.internal\" not found" Sep 17 07:27:59.362277 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.362246 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-54.ec2.internal" Sep 17 07:27:59.362367 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.362327 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-54.ec2.internal" Sep 17 07:27:59.392506 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:59.392475 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-54.ec2.internal\" not found" Sep 17 07:27:59.498949 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:59.498877 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-54.ec2.internal\" not found" Sep 17 07:27:59.599067 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:59.598989 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-54.ec2.internal\" not found" Sep 17 07:27:59.680706 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.680674 2553 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 07:27:59.700920 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.700892 2553 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-54.ec2.internal" Sep 17 07:27:59.719268 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.719243 2553 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 17 07:27:59.721567 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.721551 2553 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-54.ec2.internal" Sep 17 07:27:59.732532 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.732512 2553 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 17 07:27:59.737746 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.737732 2553 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 17 07:27:59.737854 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.737838 2553 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 17 07:27:59.737917 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.737896 2553 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 17 07:27:59.781562 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.781536 2553 apiserver.go:52] "Watching apiserver" Sep 17 07:27:59.798766 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.798742 2553 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 17 07:27:59.802175 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.802151 2553 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 17 07:27:59.804034 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.804002 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-spnx4","openshift-multus/multus-rb9tt","openshift-cluster-node-tuning-operator/tuned-bb6f2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-54.ec2.internal","openshift-multus/network-metrics-daemon-xh2n2","openshift-network-diagnostics/network-check-target-tsc9z","openshift-network-operator/iptables-alerter-h5j5d","openshift-ovn-kubernetes/ovnkube-node-zhw5d","kube-system/global-pull-secret-syncer-gh9zp","kube-system/konnectivity-agent-crmdq","kube-system/kube-apiserver-proxy-ip-10-0-141-54.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n","openshift-dns/node-resolver-ddqsw","openshift-image-registry/node-ca-7l8hw"] Sep 17 07:27:59.806569 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.806545 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:27:59.807759 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.807740 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rb9tt" Sep 17 07:27:59.808858 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.808841 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:27:59.810039 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.810021 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ddqsw" Sep 17 07:27:59.810478 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.810460 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 17 07:27:59.810575 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.810492 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 17 07:27:59.811379 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.811363 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:27:59.811476 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:59.811445 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2n2" podUID="c35e7d45-1111-40a8-acf5-666b633e8180" Sep 17 07:27:59.812343 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.812318 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 17 07:27:59.812458 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.812439 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 17 07:27:59.812567 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.812552 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" Sep 17 07:27:59.812736 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.812705 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-t9xqk\"" Sep 17 07:27:59.812823 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.812744 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 17 07:27:59.812823 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.812770 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 17 07:27:59.813004 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.812940 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 17 07:27:59.813004 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.812973 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-w8cjt\"" Sep 17 07:27:59.813004 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.812996 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 17 07:27:59.813111 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.813039 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 17 07:27:59.813111 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.813062 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 17 07:27:59.813111 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.813100 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 17 07:27:59.813343 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.813329 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-62wxf\"" Sep 17 07:27:59.813470 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.813385 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 17 07:27:59.813470 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.813422 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2rx4j\"" Sep 17 07:27:59.813470 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.813429 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 17 07:27:59.813651 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.813473 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 17 07:27:59.814115 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.814097 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-spnx4" Sep 17 07:27:59.815527 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.815508 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h5j5d" Sep 17 07:27:59.816977 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.816946 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:27:59.817072 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:59.817012 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gh9zp" podUID="f68f4c1a-7946-43c1-80ca-b472129b2bfe" Sep 17 07:27:59.818283 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.818265 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-crmdq" Sep 17 07:27:59.819769 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.819750 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:27:59.819871 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:27:59.819823 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsc9z" podUID="6a96dd04-5687-4059-bf3f-ee1864ed8e40" Sep 17 07:27:59.820565 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.820546 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 17 07:27:59.821450 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.821432 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7l8hw" Sep 17 07:27:59.823232 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.823192 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 17 07:27:59.824714 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.824699 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 17 07:27:59.825760 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.825742 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-f8d27\"" Sep 17 07:27:59.825760 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.825752 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 17 07:27:59.826420 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.826402 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 17 07:27:59.826779 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.826761 2553 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 17 07:27:59.827211 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.827195 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-558ds\"" Sep 17 07:27:59.827342 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.827327 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 17 07:27:59.828354 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.828338 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 17 07:27:59.828838 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.828821 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 17 07:27:59.833201 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.833181 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mg29x\"" Sep 17 07:27:59.833470 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.833445 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 17 07:27:59.833547 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.833478 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 17 07:27:59.833601 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.833478 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 17 07:27:59.833793 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.833770 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xkmph\"" Sep 17 07:27:59.833873 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.833819 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 17 07:27:59.833873 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.833783 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jrjlc\"" Sep 17 07:27:59.833977 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.833880 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 17 07:27:59.862792 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.862756 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 07:22:59 +0000 UTC" deadline="2028-02-19 20:23:52.081436934 +0000 UTC" Sep 17 07:27:59.862792 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.862791 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12492h55m52.21865046s" Sep 17 07:27:59.862972 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.862813 2553 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-92qq8" Sep 17 07:27:59.878300 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.878280 2553 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-92qq8" Sep 17 07:27:59.902924 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.902907 2553 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 17 07:27:59.908042 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908021 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-etc-sysctl-conf\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:27:59.908133 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908048 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-etc-systemd\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:27:59.908133 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908066 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-sys\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:27:59.908133 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908084 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4bjq\" (UniqueName: \"kubernetes.io/projected/76674b13-2b8e-49bf-b508-773f53a76a95-kube-api-access-j4bjq\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:27:59.908133 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908117 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f097781e-5016-483b-8e25-85ac5ef23767-etc-selinux\") pod \"aws-ebs-csi-driver-node-s6r7n\" (UID: \"f097781e-5016-483b-8e25-85ac5ef23767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" Sep 17 07:27:59.908266 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908134 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f68f4c1a-7946-43c1-80ca-b472129b2bfe-original-pull-secret\") pod \"global-pull-secret-syncer-gh9zp\" (UID: \"f68f4c1a-7946-43c1-80ca-b472129b2bfe\") " pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:27:59.908266 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908151 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mljcs\" (UniqueName: \"kubernetes.io/projected/b388be8d-ade7-4e8d-b182-c27d6b5c5b1a-kube-api-access-mljcs\") pod \"node-ca-7l8hw\" (UID: \"b388be8d-ade7-4e8d-b182-c27d6b5c5b1a\") " pod="openshift-image-registry/node-ca-7l8hw" Sep 17 07:27:59.908266 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908166 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-run\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:27:59.908266 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908192 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-host-run-ovn-kubernetes\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:27:59.908266 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908215 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-host-cni-netd\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:27:59.908266 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908236 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-host-var-lib-cni-bin\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:27:59.908266 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908252 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-host-run-multus-certs\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:27:59.908266 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908266 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-etc-modprobe-d\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:27:59.908499 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908284 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c35e7d45-1111-40a8-acf5-666b633e8180-metrics-certs\") pod \"network-metrics-daemon-xh2n2\" (UID: \"c35e7d45-1111-40a8-acf5-666b633e8180\") " pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:27:59.908499 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908310 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8af26520-8854-41c6-a082-58fd1e819408-system-cni-dir\") pod \"multus-additional-cni-plugins-spnx4\" (UID: \"8af26520-8854-41c6-a082-58fd1e819408\") " pod="openshift-multus/multus-additional-cni-plugins-spnx4" Sep 17 07:27:59.908499 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908328 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f097781e-5016-483b-8e25-85ac5ef23767-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s6r7n\" (UID: \"f097781e-5016-483b-8e25-85ac5ef23767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" Sep 17 07:27:59.908499 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908342 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-multus-socket-dir-parent\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:27:59.908499 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908356 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-hostroot\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:27:59.908499 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908383 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv79h\" (UniqueName: \"kubernetes.io/projected/251d0b36-1511-412f-8a1c-d599e9c134b6-kube-api-access-xv79h\") pod \"iptables-alerter-h5j5d\" (UID: \"251d0b36-1511-412f-8a1c-d599e9c134b6\") " pod="openshift-network-operator/iptables-alerter-h5j5d" Sep 17 07:27:59.908499 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908406 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b388be8d-ade7-4e8d-b182-c27d6b5c5b1a-serviceca\") pod \"node-ca-7l8hw\" (UID: \"b388be8d-ade7-4e8d-b182-c27d6b5c5b1a\") " pod="openshift-image-registry/node-ca-7l8hw" Sep 17 07:27:59.908499 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908423 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-etc-sysctl-d\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:27:59.908499 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908441 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-lib-modules\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:27:59.908499 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908455 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8af26520-8854-41c6-a082-58fd1e819408-cni-binary-copy\") pod \"multus-additional-cni-plugins-spnx4\" (UID: \"8af26520-8854-41c6-a082-58fd1e819408\") " pod="openshift-multus/multus-additional-cni-plugins-spnx4" Sep 17 07:27:59.908499 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908478 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f097781e-5016-483b-8e25-85ac5ef23767-device-dir\") pod \"aws-ebs-csi-driver-node-s6r7n\" (UID: \"f097781e-5016-483b-8e25-85ac5ef23767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" Sep 17 07:27:59.908499 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908494 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-host-run-netns\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:27:59.908845 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908514 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/77d69f00-3692-47b0-845d-f07095a74966-ovnkube-script-lib\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:27:59.908845 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908529 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-host-run-netns\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:27:59.908845 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908554 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d54b36a-cd25-4e0e-8a1d-a0b1ce6de23e-tmp-dir\") pod \"node-resolver-ddqsw\" (UID: \"6d54b36a-cd25-4e0e-8a1d-a0b1ce6de23e\") " pod="openshift-dns/node-resolver-ddqsw" Sep 17 07:27:59.908845 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908585 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8af26520-8854-41c6-a082-58fd1e819408-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-spnx4\" (UID: \"8af26520-8854-41c6-a082-58fd1e819408\") " pod="openshift-multus/multus-additional-cni-plugins-spnx4" Sep 17 07:27:59.908845 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908618 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f097781e-5016-483b-8e25-85ac5ef23767-socket-dir\") pod \"aws-ebs-csi-driver-node-s6r7n\" (UID: \"f097781e-5016-483b-8e25-85ac5ef23767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" Sep 17 07:27:59.908845 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908634 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/77d69f00-3692-47b0-845d-f07095a74966-ovnkube-config\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:27:59.908845 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908649 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-var-lib-kubelet\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:27:59.908845 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908664 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f097781e-5016-483b-8e25-85ac5ef23767-registration-dir\") pod \"aws-ebs-csi-driver-node-s6r7n\" (UID: \"f097781e-5016-483b-8e25-85ac5ef23767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" Sep 17 07:27:59.908845 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908680 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/251d0b36-1511-412f-8a1c-d599e9c134b6-host-slash\") pod \"iptables-alerter-h5j5d\" (UID: \"251d0b36-1511-412f-8a1c-d599e9c134b6\") " pod="openshift-network-operator/iptables-alerter-h5j5d" Sep 17 07:27:59.908845 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908694 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b388be8d-ade7-4e8d-b182-c27d6b5c5b1a-host\") pod \"node-ca-7l8hw\" (UID: \"b388be8d-ade7-4e8d-b182-c27d6b5c5b1a\") " pod="openshift-image-registry/node-ca-7l8hw" Sep 17 07:27:59.908845 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908710 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-run-systemd\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:27:59.908845 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908725 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-run-openvswitch\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:27:59.908845 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908739 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/77d69f00-3692-47b0-845d-f07095a74966-ovn-node-metrics-cert\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:27:59.908845 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908753 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-cnibin\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:27:59.908845 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908767 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48ssr\" (UniqueName: \"kubernetes.io/projected/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-kube-api-access-48ssr\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:27:59.908845 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908782 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f68f4c1a-7946-43c1-80ca-b472129b2bfe-dbus\") pod \"global-pull-secret-syncer-gh9zp\" (UID: \"f68f4c1a-7946-43c1-80ca-b472129b2bfe\") " pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:27:59.909473 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908797 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5bea09db-e130-47db-83df-6467274459a1-konnectivity-ca\") pod \"konnectivity-agent-crmdq\" (UID: \"5bea09db-e130-47db-83df-6467274459a1\") " pod="kube-system/konnectivity-agent-crmdq" Sep 17 07:27:59.909473 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908812 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-var-lib-openvswitch\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:27:59.909473 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908825 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-run-ovn\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:27:59.909473 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908838 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-host-cni-bin\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:27:59.909473 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908869 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-system-cni-dir\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:27:59.909473 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908906 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-cni-binary-copy\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:27:59.909473 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908925 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6d54b36a-cd25-4e0e-8a1d-a0b1ce6de23e-hosts-file\") pod \"node-resolver-ddqsw\" (UID: \"6d54b36a-cd25-4e0e-8a1d-a0b1ce6de23e\") " pod="openshift-dns/node-resolver-ddqsw" Sep 17 07:27:59.909473 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908953 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f68f4c1a-7946-43c1-80ca-b472129b2bfe-kubelet-config\") pod \"global-pull-secret-syncer-gh9zp\" (UID: \"f68f4c1a-7946-43c1-80ca-b472129b2bfe\") " pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:27:59.909473 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.908993 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-log-socket\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:27:59.909473 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909015 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-host-var-lib-kubelet\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:27:59.909473 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909031 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8af26520-8854-41c6-a082-58fd1e819408-cnibin\") pod \"multus-additional-cni-plugins-spnx4\" (UID: \"8af26520-8854-41c6-a082-58fd1e819408\") " pod="openshift-multus/multus-additional-cni-plugins-spnx4" Sep 17 07:27:59.909473 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909049 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8af26520-8854-41c6-a082-58fd1e819408-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-spnx4\" (UID: \"8af26520-8854-41c6-a082-58fd1e819408\") " pod="openshift-multus/multus-additional-cni-plugins-spnx4" Sep 17 07:27:59.909473 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909074 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bfz5\" (UniqueName: \"kubernetes.io/projected/8af26520-8854-41c6-a082-58fd1e819408-kube-api-access-2bfz5\") pod \"multus-additional-cni-plugins-spnx4\" (UID: \"8af26520-8854-41c6-a082-58fd1e819408\") " pod="openshift-multus/multus-additional-cni-plugins-spnx4" Sep 17 07:27:59.909473 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909092 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vc7c\" (UniqueName: \"kubernetes.io/projected/6a96dd04-5687-4059-bf3f-ee1864ed8e40-kube-api-access-5vc7c\") pod \"network-check-target-tsc9z\" (UID: \"6a96dd04-5687-4059-bf3f-ee1864ed8e40\") " pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:27:59.909473 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909107 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tf5c\" (UniqueName: \"kubernetes.io/projected/f097781e-5016-483b-8e25-85ac5ef23767-kube-api-access-7tf5c\") pod \"aws-ebs-csi-driver-node-s6r7n\" (UID: \"f097781e-5016-483b-8e25-85ac5ef23767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" Sep 17 07:27:59.909473 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909121 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-host-kubelet\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:27:59.910059 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909149 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:27:59.910059 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909168 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-multus-conf-dir\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:27:59.910059 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909183 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-multus-daemon-config\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:27:59.910059 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909196 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-host\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:27:59.910059 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909238 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsc6l\" (UniqueName: \"kubernetes.io/projected/6d54b36a-cd25-4e0e-8a1d-a0b1ce6de23e-kube-api-access-tsc6l\") pod \"node-resolver-ddqsw\" (UID: \"6d54b36a-cd25-4e0e-8a1d-a0b1ce6de23e\") " pod="openshift-dns/node-resolver-ddqsw" Sep 17 07:27:59.910059 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909254 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8af26520-8854-41c6-a082-58fd1e819408-tuning-conf-dir\") pod \"multus-additional-cni-plugins-spnx4\" (UID: \"8af26520-8854-41c6-a082-58fd1e819408\") " pod="openshift-multus/multus-additional-cni-plugins-spnx4" Sep 17 07:27:59.910059 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909269 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-host-slash\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:27:59.910059 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909285 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxb5g\" (UniqueName: \"kubernetes.io/projected/77d69f00-3692-47b0-845d-f07095a74966-kube-api-access-sxb5g\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:27:59.910059 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909323 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-etc-kubernetes\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:27:59.910059 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909367 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/76674b13-2b8e-49bf-b508-773f53a76a95-etc-tuned\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:27:59.910059 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909404 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5bea09db-e130-47db-83df-6467274459a1-agent-certs\") pod \"konnectivity-agent-crmdq\" (UID: \"5bea09db-e130-47db-83df-6467274459a1\") " pod="kube-system/konnectivity-agent-crmdq" Sep 17 07:27:59.910059 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909436 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-etc-openvswitch\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:27:59.910059 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909464 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/251d0b36-1511-412f-8a1c-d599e9c134b6-iptables-alerter-script\") pod \"iptables-alerter-h5j5d\" (UID: \"251d0b36-1511-412f-8a1c-d599e9c134b6\") " pod="openshift-network-operator/iptables-alerter-h5j5d" Sep 17 07:27:59.910059 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909490 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-multus-cni-dir\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:27:59.910059 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909513 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-etc-kubernetes\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:27:59.910059 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909567 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-etc-sysconfig\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:27:59.910610 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909602 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-node-log\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:27:59.910610 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909619 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/77d69f00-3692-47b0-845d-f07095a74966-env-overrides\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:27:59.910610 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909639 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-host-run-k8s-cni-cncf-io\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:27:59.910610 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909665 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76674b13-2b8e-49bf-b508-773f53a76a95-tmp\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:27:59.910610 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909682 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llvf6\" (UniqueName: \"kubernetes.io/projected/c35e7d45-1111-40a8-acf5-666b633e8180-kube-api-access-llvf6\") pod \"network-metrics-daemon-xh2n2\" (UID: \"c35e7d45-1111-40a8-acf5-666b633e8180\") " pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:27:59.910610 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909698 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f097781e-5016-483b-8e25-85ac5ef23767-sys-fs\") pod \"aws-ebs-csi-driver-node-s6r7n\" (UID: \"f097781e-5016-483b-8e25-85ac5ef23767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" Sep 17 07:27:59.910610 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909740 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-systemd-units\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:27:59.910610 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.909765 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-host-var-lib-cni-multus\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:27:59.955273 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:27:59.955248 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 07:28:00.010621 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010583 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-node-log\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.010621 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010621 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/77d69f00-3692-47b0-845d-f07095a74966-env-overrides\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.010857 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010638 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-host-run-k8s-cni-cncf-io\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.010857 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010654 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76674b13-2b8e-49bf-b508-773f53a76a95-tmp\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.010857 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010677 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llvf6\" (UniqueName: \"kubernetes.io/projected/c35e7d45-1111-40a8-acf5-666b633e8180-kube-api-access-llvf6\") pod \"network-metrics-daemon-xh2n2\" (UID: \"c35e7d45-1111-40a8-acf5-666b633e8180\") " pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:00.010857 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010694 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f097781e-5016-483b-8e25-85ac5ef23767-sys-fs\") pod \"aws-ebs-csi-driver-node-s6r7n\" (UID: \"f097781e-5016-483b-8e25-85ac5ef23767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" Sep 17 07:28:00.010857 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010696 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-node-log\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.010857 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010723 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-systemd-units\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.010857 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010748 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-host-var-lib-cni-multus\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.010857 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010773 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-etc-sysctl-conf\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.010857 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010798 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-etc-systemd\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.010857 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010799 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f097781e-5016-483b-8e25-85ac5ef23767-sys-fs\") pod \"aws-ebs-csi-driver-node-s6r7n\" (UID: \"f097781e-5016-483b-8e25-85ac5ef23767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" Sep 17 07:28:00.010857 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010721 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-host-run-k8s-cni-cncf-io\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.010857 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010833 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-sys\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.010857 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010846 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-systemd-units\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.010857 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010858 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4bjq\" (UniqueName: \"kubernetes.io/projected/76674b13-2b8e-49bf-b508-773f53a76a95-kube-api-access-j4bjq\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.011487 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010870 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-etc-systemd\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.011487 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010884 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f097781e-5016-483b-8e25-85ac5ef23767-etc-selinux\") pod \"aws-ebs-csi-driver-node-s6r7n\" (UID: \"f097781e-5016-483b-8e25-85ac5ef23767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" Sep 17 07:28:00.011487 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010924 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-sys\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.011487 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010925 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-host-var-lib-cni-multus\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.011487 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010928 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f68f4c1a-7946-43c1-80ca-b472129b2bfe-original-pull-secret\") pod \"global-pull-secret-syncer-gh9zp\" (UID: \"f68f4c1a-7946-43c1-80ca-b472129b2bfe\") " pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:00.011487 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010975 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mljcs\" (UniqueName: \"kubernetes.io/projected/b388be8d-ade7-4e8d-b182-c27d6b5c5b1a-kube-api-access-mljcs\") pod \"node-ca-7l8hw\" (UID: \"b388be8d-ade7-4e8d-b182-c27d6b5c5b1a\") " pod="openshift-image-registry/node-ca-7l8hw" Sep 17 07:28:00.011487 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010976 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f097781e-5016-483b-8e25-85ac5ef23767-etc-selinux\") pod \"aws-ebs-csi-driver-node-s6r7n\" (UID: \"f097781e-5016-483b-8e25-85ac5ef23767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" Sep 17 07:28:00.011487 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010984 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-etc-sysctl-conf\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.011487 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.010996 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-run\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.011487 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:00.011067 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:00.011487 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:00.011139 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f68f4c1a-7946-43c1-80ca-b472129b2bfe-original-pull-secret podName:f68f4c1a-7946-43c1-80ca-b472129b2bfe nodeName:}" failed. No retries permitted until 2026-09-17 07:28:00.511117845 +0000 UTC m=+2.069302653 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f68f4c1a-7946-43c1-80ca-b472129b2bfe-original-pull-secret") pod "global-pull-secret-syncer-gh9zp" (UID: "f68f4c1a-7946-43c1-80ca-b472129b2bfe") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:00.011487 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011163 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-host-run-ovn-kubernetes\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.011487 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011194 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-host-cni-netd\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.011487 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011224 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-host-var-lib-cni-bin\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.011487 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011228 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-host-run-ovn-kubernetes\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.011487 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011242 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/77d69f00-3692-47b0-845d-f07095a74966-env-overrides\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.011487 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011203 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-run\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.012369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011252 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-host-run-multus-certs\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.012369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011258 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-host-cni-netd\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.012369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011281 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-etc-modprobe-d\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.012369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011292 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-host-var-lib-cni-bin\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.012369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011320 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c35e7d45-1111-40a8-acf5-666b633e8180-metrics-certs\") pod \"network-metrics-daemon-xh2n2\" (UID: \"c35e7d45-1111-40a8-acf5-666b633e8180\") " pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:00.012369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011327 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-host-run-multus-certs\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.012369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011350 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8af26520-8854-41c6-a082-58fd1e819408-system-cni-dir\") pod \"multus-additional-cni-plugins-spnx4\" (UID: \"8af26520-8854-41c6-a082-58fd1e819408\") " pod="openshift-multus/multus-additional-cni-plugins-spnx4" Sep 17 07:28:00.012369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011387 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-etc-modprobe-d\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.012369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011393 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f097781e-5016-483b-8e25-85ac5ef23767-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s6r7n\" (UID: \"f097781e-5016-483b-8e25-85ac5ef23767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" Sep 17 07:28:00.012369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011409 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8af26520-8854-41c6-a082-58fd1e819408-system-cni-dir\") pod \"multus-additional-cni-plugins-spnx4\" (UID: \"8af26520-8854-41c6-a082-58fd1e819408\") " pod="openshift-multus/multus-additional-cni-plugins-spnx4" Sep 17 07:28:00.012369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011415 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-multus-socket-dir-parent\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.012369 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:00.011441 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:00.012369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011451 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-hostroot\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.012369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011467 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f097781e-5016-483b-8e25-85ac5ef23767-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s6r7n\" (UID: \"f097781e-5016-483b-8e25-85ac5ef23767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" Sep 17 07:28:00.012369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011466 2553 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 17 07:28:00.012369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011481 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv79h\" (UniqueName: \"kubernetes.io/projected/251d0b36-1511-412f-8a1c-d599e9c134b6-kube-api-access-xv79h\") pod \"iptables-alerter-h5j5d\" (UID: \"251d0b36-1511-412f-8a1c-d599e9c134b6\") " pod="openshift-network-operator/iptables-alerter-h5j5d" Sep 17 07:28:00.012369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011463 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-multus-socket-dir-parent\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.013241 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:00.011498 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c35e7d45-1111-40a8-acf5-666b633e8180-metrics-certs podName:c35e7d45-1111-40a8-acf5-666b633e8180 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:00.511476518 +0000 UTC m=+2.069661306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c35e7d45-1111-40a8-acf5-666b633e8180-metrics-certs") pod "network-metrics-daemon-xh2n2" (UID: "c35e7d45-1111-40a8-acf5-666b633e8180") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:00.013241 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011511 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-hostroot\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.013241 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011527 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b388be8d-ade7-4e8d-b182-c27d6b5c5b1a-serviceca\") pod \"node-ca-7l8hw\" (UID: \"b388be8d-ade7-4e8d-b182-c27d6b5c5b1a\") " pod="openshift-image-registry/node-ca-7l8hw" Sep 17 07:28:00.013241 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011548 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-etc-sysctl-d\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.013241 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011573 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-lib-modules\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.013241 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011591 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8af26520-8854-41c6-a082-58fd1e819408-cni-binary-copy\") pod \"multus-additional-cni-plugins-spnx4\" (UID: \"8af26520-8854-41c6-a082-58fd1e819408\") " pod="openshift-multus/multus-additional-cni-plugins-spnx4" Sep 17 07:28:00.013241 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011609 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f097781e-5016-483b-8e25-85ac5ef23767-device-dir\") pod \"aws-ebs-csi-driver-node-s6r7n\" (UID: \"f097781e-5016-483b-8e25-85ac5ef23767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" Sep 17 07:28:00.013241 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011627 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-host-run-netns\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.013241 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011654 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/77d69f00-3692-47b0-845d-f07095a74966-ovnkube-script-lib\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.013241 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011679 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-etc-sysctl-d\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.013241 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011740 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f097781e-5016-483b-8e25-85ac5ef23767-device-dir\") pod \"aws-ebs-csi-driver-node-s6r7n\" (UID: \"f097781e-5016-483b-8e25-85ac5ef23767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" Sep 17 07:28:00.013241 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011759 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-lib-modules\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.013241 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011761 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-host-run-netns\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.013241 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011802 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-host-run-netns\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.013241 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011830 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d54b36a-cd25-4e0e-8a1d-a0b1ce6de23e-tmp-dir\") pod \"node-resolver-ddqsw\" (UID: \"6d54b36a-cd25-4e0e-8a1d-a0b1ce6de23e\") " pod="openshift-dns/node-resolver-ddqsw" Sep 17 07:28:00.013241 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011850 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8af26520-8854-41c6-a082-58fd1e819408-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-spnx4\" (UID: \"8af26520-8854-41c6-a082-58fd1e819408\") " pod="openshift-multus/multus-additional-cni-plugins-spnx4" Sep 17 07:28:00.013241 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011864 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-host-run-netns\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.014124 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011872 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f097781e-5016-483b-8e25-85ac5ef23767-socket-dir\") pod \"aws-ebs-csi-driver-node-s6r7n\" (UID: \"f097781e-5016-483b-8e25-85ac5ef23767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" Sep 17 07:28:00.014124 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011929 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/77d69f00-3692-47b0-845d-f07095a74966-ovnkube-config\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.014124 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011956 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-var-lib-kubelet\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.014124 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.011982 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f097781e-5016-483b-8e25-85ac5ef23767-registration-dir\") pod \"aws-ebs-csi-driver-node-s6r7n\" (UID: \"f097781e-5016-483b-8e25-85ac5ef23767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" Sep 17 07:28:00.014124 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012010 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/251d0b36-1511-412f-8a1c-d599e9c134b6-host-slash\") pod \"iptables-alerter-h5j5d\" (UID: \"251d0b36-1511-412f-8a1c-d599e9c134b6\") " pod="openshift-network-operator/iptables-alerter-h5j5d" Sep 17 07:28:00.014124 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012039 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b388be8d-ade7-4e8d-b182-c27d6b5c5b1a-host\") pod \"node-ca-7l8hw\" (UID: \"b388be8d-ade7-4e8d-b182-c27d6b5c5b1a\") " pod="openshift-image-registry/node-ca-7l8hw" Sep 17 07:28:00.014124 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012064 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-run-systemd\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.014124 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012089 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-run-openvswitch\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.014124 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012108 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f097781e-5016-483b-8e25-85ac5ef23767-socket-dir\") pod \"aws-ebs-csi-driver-node-s6r7n\" (UID: \"f097781e-5016-483b-8e25-85ac5ef23767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" Sep 17 07:28:00.014124 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012112 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/77d69f00-3692-47b0-845d-f07095a74966-ovn-node-metrics-cert\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.014124 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012145 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-cnibin\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.014124 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012168 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48ssr\" (UniqueName: \"kubernetes.io/projected/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-kube-api-access-48ssr\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.014124 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012190 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f68f4c1a-7946-43c1-80ca-b472129b2bfe-dbus\") pod \"global-pull-secret-syncer-gh9zp\" (UID: \"f68f4c1a-7946-43c1-80ca-b472129b2bfe\") " pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:00.014124 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012212 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5bea09db-e130-47db-83df-6467274459a1-konnectivity-ca\") pod \"konnectivity-agent-crmdq\" (UID: \"5bea09db-e130-47db-83df-6467274459a1\") " pod="kube-system/konnectivity-agent-crmdq" Sep 17 07:28:00.014124 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012227 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d54b36a-cd25-4e0e-8a1d-a0b1ce6de23e-tmp-dir\") pod \"node-resolver-ddqsw\" (UID: \"6d54b36a-cd25-4e0e-8a1d-a0b1ce6de23e\") " pod="openshift-dns/node-resolver-ddqsw" Sep 17 07:28:00.014124 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012235 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-var-lib-openvswitch\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.014124 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012264 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8af26520-8854-41c6-a082-58fd1e819408-cni-binary-copy\") pod \"multus-additional-cni-plugins-spnx4\" (UID: \"8af26520-8854-41c6-a082-58fd1e819408\") " pod="openshift-multus/multus-additional-cni-plugins-spnx4" Sep 17 07:28:00.015081 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012282 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-run-ovn\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.015081 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012320 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-run-ovn\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.015081 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012320 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-host-cni-bin\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.015081 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012360 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-host-cni-bin\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.015081 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012371 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/251d0b36-1511-412f-8a1c-d599e9c134b6-host-slash\") pod \"iptables-alerter-h5j5d\" (UID: \"251d0b36-1511-412f-8a1c-d599e9c134b6\") " pod="openshift-network-operator/iptables-alerter-h5j5d" Sep 17 07:28:00.015081 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012376 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-system-cni-dir\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.015081 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012430 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f68f4c1a-7946-43c1-80ca-b472129b2bfe-dbus\") pod \"global-pull-secret-syncer-gh9zp\" (UID: \"f68f4c1a-7946-43c1-80ca-b472129b2bfe\") " pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:00.015081 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012446 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-cni-binary-copy\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.015081 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012461 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8af26520-8854-41c6-a082-58fd1e819408-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-spnx4\" (UID: \"8af26520-8854-41c6-a082-58fd1e819408\") " pod="openshift-multus/multus-additional-cni-plugins-spnx4" Sep 17 07:28:00.015081 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012477 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6d54b36a-cd25-4e0e-8a1d-a0b1ce6de23e-hosts-file\") pod \"node-resolver-ddqsw\" (UID: \"6d54b36a-cd25-4e0e-8a1d-a0b1ce6de23e\") " pod="openshift-dns/node-resolver-ddqsw" Sep 17 07:28:00.015081 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012512 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-system-cni-dir\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.015081 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012519 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b388be8d-ade7-4e8d-b182-c27d6b5c5b1a-serviceca\") pod \"node-ca-7l8hw\" (UID: \"b388be8d-ade7-4e8d-b182-c27d6b5c5b1a\") " pod="openshift-image-registry/node-ca-7l8hw" Sep 17 07:28:00.015081 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012524 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f68f4c1a-7946-43c1-80ca-b472129b2bfe-kubelet-config\") pod \"global-pull-secret-syncer-gh9zp\" (UID: \"f68f4c1a-7946-43c1-80ca-b472129b2bfe\") " pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:00.015081 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012554 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-log-socket\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.015081 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012561 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-cnibin\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.015081 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012579 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-var-lib-openvswitch\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.015081 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012615 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f68f4c1a-7946-43c1-80ca-b472129b2bfe-kubelet-config\") pod \"global-pull-secret-syncer-gh9zp\" (UID: \"f68f4c1a-7946-43c1-80ca-b472129b2bfe\") " pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:00.015081 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012631 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f097781e-5016-483b-8e25-85ac5ef23767-registration-dir\") pod \"aws-ebs-csi-driver-node-s6r7n\" (UID: \"f097781e-5016-483b-8e25-85ac5ef23767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" Sep 17 07:28:00.016022 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012657 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-log-socket\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.016022 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012681 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-var-lib-kubelet\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.016022 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012711 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-run-systemd\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.016022 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012752 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b388be8d-ade7-4e8d-b182-c27d6b5c5b1a-host\") pod \"node-ca-7l8hw\" (UID: \"b388be8d-ade7-4e8d-b182-c27d6b5c5b1a\") " pod="openshift-image-registry/node-ca-7l8hw" Sep 17 07:28:00.016022 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012771 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-host-var-lib-kubelet\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.016022 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012794 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-run-openvswitch\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.016022 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012802 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8af26520-8854-41c6-a082-58fd1e819408-cnibin\") pod \"multus-additional-cni-plugins-spnx4\" (UID: \"8af26520-8854-41c6-a082-58fd1e819408\") " pod="openshift-multus/multus-additional-cni-plugins-spnx4" Sep 17 07:28:00.016022 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012829 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8af26520-8854-41c6-a082-58fd1e819408-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-spnx4\" (UID: \"8af26520-8854-41c6-a082-58fd1e819408\") " pod="openshift-multus/multus-additional-cni-plugins-spnx4" Sep 17 07:28:00.016022 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012836 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-host-var-lib-kubelet\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.016022 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012850 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5bea09db-e130-47db-83df-6467274459a1-konnectivity-ca\") pod \"konnectivity-agent-crmdq\" (UID: \"5bea09db-e130-47db-83df-6467274459a1\") " pod="kube-system/konnectivity-agent-crmdq" Sep 17 07:28:00.016022 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012857 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bfz5\" (UniqueName: \"kubernetes.io/projected/8af26520-8854-41c6-a082-58fd1e819408-kube-api-access-2bfz5\") pod \"multus-additional-cni-plugins-spnx4\" (UID: \"8af26520-8854-41c6-a082-58fd1e819408\") " pod="openshift-multus/multus-additional-cni-plugins-spnx4" Sep 17 07:28:00.016022 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012879 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8af26520-8854-41c6-a082-58fd1e819408-cnibin\") pod \"multus-additional-cni-plugins-spnx4\" (UID: \"8af26520-8854-41c6-a082-58fd1e819408\") " pod="openshift-multus/multus-additional-cni-plugins-spnx4" Sep 17 07:28:00.016022 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012883 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vc7c\" (UniqueName: \"kubernetes.io/projected/6a96dd04-5687-4059-bf3f-ee1864ed8e40-kube-api-access-5vc7c\") pod \"network-check-target-tsc9z\" (UID: \"6a96dd04-5687-4059-bf3f-ee1864ed8e40\") " pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:28:00.016022 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012910 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tf5c\" (UniqueName: \"kubernetes.io/projected/f097781e-5016-483b-8e25-85ac5ef23767-kube-api-access-7tf5c\") pod \"aws-ebs-csi-driver-node-s6r7n\" (UID: \"f097781e-5016-483b-8e25-85ac5ef23767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" Sep 17 07:28:00.016022 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012973 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-host-kubelet\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.016022 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.012988 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/77d69f00-3692-47b0-845d-f07095a74966-ovnkube-config\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.016022 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013005 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.016586 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013040 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-multus-conf-dir\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.016586 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013065 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-multus-daemon-config\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.016586 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013051 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6d54b36a-cd25-4e0e-8a1d-a0b1ce6de23e-hosts-file\") pod \"node-resolver-ddqsw\" (UID: \"6d54b36a-cd25-4e0e-8a1d-a0b1ce6de23e\") " pod="openshift-dns/node-resolver-ddqsw" Sep 17 07:28:00.016586 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013094 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-host\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.016586 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013123 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsc6l\" (UniqueName: \"kubernetes.io/projected/6d54b36a-cd25-4e0e-8a1d-a0b1ce6de23e-kube-api-access-tsc6l\") pod \"node-resolver-ddqsw\" (UID: \"6d54b36a-cd25-4e0e-8a1d-a0b1ce6de23e\") " pod="openshift-dns/node-resolver-ddqsw" Sep 17 07:28:00.016586 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013132 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-host-kubelet\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.016586 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013152 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8af26520-8854-41c6-a082-58fd1e819408-tuning-conf-dir\") pod \"multus-additional-cni-plugins-spnx4\" (UID: \"8af26520-8854-41c6-a082-58fd1e819408\") " pod="openshift-multus/multus-additional-cni-plugins-spnx4" Sep 17 07:28:00.016586 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013194 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-host-slash\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.016586 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013220 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxb5g\" (UniqueName: \"kubernetes.io/projected/77d69f00-3692-47b0-845d-f07095a74966-kube-api-access-sxb5g\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.016586 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013246 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-etc-kubernetes\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.016586 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013275 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/76674b13-2b8e-49bf-b508-773f53a76a95-etc-tuned\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.016586 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013289 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-host-slash\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.016586 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013292 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8af26520-8854-41c6-a082-58fd1e819408-tuning-conf-dir\") pod \"multus-additional-cni-plugins-spnx4\" (UID: \"8af26520-8854-41c6-a082-58fd1e819408\") " pod="openshift-multus/multus-additional-cni-plugins-spnx4" Sep 17 07:28:00.016586 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013312 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5bea09db-e130-47db-83df-6467274459a1-agent-certs\") pod \"konnectivity-agent-crmdq\" (UID: \"5bea09db-e130-47db-83df-6467274459a1\") " pod="kube-system/konnectivity-agent-crmdq" Sep 17 07:28:00.016586 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013341 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-host\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.016586 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013367 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8af26520-8854-41c6-a082-58fd1e819408-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-spnx4\" (UID: \"8af26520-8854-41c6-a082-58fd1e819408\") " pod="openshift-multus/multus-additional-cni-plugins-spnx4" Sep 17 07:28:00.016586 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013370 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-etc-openvswitch\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.016586 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013405 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-etc-openvswitch\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.017138 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013424 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/251d0b36-1511-412f-8a1c-d599e9c134b6-iptables-alerter-script\") pod \"iptables-alerter-h5j5d\" (UID: \"251d0b36-1511-412f-8a1c-d599e9c134b6\") " pod="openshift-network-operator/iptables-alerter-h5j5d" Sep 17 07:28:00.017138 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013249 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-cni-binary-copy\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.017138 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013470 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-multus-cni-dir\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.017138 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013516 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-etc-kubernetes\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.017138 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013562 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-etc-kubernetes\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.017138 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013565 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-etc-sysconfig\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.017138 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013622 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-etc-kubernetes\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.017138 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013640 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-multus-cni-dir\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.017138 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013665 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77d69f00-3692-47b0-845d-f07095a74966-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.017138 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013743 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-multus-conf-dir\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.017138 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013816 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/76674b13-2b8e-49bf-b508-773f53a76a95-etc-sysconfig\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.017138 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.013982 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-multus-daemon-config\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.017138 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.014013 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/77d69f00-3692-47b0-845d-f07095a74966-ovnkube-script-lib\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.017138 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.014182 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/251d0b36-1511-412f-8a1c-d599e9c134b6-iptables-alerter-script\") pod \"iptables-alerter-h5j5d\" (UID: \"251d0b36-1511-412f-8a1c-d599e9c134b6\") " pod="openshift-network-operator/iptables-alerter-h5j5d" Sep 17 07:28:00.017138 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.015760 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/77d69f00-3692-47b0-845d-f07095a74966-ovn-node-metrics-cert\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.017138 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.015798 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/76674b13-2b8e-49bf-b508-773f53a76a95-tmp\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.017138 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.015931 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/76674b13-2b8e-49bf-b508-773f53a76a95-etc-tuned\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.017138 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.016378 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5bea09db-e130-47db-83df-6467274459a1-agent-certs\") pod \"konnectivity-agent-crmdq\" (UID: \"5bea09db-e130-47db-83df-6467274459a1\") " pod="kube-system/konnectivity-agent-crmdq" Sep 17 07:28:00.035946 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:00.035924 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:28:00.035946 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:00.035944 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:28:00.036081 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:00.035959 2553 projected.go:196] Error preparing data for projected volume kube-api-access-5vc7c for pod openshift-network-diagnostics/network-check-target-tsc9z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:00.036081 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:00.036012 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a96dd04-5687-4059-bf3f-ee1864ed8e40-kube-api-access-5vc7c podName:6a96dd04-5687-4059-bf3f-ee1864ed8e40 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:00.53599833 +0000 UTC m=+2.094183122 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5vc7c" (UniqueName: "kubernetes.io/projected/6a96dd04-5687-4059-bf3f-ee1864ed8e40-kube-api-access-5vc7c") pod "network-check-target-tsc9z" (UID: "6a96dd04-5687-4059-bf3f-ee1864ed8e40") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:00.038291 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.038261 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mljcs\" (UniqueName: \"kubernetes.io/projected/b388be8d-ade7-4e8d-b182-c27d6b5c5b1a-kube-api-access-mljcs\") pod \"node-ca-7l8hw\" (UID: \"b388be8d-ade7-4e8d-b182-c27d6b5c5b1a\") " pod="openshift-image-registry/node-ca-7l8hw" Sep 17 07:28:00.039416 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.039391 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4bjq\" (UniqueName: \"kubernetes.io/projected/76674b13-2b8e-49bf-b508-773f53a76a95-kube-api-access-j4bjq\") pod \"tuned-bb6f2\" (UID: \"76674b13-2b8e-49bf-b508-773f53a76a95\") " pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.039906 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.039878 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bfz5\" (UniqueName: \"kubernetes.io/projected/8af26520-8854-41c6-a082-58fd1e819408-kube-api-access-2bfz5\") pod \"multus-additional-cni-plugins-spnx4\" (UID: \"8af26520-8854-41c6-a082-58fd1e819408\") " pod="openshift-multus/multus-additional-cni-plugins-spnx4" Sep 17 07:28:00.043234 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.043211 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv79h\" (UniqueName: \"kubernetes.io/projected/251d0b36-1511-412f-8a1c-d599e9c134b6-kube-api-access-xv79h\") pod \"iptables-alerter-h5j5d\" (UID: \"251d0b36-1511-412f-8a1c-d599e9c134b6\") " pod="openshift-network-operator/iptables-alerter-h5j5d" Sep 17 07:28:00.043962 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.043916 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsc6l\" (UniqueName: \"kubernetes.io/projected/6d54b36a-cd25-4e0e-8a1d-a0b1ce6de23e-kube-api-access-tsc6l\") pod \"node-resolver-ddqsw\" (UID: \"6d54b36a-cd25-4e0e-8a1d-a0b1ce6de23e\") " pod="openshift-dns/node-resolver-ddqsw" Sep 17 07:28:00.044131 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.044107 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48ssr\" (UniqueName: \"kubernetes.io/projected/9ab08789-ddd8-47f2-af25-0e3ab923c5ff-kube-api-access-48ssr\") pod \"multus-rb9tt\" (UID: \"9ab08789-ddd8-47f2-af25-0e3ab923c5ff\") " pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.044178 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.044112 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llvf6\" (UniqueName: \"kubernetes.io/projected/c35e7d45-1111-40a8-acf5-666b633e8180-kube-api-access-llvf6\") pod \"network-metrics-daemon-xh2n2\" (UID: \"c35e7d45-1111-40a8-acf5-666b633e8180\") " pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:00.044541 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.044517 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tf5c\" (UniqueName: \"kubernetes.io/projected/f097781e-5016-483b-8e25-85ac5ef23767-kube-api-access-7tf5c\") pod \"aws-ebs-csi-driver-node-s6r7n\" (UID: \"f097781e-5016-483b-8e25-85ac5ef23767\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" Sep 17 07:28:00.044809 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.044791 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxb5g\" (UniqueName: \"kubernetes.io/projected/77d69f00-3692-47b0-845d-f07095a74966-kube-api-access-sxb5g\") pod \"ovnkube-node-zhw5d\" (UID: \"77d69f00-3692-47b0-845d-f07095a74966\") " pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.117815 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.117736 2553 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 07:28:00.139037 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.139006 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:00.148470 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:28:00.148439 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77d69f00_3692_47b0_845d_f07095a74966.slice/crio-23ba6a0fc00df1380ff8d8eb6fb4faaa7e20627534f9c022fabbd58aabebf06b WatchSource:0}: Error finding container 23ba6a0fc00df1380ff8d8eb6fb4faaa7e20627534f9c022fabbd58aabebf06b: Status 404 returned error can't find the container with id 23ba6a0fc00df1380ff8d8eb6fb4faaa7e20627534f9c022fabbd58aabebf06b Sep 17 07:28:00.158152 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.158128 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rb9tt" Sep 17 07:28:00.165135 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:28:00.165109 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ab08789_ddd8_47f2_af25_0e3ab923c5ff.slice/crio-a450e662cba2dfc71360c96e104dfaafa9bfa4299e09f51ed0ae122b6476458c WatchSource:0}: Error finding container a450e662cba2dfc71360c96e104dfaafa9bfa4299e09f51ed0ae122b6476458c: Status 404 returned error can't find the container with id a450e662cba2dfc71360c96e104dfaafa9bfa4299e09f51ed0ae122b6476458c Sep 17 07:28:00.172060 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.172039 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" Sep 17 07:28:00.179265 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:28:00.179247 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76674b13_2b8e_49bf_b508_773f53a76a95.slice/crio-d64e622a7da3e9f9d36af7b70888bfae0136197663204c00b587a716dec1e1d9 WatchSource:0}: Error finding container d64e622a7da3e9f9d36af7b70888bfae0136197663204c00b587a716dec1e1d9: Status 404 returned error can't find the container with id d64e622a7da3e9f9d36af7b70888bfae0136197663204c00b587a716dec1e1d9 Sep 17 07:28:00.192347 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.192327 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ddqsw" Sep 17 07:28:00.198519 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.198330 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" Sep 17 07:28:00.199936 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:28:00.199911 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d54b36a_cd25_4e0e_8a1d_a0b1ce6de23e.slice/crio-630307c0d7acea1ff284dd8b756298664030c1dfe3a74398d3436a078cf34bcf WatchSource:0}: Error finding container 630307c0d7acea1ff284dd8b756298664030c1dfe3a74398d3436a078cf34bcf: Status 404 returned error can't find the container with id 630307c0d7acea1ff284dd8b756298664030c1dfe3a74398d3436a078cf34bcf Sep 17 07:28:00.206531 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:28:00.206507 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf097781e_5016_483b_8e25_85ac5ef23767.slice/crio-638ecbde52b3b3b55f575fe8e6b22e93ad793bfe36529ebb264fbfd7c9d6a3d0 WatchSource:0}: Error finding container 638ecbde52b3b3b55f575fe8e6b22e93ad793bfe36529ebb264fbfd7c9d6a3d0: Status 404 returned error can't find the container with id 638ecbde52b3b3b55f575fe8e6b22e93ad793bfe36529ebb264fbfd7c9d6a3d0 Sep 17 07:28:00.216062 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.216040 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-spnx4" Sep 17 07:28:00.224101 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:28:00.224073 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8af26520_8854_41c6_a082_58fd1e819408.slice/crio-05756604b5527d4d1e6aee52fef6c164fe081d326c84f325f4b0b38433f961bc WatchSource:0}: Error finding container 05756604b5527d4d1e6aee52fef6c164fe081d326c84f325f4b0b38433f961bc: Status 404 returned error can't find the container with id 05756604b5527d4d1e6aee52fef6c164fe081d326c84f325f4b0b38433f961bc Sep 17 07:28:00.228421 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.228401 2553 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 07:28:00.233823 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.233806 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h5j5d" Sep 17 07:28:00.241644 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:28:00.241622 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod251d0b36_1511_412f_8a1c_d599e9c134b6.slice/crio-daa854e2ca20cd37b2b8cd5718d4746c0cce2a5c77cdc15ab2df2dc55a0ba12f WatchSource:0}: Error finding container daa854e2ca20cd37b2b8cd5718d4746c0cce2a5c77cdc15ab2df2dc55a0ba12f: Status 404 returned error can't find the container with id daa854e2ca20cd37b2b8cd5718d4746c0cce2a5c77cdc15ab2df2dc55a0ba12f Sep 17 07:28:00.258767 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.258735 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-crmdq" Sep 17 07:28:00.264604 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.264574 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7l8hw" Sep 17 07:28:00.267542 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:28:00.266171 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bea09db_e130_47db_83df_6467274459a1.slice/crio-6160ad71da9f991b40b8f195b655ac352882c087f54cb015dd1f0804da622124 WatchSource:0}: Error finding container 6160ad71da9f991b40b8f195b655ac352882c087f54cb015dd1f0804da622124: Status 404 returned error can't find the container with id 6160ad71da9f991b40b8f195b655ac352882c087f54cb015dd1f0804da622124 Sep 17 07:28:00.274359 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:28:00.274335 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb388be8d_ade7_4e8d_b182_c27d6b5c5b1a.slice/crio-e7609847235c4d85f2f07417606f26335167ecddc79405951e70f9a4c8c9da6c WatchSource:0}: Error finding container e7609847235c4d85f2f07417606f26335167ecddc79405951e70f9a4c8c9da6c: Status 404 returned error can't find the container with id e7609847235c4d85f2f07417606f26335167ecddc79405951e70f9a4c8c9da6c Sep 17 07:28:00.517021 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.516984 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f68f4c1a-7946-43c1-80ca-b472129b2bfe-original-pull-secret\") pod \"global-pull-secret-syncer-gh9zp\" (UID: \"f68f4c1a-7946-43c1-80ca-b472129b2bfe\") " pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:00.517219 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.517040 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c35e7d45-1111-40a8-acf5-666b633e8180-metrics-certs\") pod \"network-metrics-daemon-xh2n2\" (UID: \"c35e7d45-1111-40a8-acf5-666b633e8180\") " pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:00.517219 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:00.517208 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:00.517341 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:00.517269 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c35e7d45-1111-40a8-acf5-666b633e8180-metrics-certs podName:c35e7d45-1111-40a8-acf5-666b633e8180 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:01.51724913 +0000 UTC m=+3.075433939 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c35e7d45-1111-40a8-acf5-666b633e8180-metrics-certs") pod "network-metrics-daemon-xh2n2" (UID: "c35e7d45-1111-40a8-acf5-666b633e8180") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:00.517408 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:00.517341 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:00.517408 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:00.517387 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f68f4c1a-7946-43c1-80ca-b472129b2bfe-original-pull-secret podName:f68f4c1a-7946-43c1-80ca-b472129b2bfe nodeName:}" failed. No retries permitted until 2026-09-17 07:28:01.517372807 +0000 UTC m=+3.075557600 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f68f4c1a-7946-43c1-80ca-b472129b2bfe-original-pull-secret") pod "global-pull-secret-syncer-gh9zp" (UID: "f68f4c1a-7946-43c1-80ca-b472129b2bfe") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:00.618735 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.618698 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vc7c\" (UniqueName: \"kubernetes.io/projected/6a96dd04-5687-4059-bf3f-ee1864ed8e40-kube-api-access-5vc7c\") pod \"network-check-target-tsc9z\" (UID: \"6a96dd04-5687-4059-bf3f-ee1864ed8e40\") " pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:28:00.618964 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:00.618852 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:28:00.618964 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:00.618875 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:28:00.618964 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:00.618888 2553 projected.go:196] Error preparing data for projected volume kube-api-access-5vc7c for pod openshift-network-diagnostics/network-check-target-tsc9z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:00.619138 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:00.618970 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a96dd04-5687-4059-bf3f-ee1864ed8e40-kube-api-access-5vc7c podName:6a96dd04-5687-4059-bf3f-ee1864ed8e40 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:01.618928356 +0000 UTC m=+3.177113151 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5vc7c" (UniqueName: "kubernetes.io/projected/6a96dd04-5687-4059-bf3f-ee1864ed8e40-kube-api-access-5vc7c") pod "network-check-target-tsc9z" (UID: "6a96dd04-5687-4059-bf3f-ee1864ed8e40") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:00.867521 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.867045 2553 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 07:28:00.879915 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.879855 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 07:23:00 +0000 UTC" deadline="2028-05-19 16:31:41.214355577 +0000 UTC" Sep 17 07:28:00.879915 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.879913 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14649h3m40.334447326s" Sep 17 07:28:00.931490 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.931450 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:00.931665 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:00.931604 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2n2" podUID="c35e7d45-1111-40a8-acf5-666b633e8180" Sep 17 07:28:00.951142 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.951022 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-crmdq" event={"ID":"5bea09db-e130-47db-83df-6467274459a1","Type":"ContainerStarted","Data":"6160ad71da9f991b40b8f195b655ac352882c087f54cb015dd1f0804da622124"} Sep 17 07:28:00.964403 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.964362 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h5j5d" event={"ID":"251d0b36-1511-412f-8a1c-d599e9c134b6","Type":"ContainerStarted","Data":"daa854e2ca20cd37b2b8cd5718d4746c0cce2a5c77cdc15ab2df2dc55a0ba12f"} Sep 17 07:28:00.980779 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.980730 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" event={"ID":"76674b13-2b8e-49bf-b508-773f53a76a95","Type":"ContainerStarted","Data":"d64e622a7da3e9f9d36af7b70888bfae0136197663204c00b587a716dec1e1d9"} Sep 17 07:28:00.994477 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:00.994347 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rb9tt" event={"ID":"9ab08789-ddd8-47f2-af25-0e3ab923c5ff","Type":"ContainerStarted","Data":"a450e662cba2dfc71360c96e104dfaafa9bfa4299e09f51ed0ae122b6476458c"} Sep 17 07:28:01.006630 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:01.006523 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-54.ec2.internal" event={"ID":"7bf34cff4e5d7631f28563807f0551e5","Type":"ContainerStarted","Data":"481032058bd993c0440c1959bf84a94fd749a296939a5f80ac794dc799e924a0"} Sep 17 07:28:01.029922 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:01.029809 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7l8hw" event={"ID":"b388be8d-ade7-4e8d-b182-c27d6b5c5b1a","Type":"ContainerStarted","Data":"e7609847235c4d85f2f07417606f26335167ecddc79405951e70f9a4c8c9da6c"} Sep 17 07:28:01.044679 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:01.044639 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-spnx4" event={"ID":"8af26520-8854-41c6-a082-58fd1e819408","Type":"ContainerStarted","Data":"05756604b5527d4d1e6aee52fef6c164fe081d326c84f325f4b0b38433f961bc"} Sep 17 07:28:01.067533 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:01.067490 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" event={"ID":"f097781e-5016-483b-8e25-85ac5ef23767","Type":"ContainerStarted","Data":"638ecbde52b3b3b55f575fe8e6b22e93ad793bfe36529ebb264fbfd7c9d6a3d0"} Sep 17 07:28:01.078198 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:01.078160 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ddqsw" event={"ID":"6d54b36a-cd25-4e0e-8a1d-a0b1ce6de23e","Type":"ContainerStarted","Data":"630307c0d7acea1ff284dd8b756298664030c1dfe3a74398d3436a078cf34bcf"} Sep 17 07:28:01.098706 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:01.098655 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" event={"ID":"77d69f00-3692-47b0-845d-f07095a74966","Type":"ContainerStarted","Data":"23ba6a0fc00df1380ff8d8eb6fb4faaa7e20627534f9c022fabbd58aabebf06b"} Sep 17 07:28:01.126434 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:01.126282 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-54.ec2.internal" event={"ID":"ea85235c507e0a22eb3b6e21763019df","Type":"ContainerStarted","Data":"fbc11803b42a736d3294c8caf5db8591891d928d585114004dcb62e289b6a2aa"} Sep 17 07:28:01.528933 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:01.528876 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f68f4c1a-7946-43c1-80ca-b472129b2bfe-original-pull-secret\") pod \"global-pull-secret-syncer-gh9zp\" (UID: \"f68f4c1a-7946-43c1-80ca-b472129b2bfe\") " pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:01.529157 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:01.528954 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c35e7d45-1111-40a8-acf5-666b633e8180-metrics-certs\") pod \"network-metrics-daemon-xh2n2\" (UID: \"c35e7d45-1111-40a8-acf5-666b633e8180\") " pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:01.529157 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:01.529113 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:01.529291 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:01.529182 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c35e7d45-1111-40a8-acf5-666b633e8180-metrics-certs podName:c35e7d45-1111-40a8-acf5-666b633e8180 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:03.529160854 +0000 UTC m=+5.087345647 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c35e7d45-1111-40a8-acf5-666b633e8180-metrics-certs") pod "network-metrics-daemon-xh2n2" (UID: "c35e7d45-1111-40a8-acf5-666b633e8180") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:01.529660 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:01.529636 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:01.529747 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:01.529720 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f68f4c1a-7946-43c1-80ca-b472129b2bfe-original-pull-secret podName:f68f4c1a-7946-43c1-80ca-b472129b2bfe nodeName:}" failed. No retries permitted until 2026-09-17 07:28:03.529677796 +0000 UTC m=+5.087862610 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f68f4c1a-7946-43c1-80ca-b472129b2bfe-original-pull-secret") pod "global-pull-secret-syncer-gh9zp" (UID: "f68f4c1a-7946-43c1-80ca-b472129b2bfe") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:01.629337 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:01.629294 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vc7c\" (UniqueName: \"kubernetes.io/projected/6a96dd04-5687-4059-bf3f-ee1864ed8e40-kube-api-access-5vc7c\") pod \"network-check-target-tsc9z\" (UID: \"6a96dd04-5687-4059-bf3f-ee1864ed8e40\") " pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:28:01.629560 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:01.629540 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:28:01.629645 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:01.629573 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:28:01.629645 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:01.629589 2553 projected.go:196] Error preparing data for projected volume kube-api-access-5vc7c for pod openshift-network-diagnostics/network-check-target-tsc9z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:01.629777 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:01.629654 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a96dd04-5687-4059-bf3f-ee1864ed8e40-kube-api-access-5vc7c podName:6a96dd04-5687-4059-bf3f-ee1864ed8e40 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:03.62963275 +0000 UTC m=+5.187817559 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5vc7c" (UniqueName: "kubernetes.io/projected/6a96dd04-5687-4059-bf3f-ee1864ed8e40-kube-api-access-5vc7c") pod "network-check-target-tsc9z" (UID: "6a96dd04-5687-4059-bf3f-ee1864ed8e40") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:01.880476 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:01.880349 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 07:23:00 +0000 UTC" deadline="2028-04-04 01:34:15.428290179 +0000 UTC" Sep 17 07:28:01.880476 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:01.880421 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13554h6m13.547873847s" Sep 17 07:28:01.928584 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:01.928550 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:01.928764 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:01.928679 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gh9zp" podUID="f68f4c1a-7946-43c1-80ca-b472129b2bfe" Sep 17 07:28:01.929122 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:01.929102 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:28:01.929254 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:01.929214 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsc9z" podUID="6a96dd04-5687-4059-bf3f-ee1864ed8e40" Sep 17 07:28:02.731414 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:02.731170 2553 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 07:28:02.931115 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:02.931085 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:02.931601 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:02.931238 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2n2" podUID="c35e7d45-1111-40a8-acf5-666b633e8180" Sep 17 07:28:03.548849 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:03.548695 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f68f4c1a-7946-43c1-80ca-b472129b2bfe-original-pull-secret\") pod \"global-pull-secret-syncer-gh9zp\" (UID: \"f68f4c1a-7946-43c1-80ca-b472129b2bfe\") " pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:03.548849 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:03.548756 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c35e7d45-1111-40a8-acf5-666b633e8180-metrics-certs\") pod \"network-metrics-daemon-xh2n2\" (UID: \"c35e7d45-1111-40a8-acf5-666b633e8180\") " pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:03.549158 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:03.548892 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:03.549158 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:03.548976 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c35e7d45-1111-40a8-acf5-666b633e8180-metrics-certs podName:c35e7d45-1111-40a8-acf5-666b633e8180 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:07.548956143 +0000 UTC m=+9.107140938 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c35e7d45-1111-40a8-acf5-666b633e8180-metrics-certs") pod "network-metrics-daemon-xh2n2" (UID: "c35e7d45-1111-40a8-acf5-666b633e8180") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:03.549422 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:03.549402 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:03.549492 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:03.549460 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f68f4c1a-7946-43c1-80ca-b472129b2bfe-original-pull-secret podName:f68f4c1a-7946-43c1-80ca-b472129b2bfe nodeName:}" failed. No retries permitted until 2026-09-17 07:28:07.549444804 +0000 UTC m=+9.107629598 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f68f4c1a-7946-43c1-80ca-b472129b2bfe-original-pull-secret") pod "global-pull-secret-syncer-gh9zp" (UID: "f68f4c1a-7946-43c1-80ca-b472129b2bfe") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:03.649180 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:03.649127 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vc7c\" (UniqueName: \"kubernetes.io/projected/6a96dd04-5687-4059-bf3f-ee1864ed8e40-kube-api-access-5vc7c\") pod \"network-check-target-tsc9z\" (UID: \"6a96dd04-5687-4059-bf3f-ee1864ed8e40\") " pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:28:03.649375 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:03.649347 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:28:03.649375 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:03.649367 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:28:03.649499 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:03.649389 2553 projected.go:196] Error preparing data for projected volume kube-api-access-5vc7c for pod openshift-network-diagnostics/network-check-target-tsc9z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:03.649499 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:03.649453 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a96dd04-5687-4059-bf3f-ee1864ed8e40-kube-api-access-5vc7c podName:6a96dd04-5687-4059-bf3f-ee1864ed8e40 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:07.649433476 +0000 UTC m=+9.207618283 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5vc7c" (UniqueName: "kubernetes.io/projected/6a96dd04-5687-4059-bf3f-ee1864ed8e40-kube-api-access-5vc7c") pod "network-check-target-tsc9z" (UID: "6a96dd04-5687-4059-bf3f-ee1864ed8e40") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:03.928543 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:03.928449 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:28:03.928723 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:03.928608 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsc9z" podUID="6a96dd04-5687-4059-bf3f-ee1864ed8e40" Sep 17 07:28:03.928723 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:03.928655 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:03.928857 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:03.928773 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gh9zp" podUID="f68f4c1a-7946-43c1-80ca-b472129b2bfe" Sep 17 07:28:04.929708 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:04.929167 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:04.929708 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:04.929336 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2n2" podUID="c35e7d45-1111-40a8-acf5-666b633e8180" Sep 17 07:28:05.929107 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:05.928442 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:28:05.929107 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:05.928613 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsc9z" podUID="6a96dd04-5687-4059-bf3f-ee1864ed8e40" Sep 17 07:28:05.929107 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:05.929013 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:05.929424 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:05.929124 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gh9zp" podUID="f68f4c1a-7946-43c1-80ca-b472129b2bfe" Sep 17 07:28:06.929342 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:06.929139 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:06.929342 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:06.929309 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2n2" podUID="c35e7d45-1111-40a8-acf5-666b633e8180" Sep 17 07:28:07.588331 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:07.588286 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f68f4c1a-7946-43c1-80ca-b472129b2bfe-original-pull-secret\") pod \"global-pull-secret-syncer-gh9zp\" (UID: \"f68f4c1a-7946-43c1-80ca-b472129b2bfe\") " pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:07.588521 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:07.588346 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c35e7d45-1111-40a8-acf5-666b633e8180-metrics-certs\") pod \"network-metrics-daemon-xh2n2\" (UID: \"c35e7d45-1111-40a8-acf5-666b633e8180\") " pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:07.588521 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:07.588505 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:07.588667 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:07.588537 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:07.588667 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:07.588598 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f68f4c1a-7946-43c1-80ca-b472129b2bfe-original-pull-secret podName:f68f4c1a-7946-43c1-80ca-b472129b2bfe nodeName:}" failed. No retries permitted until 2026-09-17 07:28:15.588574611 +0000 UTC m=+17.146759405 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f68f4c1a-7946-43c1-80ca-b472129b2bfe-original-pull-secret") pod "global-pull-secret-syncer-gh9zp" (UID: "f68f4c1a-7946-43c1-80ca-b472129b2bfe") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:07.588667 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:07.588622 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c35e7d45-1111-40a8-acf5-666b633e8180-metrics-certs podName:c35e7d45-1111-40a8-acf5-666b633e8180 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:15.58860948 +0000 UTC m=+17.146794269 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c35e7d45-1111-40a8-acf5-666b633e8180-metrics-certs") pod "network-metrics-daemon-xh2n2" (UID: "c35e7d45-1111-40a8-acf5-666b633e8180") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:07.689535 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:07.689440 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vc7c\" (UniqueName: \"kubernetes.io/projected/6a96dd04-5687-4059-bf3f-ee1864ed8e40-kube-api-access-5vc7c\") pod \"network-check-target-tsc9z\" (UID: \"6a96dd04-5687-4059-bf3f-ee1864ed8e40\") " pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:28:07.689737 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:07.689689 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:28:07.689737 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:07.689717 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:28:07.689737 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:07.689733 2553 projected.go:196] Error preparing data for projected volume kube-api-access-5vc7c for pod openshift-network-diagnostics/network-check-target-tsc9z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:07.689900 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:07.689807 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a96dd04-5687-4059-bf3f-ee1864ed8e40-kube-api-access-5vc7c podName:6a96dd04-5687-4059-bf3f-ee1864ed8e40 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:15.68978629 +0000 UTC m=+17.247971086 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5vc7c" (UniqueName: "kubernetes.io/projected/6a96dd04-5687-4059-bf3f-ee1864ed8e40-kube-api-access-5vc7c") pod "network-check-target-tsc9z" (UID: "6a96dd04-5687-4059-bf3f-ee1864ed8e40") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:07.929130 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:07.929036 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:28:07.929130 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:07.929084 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:07.929358 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:07.929185 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsc9z" podUID="6a96dd04-5687-4059-bf3f-ee1864ed8e40" Sep 17 07:28:07.929358 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:07.929275 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gh9zp" podUID="f68f4c1a-7946-43c1-80ca-b472129b2bfe" Sep 17 07:28:08.930300 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:08.930268 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:08.930859 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:08.930401 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2n2" podUID="c35e7d45-1111-40a8-acf5-666b633e8180" Sep 17 07:28:09.928395 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:09.928348 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:28:09.928627 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:09.928484 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsc9z" podUID="6a96dd04-5687-4059-bf3f-ee1864ed8e40" Sep 17 07:28:09.928627 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:09.928567 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:09.928748 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:09.928642 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gh9zp" podUID="f68f4c1a-7946-43c1-80ca-b472129b2bfe" Sep 17 07:28:10.928628 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:10.928596 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:10.929138 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:10.928759 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2n2" podUID="c35e7d45-1111-40a8-acf5-666b633e8180" Sep 17 07:28:11.928365 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:11.928333 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:11.928365 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:11.928362 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:28:11.928572 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:11.928453 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gh9zp" podUID="f68f4c1a-7946-43c1-80ca-b472129b2bfe" Sep 17 07:28:11.928639 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:11.928571 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsc9z" podUID="6a96dd04-5687-4059-bf3f-ee1864ed8e40" Sep 17 07:28:12.928938 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:12.928900 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:12.929344 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:12.929047 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2n2" podUID="c35e7d45-1111-40a8-acf5-666b633e8180" Sep 17 07:28:13.929049 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:13.929013 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:28:13.929049 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:13.929051 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:13.929550 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:13.929138 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsc9z" podUID="6a96dd04-5687-4059-bf3f-ee1864ed8e40" Sep 17 07:28:13.929550 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:13.929200 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gh9zp" podUID="f68f4c1a-7946-43c1-80ca-b472129b2bfe" Sep 17 07:28:14.928546 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:14.928515 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:14.928743 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:14.928660 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2n2" podUID="c35e7d45-1111-40a8-acf5-666b633e8180" Sep 17 07:28:15.649629 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:15.649591 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f68f4c1a-7946-43c1-80ca-b472129b2bfe-original-pull-secret\") pod \"global-pull-secret-syncer-gh9zp\" (UID: \"f68f4c1a-7946-43c1-80ca-b472129b2bfe\") " pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:15.649629 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:15.649632 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c35e7d45-1111-40a8-acf5-666b633e8180-metrics-certs\") pod \"network-metrics-daemon-xh2n2\" (UID: \"c35e7d45-1111-40a8-acf5-666b633e8180\") " pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:15.650055 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:15.649737 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:15.650055 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:15.649748 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:15.650055 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:15.649790 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c35e7d45-1111-40a8-acf5-666b633e8180-metrics-certs podName:c35e7d45-1111-40a8-acf5-666b633e8180 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:31.649776939 +0000 UTC m=+33.207961729 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c35e7d45-1111-40a8-acf5-666b633e8180-metrics-certs") pod "network-metrics-daemon-xh2n2" (UID: "c35e7d45-1111-40a8-acf5-666b633e8180") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:15.650055 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:15.649804 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f68f4c1a-7946-43c1-80ca-b472129b2bfe-original-pull-secret podName:f68f4c1a-7946-43c1-80ca-b472129b2bfe nodeName:}" failed. No retries permitted until 2026-09-17 07:28:31.649796744 +0000 UTC m=+33.207981529 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f68f4c1a-7946-43c1-80ca-b472129b2bfe-original-pull-secret") pod "global-pull-secret-syncer-gh9zp" (UID: "f68f4c1a-7946-43c1-80ca-b472129b2bfe") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:15.750590 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:15.750558 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vc7c\" (UniqueName: \"kubernetes.io/projected/6a96dd04-5687-4059-bf3f-ee1864ed8e40-kube-api-access-5vc7c\") pod \"network-check-target-tsc9z\" (UID: \"6a96dd04-5687-4059-bf3f-ee1864ed8e40\") " pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:28:15.750731 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:15.750678 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:28:15.750731 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:15.750695 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:28:15.750731 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:15.750705 2553 projected.go:196] Error preparing data for projected volume kube-api-access-5vc7c for pod openshift-network-diagnostics/network-check-target-tsc9z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:15.750840 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:15.750760 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a96dd04-5687-4059-bf3f-ee1864ed8e40-kube-api-access-5vc7c podName:6a96dd04-5687-4059-bf3f-ee1864ed8e40 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:31.750739344 +0000 UTC m=+33.308924130 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5vc7c" (UniqueName: "kubernetes.io/projected/6a96dd04-5687-4059-bf3f-ee1864ed8e40-kube-api-access-5vc7c") pod "network-check-target-tsc9z" (UID: "6a96dd04-5687-4059-bf3f-ee1864ed8e40") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:15.929383 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:15.929299 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:28:15.929383 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:15.929330 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:15.929585 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:15.929406 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsc9z" podUID="6a96dd04-5687-4059-bf3f-ee1864ed8e40" Sep 17 07:28:15.929585 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:15.929523 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gh9zp" podUID="f68f4c1a-7946-43c1-80ca-b472129b2bfe" Sep 17 07:28:16.928771 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:16.928737 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:16.929354 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:16.928873 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2n2" podUID="c35e7d45-1111-40a8-acf5-666b633e8180" Sep 17 07:28:17.928900 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:17.928832 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:28:17.929371 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:17.928978 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsc9z" podUID="6a96dd04-5687-4059-bf3f-ee1864ed8e40" Sep 17 07:28:17.929371 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:17.929037 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:17.929371 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:17.929161 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gh9zp" podUID="f68f4c1a-7946-43c1-80ca-b472129b2bfe" Sep 17 07:28:18.929820 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:18.929784 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:18.930278 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:18.929926 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2n2" podUID="c35e7d45-1111-40a8-acf5-666b633e8180" Sep 17 07:28:19.928887 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:19.928709 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:19.928993 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:19.928748 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:28:19.929050 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:19.928993 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gh9zp" podUID="f68f4c1a-7946-43c1-80ca-b472129b2bfe" Sep 17 07:28:19.929102 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:19.929039 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsc9z" podUID="6a96dd04-5687-4059-bf3f-ee1864ed8e40" Sep 17 07:28:20.176164 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:20.175845 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/ovn-acl-logging/0.log" Sep 17 07:28:20.176765 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:20.176608 2553 generic.go:356] "Generic (PLEG): container finished" podID="77d69f00-3692-47b0-845d-f07095a74966" containerID="0c0a50dba9f7b21f36dbd82647f4e6a1e90f4230c7f480ceeab4ff2c37fef679" exitCode=1 Sep 17 07:28:20.176765 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:20.176688 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" event={"ID":"77d69f00-3692-47b0-845d-f07095a74966","Type":"ContainerStarted","Data":"86fb14df9c1ede4d3a227fd482a52234bbbddef0fefedaf27b6f5fdbfc5fb2c1"} Sep 17 07:28:20.176871 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:20.176766 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" event={"ID":"77d69f00-3692-47b0-845d-f07095a74966","Type":"ContainerStarted","Data":"edc3a8f6db12676db6ab0e414108d4fadf9b201b3c5ea7256ca7fe306a331964"} Sep 17 07:28:20.176871 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:20.176829 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" event={"ID":"77d69f00-3692-47b0-845d-f07095a74966","Type":"ContainerDied","Data":"0c0a50dba9f7b21f36dbd82647f4e6a1e90f4230c7f480ceeab4ff2c37fef679"} Sep 17 07:28:20.177000 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:20.176979 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" event={"ID":"77d69f00-3692-47b0-845d-f07095a74966","Type":"ContainerStarted","Data":"83f9b78c4f355e88dd715e21da2a95b7ec39f97689df43d85e02cb3a9d484d6a"} Sep 17 07:28:20.178733 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:20.178711 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-54.ec2.internal" event={"ID":"ea85235c507e0a22eb3b6e21763019df","Type":"ContainerStarted","Data":"cb7e6b83d1ff7e25c1ad66f6b7607604408985460f1095a507a6bcabb5b0fd9f"} Sep 17 07:28:20.180354 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:20.180336 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" event={"ID":"76674b13-2b8e-49bf-b508-773f53a76a95","Type":"ContainerStarted","Data":"bc55560e8dc5eefa3ef1658a2d1d382e89b80351f351e8ee5ee091873fd990ef"} Sep 17 07:28:20.182790 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:20.182769 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rb9tt" event={"ID":"9ab08789-ddd8-47f2-af25-0e3ab923c5ff","Type":"ContainerStarted","Data":"8a8cd5975aea398128aa377e4ba0333970b864a749719b1b3e7dcc516f722544"} Sep 17 07:28:20.199127 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:20.198411 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-54.ec2.internal" podStartSLOduration=20.1983952 podStartE2EDuration="20.1983952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:28:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:20.197693112 +0000 UTC m=+21.755877945" watchObservedRunningTime="2026-09-17 07:28:20.1983952 +0000 UTC m=+21.756580008" Sep 17 07:28:20.226991 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:20.226921 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rb9tt" podStartSLOduration=1.498978044 podStartE2EDuration="21.226868997s" totalImagesPullingTime="19.727890953s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:27:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:20.22656123 +0000 UTC m=+21.784746041" watchObservedRunningTime="2026-09-17 07:28:20.226868997 +0000 UTC m=+21.785053805" Sep 17 07:28:20.270520 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:20.270459 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bb6f2" podStartSLOduration=1.770410421 podStartE2EDuration="21.270429582s" totalImagesPullingTime="19.500019161s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:27:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:20.269096872 +0000 UTC m=+21.827281674" watchObservedRunningTime="2026-09-17 07:28:20.270429582 +0000 UTC m=+21.828614391" Sep 17 07:28:20.928848 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:20.928822 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:20.929028 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:20.928944 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2n2" podUID="c35e7d45-1111-40a8-acf5-666b633e8180" Sep 17 07:28:21.176374 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.176350 2553 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 17 07:28:21.186251 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.186229 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7l8hw" event={"ID":"b388be8d-ade7-4e8d-b182-c27d6b5c5b1a","Type":"ContainerStarted","Data":"67b4a614f71e6fb05f264a1d64f0b1ec33b09d985e59e9085fd371a3a33d2c3a"} Sep 17 07:28:21.187743 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.187720 2553 generic.go:356] "Generic (PLEG): container finished" podID="8af26520-8854-41c6-a082-58fd1e819408" containerID="467a78cc3a18c0a8edfc8ba33d2604b1e06d8435c5ca8af8a58ab98bc50c3dd6" exitCode=0 Sep 17 07:28:21.187827 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.187796 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-spnx4" event={"ID":"8af26520-8854-41c6-a082-58fd1e819408","Type":"ContainerDied","Data":"467a78cc3a18c0a8edfc8ba33d2604b1e06d8435c5ca8af8a58ab98bc50c3dd6"} Sep 17 07:28:21.189749 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.189720 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" event={"ID":"f097781e-5016-483b-8e25-85ac5ef23767","Type":"ContainerStarted","Data":"b40b2a5ab465742ea9375165dc970af5c496efb148cfd67eba5053ccfeffd893"} Sep 17 07:28:21.189838 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.189773 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" event={"ID":"f097781e-5016-483b-8e25-85ac5ef23767","Type":"ContainerStarted","Data":"8e0c72023857d29532cfde84166f159b623e57342d55465da3d767d9bfd82429"} Sep 17 07:28:21.191178 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.191150 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ddqsw" event={"ID":"6d54b36a-cd25-4e0e-8a1d-a0b1ce6de23e","Type":"ContainerStarted","Data":"1c8ee0c1680944767dc27ca00beb01692154a774426dc443c7a60d188e832ccc"} Sep 17 07:28:21.194355 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.194340 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/ovn-acl-logging/0.log" Sep 17 07:28:21.194708 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.194690 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" event={"ID":"77d69f00-3692-47b0-845d-f07095a74966","Type":"ContainerStarted","Data":"4ae761f66fb57ed571c19db90e9c9682bfe62e9822c6f6e371dca2b5cd5984bc"} Sep 17 07:28:21.194761 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.194753 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" event={"ID":"77d69f00-3692-47b0-845d-f07095a74966","Type":"ContainerStarted","Data":"572671e2c36476ff8df2ee4bb2ef8f739eb7d03daeb6abdd57c6c67d78237cd8"} Sep 17 07:28:21.195984 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.195958 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-crmdq" event={"ID":"5bea09db-e130-47db-83df-6467274459a1","Type":"ContainerStarted","Data":"2d72f2d7cfe294b6479c960985ebf7f91bd75cb66cc0301a234f3c8507c0a8c3"} Sep 17 07:28:21.197258 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.197236 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h5j5d" event={"ID":"251d0b36-1511-412f-8a1c-d599e9c134b6","Type":"ContainerStarted","Data":"d0a23febe2844ada412bb3c84a8979d16b21c351e3d548dab2d7cd8446431a32"} Sep 17 07:28:21.198615 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.198597 2553 generic.go:356] "Generic (PLEG): container finished" podID="7bf34cff4e5d7631f28563807f0551e5" containerID="10e1b2741723a1479e73816feab9c9b2016dbb8977da105bdbe50b00d1da22b8" exitCode=0 Sep 17 07:28:21.198705 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.198685 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-54.ec2.internal" event={"ID":"7bf34cff4e5d7631f28563807f0551e5","Type":"ContainerDied","Data":"10e1b2741723a1479e73816feab9c9b2016dbb8977da105bdbe50b00d1da22b8"} Sep 17 07:28:21.228416 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.228370 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7l8hw" podStartSLOduration=1.823325319 podStartE2EDuration="21.228353591s" totalImagesPullingTime="19.405028272s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:28:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:21.209982957 +0000 UTC m=+22.768167896" watchObservedRunningTime="2026-09-17 07:28:21.228353591 +0000 UTC m=+22.786538403" Sep 17 07:28:21.249684 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.249650 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-h5j5d" podStartSLOduration=1.824100993 podStartE2EDuration="21.249638462s" totalImagesPullingTime="19.425537469s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:28:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:21.227998865 +0000 UTC m=+22.786183685" watchObservedRunningTime="2026-09-17 07:28:21.249638462 +0000 UTC m=+22.807823271" Sep 17 07:28:21.295258 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.295234 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-crmdq" Sep 17 07:28:21.295916 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.295900 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-crmdq" Sep 17 07:28:21.314484 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.314445 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ddqsw" podStartSLOduration=2.835223476 podStartE2EDuration="22.314432777s" totalImagesPullingTime="19.479209301s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:27:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:21.279899974 +0000 UTC m=+22.838084781" watchObservedRunningTime="2026-09-17 07:28:21.314432777 +0000 UTC m=+22.872617586" Sep 17 07:28:21.350241 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.350174 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-crmdq" podStartSLOduration=1.937958327 podStartE2EDuration="21.350161442s" totalImagesPullingTime="19.412203115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:28:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:21.348815357 +0000 UTC m=+22.907000162" watchObservedRunningTime="2026-09-17 07:28:21.350161442 +0000 UTC m=+22.908346250" Sep 17 07:28:21.865687 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.865577 2553 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T07:28:21.176369087Z","UUID":"fe28c3a7-67ff-433c-9876-e059367f708f","Handler":null,"Name":"","Endpoint":""} Sep 17 07:28:21.867774 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.867741 2553 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 17 07:28:21.867937 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.867785 2553 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 17 07:28:21.928980 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.928863 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:28:21.928980 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:21.928891 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:21.929153 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:21.928981 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsc9z" podUID="6a96dd04-5687-4059-bf3f-ee1864ed8e40" Sep 17 07:28:21.929153 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:21.929090 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gh9zp" podUID="f68f4c1a-7946-43c1-80ca-b472129b2bfe" Sep 17 07:28:22.203391 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:22.203203 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" event={"ID":"f097781e-5016-483b-8e25-85ac5ef23767","Type":"ContainerStarted","Data":"43f720f8b09cf5fab2c8ba5b30544790efc9f8da1b41c05f3ca6c8e97f69698b"} Sep 17 07:28:22.204932 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:22.204905 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-54.ec2.internal" event={"ID":"7bf34cff4e5d7631f28563807f0551e5","Type":"ContainerStarted","Data":"ba215aabe3ea75f800b8b7bc42aada910304ae1a24576e109b126f727c6d4f18"} Sep 17 07:28:22.205178 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:22.205153 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-crmdq" Sep 17 07:28:22.205550 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:22.205535 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-crmdq" Sep 17 07:28:22.235052 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:22.235000 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s6r7n" podStartSLOduration=2.174171842 podStartE2EDuration="23.234984701s" totalImagesPullingTime="21.060812859s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:27:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:22.234489859 +0000 UTC m=+23.792674699" watchObservedRunningTime="2026-09-17 07:28:22.234984701 +0000 UTC m=+23.793169510" Sep 17 07:28:22.303635 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:22.303596 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-54.ec2.internal" podStartSLOduration=22.303582698 podStartE2EDuration="22.303582698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:28:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:22.265727624 +0000 UTC m=+23.823912435" watchObservedRunningTime="2026-09-17 07:28:22.303582698 +0000 UTC m=+23.861767507" Sep 17 07:28:22.929408 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:22.929379 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:22.929604 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:22.929515 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2n2" podUID="c35e7d45-1111-40a8-acf5-666b633e8180" Sep 17 07:28:23.211048 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:23.211024 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/ovn-acl-logging/0.log" Sep 17 07:28:23.211546 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:23.211511 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" event={"ID":"77d69f00-3692-47b0-845d-f07095a74966","Type":"ContainerStarted","Data":"e7dfc2a27e511231b9f92e57e617d7f366acc1b66d4bec2f2366a4d120f4fd7b"} Sep 17 07:28:23.928334 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:23.928302 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:23.928527 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:23.928425 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gh9zp" podUID="f68f4c1a-7946-43c1-80ca-b472129b2bfe" Sep 17 07:28:23.928527 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:23.928481 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:28:23.928627 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:23.928606 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsc9z" podUID="6a96dd04-5687-4059-bf3f-ee1864ed8e40" Sep 17 07:28:24.931864 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:24.928866 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:24.931864 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:24.929069 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2n2" podUID="c35e7d45-1111-40a8-acf5-666b633e8180" Sep 17 07:28:25.928811 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:25.928775 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:28:25.928811 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:25.928804 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:25.928985 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:25.928906 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsc9z" podUID="6a96dd04-5687-4059-bf3f-ee1864ed8e40" Sep 17 07:28:25.928985 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:25.928953 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gh9zp" podUID="f68f4c1a-7946-43c1-80ca-b472129b2bfe" Sep 17 07:28:26.221850 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:26.221679 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/ovn-acl-logging/0.log" Sep 17 07:28:26.222210 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:26.222186 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" event={"ID":"77d69f00-3692-47b0-845d-f07095a74966","Type":"ContainerStarted","Data":"0b045ae094a97b7bc60560c033d1374c3929f60af3494af8a1482d5ba961da3d"} Sep 17 07:28:26.222463 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:26.222443 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:26.222680 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:26.222663 2553 scope.go:117] "RemoveContainer" containerID="0c0a50dba9f7b21f36dbd82647f4e6a1e90f4230c7f480ceeab4ff2c37fef679" Sep 17 07:28:26.224255 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:26.224230 2553 generic.go:356] "Generic (PLEG): container finished" podID="8af26520-8854-41c6-a082-58fd1e819408" containerID="68c3744a2aea4efb11b5d83fb948597ab0f1631234e129473b48803b73154553" exitCode=0 Sep 17 07:28:26.224342 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:26.224261 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-spnx4" event={"ID":"8af26520-8854-41c6-a082-58fd1e819408","Type":"ContainerDied","Data":"68c3744a2aea4efb11b5d83fb948597ab0f1631234e129473b48803b73154553"} Sep 17 07:28:26.236974 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:26.236956 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:26.928774 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:26.928663 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:26.928868 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:26.928815 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2n2" podUID="c35e7d45-1111-40a8-acf5-666b633e8180" Sep 17 07:28:27.228662 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:27.228627 2553 generic.go:356] "Generic (PLEG): container finished" podID="8af26520-8854-41c6-a082-58fd1e819408" containerID="f67932e7fa74d4e7268338452192d40a4446bc37dc9057b8beefc05fb61f394f" exitCode=0 Sep 17 07:28:27.229227 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:27.228703 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-spnx4" event={"ID":"8af26520-8854-41c6-a082-58fd1e819408","Type":"ContainerDied","Data":"f67932e7fa74d4e7268338452192d40a4446bc37dc9057b8beefc05fb61f394f"} Sep 17 07:28:27.232658 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:27.232637 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/ovn-acl-logging/0.log" Sep 17 07:28:27.233097 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:27.233071 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" event={"ID":"77d69f00-3692-47b0-845d-f07095a74966","Type":"ContainerStarted","Data":"9563fdb2cddff0d34521eb471d3660cffc31a280650c0bfc14a7356cc53d9788"} Sep 17 07:28:27.233345 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:27.233328 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:27.233493 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:27.233404 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:27.246943 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:27.246926 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:27.283740 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:27.283687 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" podStartSLOduration=8.728994344 podStartE2EDuration="28.283667776s" totalImagesPullingTime="19.554673432s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:27:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:27.282176108 +0000 UTC m=+28.840360915" watchObservedRunningTime="2026-09-17 07:28:27.283667776 +0000 UTC m=+28.841852585" Sep 17 07:28:27.668308 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:27.668276 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tsc9z"] Sep 17 07:28:27.668469 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:27.668388 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:28:27.668541 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:27.668469 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsc9z" podUID="6a96dd04-5687-4059-bf3f-ee1864ed8e40" Sep 17 07:28:27.676511 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:27.676489 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xh2n2"] Sep 17 07:28:27.676612 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:27.676579 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:27.676669 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:27.676653 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2n2" podUID="c35e7d45-1111-40a8-acf5-666b633e8180" Sep 17 07:28:27.680114 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:27.680090 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gh9zp"] Sep 17 07:28:27.680207 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:27.680192 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:27.680292 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:27.680274 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gh9zp" podUID="f68f4c1a-7946-43c1-80ca-b472129b2bfe" Sep 17 07:28:28.237257 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:28.237096 2553 generic.go:356] "Generic (PLEG): container finished" podID="8af26520-8854-41c6-a082-58fd1e819408" containerID="d3ebddc19bdf4850b03219491ba9d1dfc6541e79ddf236f0e3dcd5b927b4cfd9" exitCode=0 Sep 17 07:28:28.237829 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:28.237181 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-spnx4" event={"ID":"8af26520-8854-41c6-a082-58fd1e819408","Type":"ContainerDied","Data":"d3ebddc19bdf4850b03219491ba9d1dfc6541e79ddf236f0e3dcd5b927b4cfd9"} Sep 17 07:28:28.930301 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:28.930227 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:28.930429 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:28.930351 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2n2" podUID="c35e7d45-1111-40a8-acf5-666b633e8180" Sep 17 07:28:28.930429 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:28.930404 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:28:28.930541 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:28.930510 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsc9z" podUID="6a96dd04-5687-4059-bf3f-ee1864ed8e40" Sep 17 07:28:29.928689 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:29.928659 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:29.929076 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:29.928777 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gh9zp" podUID="f68f4c1a-7946-43c1-80ca-b472129b2bfe" Sep 17 07:28:30.929001 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:30.928969 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:30.929395 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:30.929071 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xh2n2" podUID="c35e7d45-1111-40a8-acf5-666b633e8180" Sep 17 07:28:30.929395 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:30.929156 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:28:30.929395 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:30.929252 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tsc9z" podUID="6a96dd04-5687-4059-bf3f-ee1864ed8e40" Sep 17 07:28:31.672058 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:31.671985 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f68f4c1a-7946-43c1-80ca-b472129b2bfe-original-pull-secret\") pod \"global-pull-secret-syncer-gh9zp\" (UID: \"f68f4c1a-7946-43c1-80ca-b472129b2bfe\") " pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:31.672058 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:31.672036 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c35e7d45-1111-40a8-acf5-666b633e8180-metrics-certs\") pod \"network-metrics-daemon-xh2n2\" (UID: \"c35e7d45-1111-40a8-acf5-666b633e8180\") " pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:31.672396 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:31.672153 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:31.672396 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:31.672175 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:31.672396 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:31.672226 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f68f4c1a-7946-43c1-80ca-b472129b2bfe-original-pull-secret podName:f68f4c1a-7946-43c1-80ca-b472129b2bfe nodeName:}" failed. No retries permitted until 2026-09-17 07:29:03.672204662 +0000 UTC m=+65.230389470 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f68f4c1a-7946-43c1-80ca-b472129b2bfe-original-pull-secret") pod "global-pull-secret-syncer-gh9zp" (UID: "f68f4c1a-7946-43c1-80ca-b472129b2bfe") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:28:31.672396 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:31.672246 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c35e7d45-1111-40a8-acf5-666b633e8180-metrics-certs podName:c35e7d45-1111-40a8-acf5-666b633e8180 nodeName:}" failed. No retries permitted until 2026-09-17 07:29:03.672236168 +0000 UTC m=+65.230420954 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c35e7d45-1111-40a8-acf5-666b633e8180-metrics-certs") pod "network-metrics-daemon-xh2n2" (UID: "c35e7d45-1111-40a8-acf5-666b633e8180") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:28:31.772792 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:31.772743 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vc7c\" (UniqueName: \"kubernetes.io/projected/6a96dd04-5687-4059-bf3f-ee1864ed8e40-kube-api-access-5vc7c\") pod \"network-check-target-tsc9z\" (UID: \"6a96dd04-5687-4059-bf3f-ee1864ed8e40\") " pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:28:31.773361 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:31.773324 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:28:31.773481 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:31.773370 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:28:31.773481 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:31.773384 2553 projected.go:196] Error preparing data for projected volume kube-api-access-5vc7c for pod openshift-network-diagnostics/network-check-target-tsc9z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:31.773481 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:31.773447 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a96dd04-5687-4059-bf3f-ee1864ed8e40-kube-api-access-5vc7c podName:6a96dd04-5687-4059-bf3f-ee1864ed8e40 nodeName:}" failed. No retries permitted until 2026-09-17 07:29:03.773428056 +0000 UTC m=+65.331612861 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5vc7c" (UniqueName: "kubernetes.io/projected/6a96dd04-5687-4059-bf3f-ee1864ed8e40-kube-api-access-5vc7c") pod "network-check-target-tsc9z" (UID: "6a96dd04-5687-4059-bf3f-ee1864ed8e40") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:28:31.929246 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:31.929164 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:31.929769 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:31.929266 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gh9zp" podUID="f68f4c1a-7946-43c1-80ca-b472129b2bfe" Sep 17 07:28:32.746468 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.746442 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-54.ec2.internal" event="NodeReady" Sep 17 07:28:32.746646 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.746581 2553 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 17 07:28:32.786100 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.786063 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7484fc99c6-dxxgt"] Sep 17 07:28:32.812838 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.812817 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7484fc99c6-dxxgt"] Sep 17 07:28:32.812974 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.812845 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ctfdd"] Sep 17 07:28:32.812974 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.812944 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:32.815923 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.815902 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cbg6t\"" Sep 17 07:28:32.815923 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.815913 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 17 07:28:32.815923 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.815905 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 17 07:28:32.816119 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.815917 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 17 07:28:32.821537 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.821515 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 17 07:28:32.846626 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.846605 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qsklm"] Sep 17 07:28:32.846730 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.846714 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ctfdd" Sep 17 07:28:32.849818 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.849796 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2jzn2\"" Sep 17 07:28:32.850110 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.850092 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 17 07:28:32.850353 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.850332 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 17 07:28:32.867770 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.867748 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ctfdd"] Sep 17 07:28:32.867856 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.867784 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qsklm"] Sep 17 07:28:32.867856 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.867849 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qsklm" Sep 17 07:28:32.870893 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.870859 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jn6qz\"" Sep 17 07:28:32.871171 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.871155 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 17 07:28:32.871357 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.871345 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 17 07:28:32.871445 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.871431 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 17 07:28:32.928920 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.928894 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:28:32.929085 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.929064 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:28:32.931503 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.931484 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 17 07:28:32.931961 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.931940 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-c888k\"" Sep 17 07:28:32.932126 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.932109 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nzmmj\"" Sep 17 07:28:32.932277 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.932259 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 17 07:28:32.932380 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.932334 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 17 07:28:32.981244 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.981223 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dab30fd6-0467-4b87-83a4-104b2fa7d30d-ca-trust-extracted\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:32.981326 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.981247 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dab30fd6-0467-4b87-83a4-104b2fa7d30d-trusted-ca\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:32.981326 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.981264 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-bound-sa-token\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:32.981326 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.981279 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnjsb\" (UniqueName: \"kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-kube-api-access-hnjsb\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:32.981326 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.981295 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-tls\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:32.981544 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.981393 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dab30fd6-0467-4b87-83a4-104b2fa7d30d-image-registry-private-configuration\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:32.981544 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.981431 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/688b5736-013f-4a52-8145-0fb44f6eb438-config-volume\") pod \"dns-default-ctfdd\" (UID: \"688b5736-013f-4a52-8145-0fb44f6eb438\") " pod="openshift-dns/dns-default-ctfdd" Sep 17 07:28:32.981544 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.981495 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/688b5736-013f-4a52-8145-0fb44f6eb438-metrics-tls\") pod \"dns-default-ctfdd\" (UID: \"688b5736-013f-4a52-8145-0fb44f6eb438\") " pod="openshift-dns/dns-default-ctfdd" Sep 17 07:28:32.981650 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.981548 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fdc52c65-c729-4ef4-9dde-14b700bc0a46-cert\") pod \"ingress-canary-qsklm\" (UID: \"fdc52c65-c729-4ef4-9dde-14b700bc0a46\") " pod="openshift-ingress-canary/ingress-canary-qsklm" Sep 17 07:28:32.981650 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.981580 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7mzz\" (UniqueName: \"kubernetes.io/projected/fdc52c65-c729-4ef4-9dde-14b700bc0a46-kube-api-access-g7mzz\") pod \"ingress-canary-qsklm\" (UID: \"fdc52c65-c729-4ef4-9dde-14b700bc0a46\") " pod="openshift-ingress-canary/ingress-canary-qsklm" Sep 17 07:28:32.981650 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.981599 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-certificates\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:32.981650 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.981614 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dab30fd6-0467-4b87-83a4-104b2fa7d30d-installation-pull-secrets\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:32.981650 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.981629 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/688b5736-013f-4a52-8145-0fb44f6eb438-tmp-dir\") pod \"dns-default-ctfdd\" (UID: \"688b5736-013f-4a52-8145-0fb44f6eb438\") " pod="openshift-dns/dns-default-ctfdd" Sep 17 07:28:32.981650 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:32.981643 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmzpd\" (UniqueName: \"kubernetes.io/projected/688b5736-013f-4a52-8145-0fb44f6eb438-kube-api-access-xmzpd\") pod \"dns-default-ctfdd\" (UID: \"688b5736-013f-4a52-8145-0fb44f6eb438\") " pod="openshift-dns/dns-default-ctfdd" Sep 17 07:28:33.082549 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.082498 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dab30fd6-0467-4b87-83a4-104b2fa7d30d-image-registry-private-configuration\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:33.082549 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.082528 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/688b5736-013f-4a52-8145-0fb44f6eb438-config-volume\") pod \"dns-default-ctfdd\" (UID: \"688b5736-013f-4a52-8145-0fb44f6eb438\") " pod="openshift-dns/dns-default-ctfdd" Sep 17 07:28:33.082686 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.082562 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/688b5736-013f-4a52-8145-0fb44f6eb438-metrics-tls\") pod \"dns-default-ctfdd\" (UID: \"688b5736-013f-4a52-8145-0fb44f6eb438\") " pod="openshift-dns/dns-default-ctfdd" Sep 17 07:28:33.082686 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.082584 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fdc52c65-c729-4ef4-9dde-14b700bc0a46-cert\") pod \"ingress-canary-qsklm\" (UID: \"fdc52c65-c729-4ef4-9dde-14b700bc0a46\") " pod="openshift-ingress-canary/ingress-canary-qsklm" Sep 17 07:28:33.082686 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.082612 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7mzz\" (UniqueName: \"kubernetes.io/projected/fdc52c65-c729-4ef4-9dde-14b700bc0a46-kube-api-access-g7mzz\") pod \"ingress-canary-qsklm\" (UID: \"fdc52c65-c729-4ef4-9dde-14b700bc0a46\") " pod="openshift-ingress-canary/ingress-canary-qsklm" Sep 17 07:28:33.082686 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.082631 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-certificates\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:33.082686 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.082651 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dab30fd6-0467-4b87-83a4-104b2fa7d30d-installation-pull-secrets\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:33.082686 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:33.082664 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:28:33.083000 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:33.082724 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/688b5736-013f-4a52-8145-0fb44f6eb438-metrics-tls podName:688b5736-013f-4a52-8145-0fb44f6eb438 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:33.582705499 +0000 UTC m=+35.140890286 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/688b5736-013f-4a52-8145-0fb44f6eb438-metrics-tls") pod "dns-default-ctfdd" (UID: "688b5736-013f-4a52-8145-0fb44f6eb438") : secret "dns-default-metrics-tls" not found Sep 17 07:28:33.083000 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.082667 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/688b5736-013f-4a52-8145-0fb44f6eb438-tmp-dir\") pod \"dns-default-ctfdd\" (UID: \"688b5736-013f-4a52-8145-0fb44f6eb438\") " pod="openshift-dns/dns-default-ctfdd" Sep 17 07:28:33.083000 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.082768 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmzpd\" (UniqueName: \"kubernetes.io/projected/688b5736-013f-4a52-8145-0fb44f6eb438-kube-api-access-xmzpd\") pod \"dns-default-ctfdd\" (UID: \"688b5736-013f-4a52-8145-0fb44f6eb438\") " pod="openshift-dns/dns-default-ctfdd" Sep 17 07:28:33.083000 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.082802 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dab30fd6-0467-4b87-83a4-104b2fa7d30d-ca-trust-extracted\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:33.083000 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.082827 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dab30fd6-0467-4b87-83a4-104b2fa7d30d-trusted-ca\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:33.083000 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.082854 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-bound-sa-token\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:33.083000 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.082901 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnjsb\" (UniqueName: \"kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-kube-api-access-hnjsb\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:33.083000 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.082948 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-tls\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:33.083000 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.082947 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/688b5736-013f-4a52-8145-0fb44f6eb438-tmp-dir\") pod \"dns-default-ctfdd\" (UID: \"688b5736-013f-4a52-8145-0fb44f6eb438\") " pod="openshift-dns/dns-default-ctfdd" Sep 17 07:28:33.083413 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.083113 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/688b5736-013f-4a52-8145-0fb44f6eb438-config-volume\") pod \"dns-default-ctfdd\" (UID: \"688b5736-013f-4a52-8145-0fb44f6eb438\") " pod="openshift-dns/dns-default-ctfdd" Sep 17 07:28:33.083413 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.083131 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dab30fd6-0467-4b87-83a4-104b2fa7d30d-ca-trust-extracted\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:33.083413 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:33.083191 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 07:28:33.083413 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:33.083210 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7484fc99c6-dxxgt: secret "image-registry-tls" not found Sep 17 07:28:33.083413 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:33.083250 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-tls podName:dab30fd6-0467-4b87-83a4-104b2fa7d30d nodeName:}" failed. No retries permitted until 2026-09-17 07:28:33.583235871 +0000 UTC m=+35.141420663 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-tls") pod "image-registry-7484fc99c6-dxxgt" (UID: "dab30fd6-0467-4b87-83a4-104b2fa7d30d") : secret "image-registry-tls" not found Sep 17 07:28:33.083413 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.083365 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-certificates\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:33.083896 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.083864 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dab30fd6-0467-4b87-83a4-104b2fa7d30d-trusted-ca\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:33.086707 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.086684 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dab30fd6-0467-4b87-83a4-104b2fa7d30d-installation-pull-secrets\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:33.086806 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.086745 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dab30fd6-0467-4b87-83a4-104b2fa7d30d-image-registry-private-configuration\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:33.086866 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.086853 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fdc52c65-c729-4ef4-9dde-14b700bc0a46-cert\") pod \"ingress-canary-qsklm\" (UID: \"fdc52c65-c729-4ef4-9dde-14b700bc0a46\") " pod="openshift-ingress-canary/ingress-canary-qsklm" Sep 17 07:28:33.091695 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.091672 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnjsb\" (UniqueName: \"kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-kube-api-access-hnjsb\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:33.091787 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.091712 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmzpd\" (UniqueName: \"kubernetes.io/projected/688b5736-013f-4a52-8145-0fb44f6eb438-kube-api-access-xmzpd\") pod \"dns-default-ctfdd\" (UID: \"688b5736-013f-4a52-8145-0fb44f6eb438\") " pod="openshift-dns/dns-default-ctfdd" Sep 17 07:28:33.091787 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.091750 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7mzz\" (UniqueName: \"kubernetes.io/projected/fdc52c65-c729-4ef4-9dde-14b700bc0a46-kube-api-access-g7mzz\") pod \"ingress-canary-qsklm\" (UID: \"fdc52c65-c729-4ef4-9dde-14b700bc0a46\") " pod="openshift-ingress-canary/ingress-canary-qsklm" Sep 17 07:28:33.092156 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.092122 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-bound-sa-token\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:33.175983 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.175960 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qsklm" Sep 17 07:28:33.531035 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.530857 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qsklm"] Sep 17 07:28:33.558666 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:28:33.558637 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdc52c65_c729_4ef4_9dde_14b700bc0a46.slice/crio-48c28d80ab4c0d479c439af01de61c412b4844cbb1f78ad18e1c9927efde5790 WatchSource:0}: Error finding container 48c28d80ab4c0d479c439af01de61c412b4844cbb1f78ad18e1c9927efde5790: Status 404 returned error can't find the container with id 48c28d80ab4c0d479c439af01de61c412b4844cbb1f78ad18e1c9927efde5790 Sep 17 07:28:33.585489 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.585466 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-tls\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:33.585571 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.585513 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/688b5736-013f-4a52-8145-0fb44f6eb438-metrics-tls\") pod \"dns-default-ctfdd\" (UID: \"688b5736-013f-4a52-8145-0fb44f6eb438\") " pod="openshift-dns/dns-default-ctfdd" Sep 17 07:28:33.585608 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:33.585598 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:28:33.585640 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:33.585606 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 07:28:33.585640 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:33.585624 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7484fc99c6-dxxgt: secret "image-registry-tls" not found Sep 17 07:28:33.585699 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:33.585641 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/688b5736-013f-4a52-8145-0fb44f6eb438-metrics-tls podName:688b5736-013f-4a52-8145-0fb44f6eb438 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:34.585628212 +0000 UTC m=+36.143812998 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/688b5736-013f-4a52-8145-0fb44f6eb438-metrics-tls") pod "dns-default-ctfdd" (UID: "688b5736-013f-4a52-8145-0fb44f6eb438") : secret "dns-default-metrics-tls" not found Sep 17 07:28:33.585699 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:33.585668 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-tls podName:dab30fd6-0467-4b87-83a4-104b2fa7d30d nodeName:}" failed. No retries permitted until 2026-09-17 07:28:34.585654436 +0000 UTC m=+36.143839253 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-tls") pod "image-registry-7484fc99c6-dxxgt" (UID: "dab30fd6-0467-4b87-83a4-104b2fa7d30d") : secret "image-registry-tls" not found Sep 17 07:28:33.929015 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.928992 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:28:33.931760 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:33.931739 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 17 07:28:34.251106 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.251071 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qsklm" event={"ID":"fdc52c65-c729-4ef4-9dde-14b700bc0a46","Type":"ContainerStarted","Data":"48c28d80ab4c0d479c439af01de61c412b4844cbb1f78ad18e1c9927efde5790"} Sep 17 07:28:34.254264 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.254227 2553 generic.go:356] "Generic (PLEG): container finished" podID="8af26520-8854-41c6-a082-58fd1e819408" containerID="c7b153408ababf7a2629235eb0cfcf53d1b165f42b41b4d20c44d48742937ef5" exitCode=0 Sep 17 07:28:34.254374 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.254266 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-spnx4" event={"ID":"8af26520-8854-41c6-a082-58fd1e819408","Type":"ContainerDied","Data":"c7b153408ababf7a2629235eb0cfcf53d1b165f42b41b4d20c44d48742937ef5"} Sep 17 07:28:34.474241 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.474215 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-6tzzh"] Sep 17 07:28:34.482033 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.482008 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6tzzh" Sep 17 07:28:34.487678 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.487640 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 17 07:28:34.487795 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.487777 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zrnzm\"" Sep 17 07:28:34.488315 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.488288 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 17 07:28:34.491105 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.491086 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 17 07:28:34.491604 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.491583 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 17 07:28:34.491986 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.491961 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-6tzzh"] Sep 17 07:28:34.556441 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.556405 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-cn25t"] Sep 17 07:28:34.569892 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.569866 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-cn25t" Sep 17 07:28:34.572683 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.572660 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 17 07:28:34.572895 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.572863 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-kgwhr\"" Sep 17 07:28:34.573083 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.573063 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 17 07:28:34.581793 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.581773 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-cn25t"] Sep 17 07:28:34.593308 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.593290 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj9ks\" (UniqueName: \"kubernetes.io/projected/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-kube-api-access-dj9ks\") pod \"cluster-monitoring-operator-6cd954b897-6tzzh\" (UID: \"7c06b4ad-93ab-43cc-a24a-9c758f3c957d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6tzzh" Sep 17 07:28:34.593394 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.593358 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-tls\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:34.593394 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.593386 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-6tzzh\" (UID: \"7c06b4ad-93ab-43cc-a24a-9c758f3c957d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6tzzh" Sep 17 07:28:34.593488 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.593415 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-6tzzh\" (UID: \"7c06b4ad-93ab-43cc-a24a-9c758f3c957d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6tzzh" Sep 17 07:28:34.593488 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:34.593452 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 07:28:34.593488 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:34.593467 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7484fc99c6-dxxgt: secret "image-registry-tls" not found Sep 17 07:28:34.593615 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.593490 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/688b5736-013f-4a52-8145-0fb44f6eb438-metrics-tls\") pod \"dns-default-ctfdd\" (UID: \"688b5736-013f-4a52-8145-0fb44f6eb438\") " pod="openshift-dns/dns-default-ctfdd" Sep 17 07:28:34.593615 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:34.593517 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-tls podName:dab30fd6-0467-4b87-83a4-104b2fa7d30d nodeName:}" failed. No retries permitted until 2026-09-17 07:28:36.593499603 +0000 UTC m=+38.151684398 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-tls") pod "image-registry-7484fc99c6-dxxgt" (UID: "dab30fd6-0467-4b87-83a4-104b2fa7d30d") : secret "image-registry-tls" not found Sep 17 07:28:34.593615 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:34.593598 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:28:34.593752 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:34.593650 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/688b5736-013f-4a52-8145-0fb44f6eb438-metrics-tls podName:688b5736-013f-4a52-8145-0fb44f6eb438 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:36.593634761 +0000 UTC m=+38.151819553 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/688b5736-013f-4a52-8145-0fb44f6eb438-metrics-tls") pod "dns-default-ctfdd" (UID: "688b5736-013f-4a52-8145-0fb44f6eb438") : secret "dns-default-metrics-tls" not found Sep 17 07:28:34.663869 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.663845 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-5kks2"] Sep 17 07:28:34.683829 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.683799 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-5kks2" Sep 17 07:28:34.693891 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.693847 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj9ks\" (UniqueName: \"kubernetes.io/projected/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-kube-api-access-dj9ks\") pod \"cluster-monitoring-operator-6cd954b897-6tzzh\" (UID: \"7c06b4ad-93ab-43cc-a24a-9c758f3c957d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6tzzh" Sep 17 07:28:34.694051 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.694031 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmz74\" (UniqueName: \"kubernetes.io/projected/bb0ab5e4-50f2-430c-ac0a-5c451b8715b6-kube-api-access-tmz74\") pod \"volume-data-source-validator-86695b7cc7-cn25t\" (UID: \"bb0ab5e4-50f2-430c-ac0a-5c451b8715b6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-cn25t" Sep 17 07:28:34.694186 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.694080 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-6tzzh\" (UID: \"7c06b4ad-93ab-43cc-a24a-9c758f3c957d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6tzzh" Sep 17 07:28:34.694186 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.694103 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-6tzzh\" (UID: \"7c06b4ad-93ab-43cc-a24a-9c758f3c957d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6tzzh" Sep 17 07:28:34.694315 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:34.694294 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 07:28:34.694394 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:34.694378 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-cluster-monitoring-operator-tls podName:7c06b4ad-93ab-43cc-a24a-9c758f3c957d nodeName:}" failed. No retries permitted until 2026-09-17 07:28:35.194346689 +0000 UTC m=+36.752531494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-6tzzh" (UID: "7c06b4ad-93ab-43cc-a24a-9c758f3c957d") : secret "cluster-monitoring-operator-tls" not found Sep 17 07:28:34.694792 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.694773 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-6tzzh\" (UID: \"7c06b4ad-93ab-43cc-a24a-9c758f3c957d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6tzzh" Sep 17 07:28:34.701864 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.701845 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 17 07:28:34.701976 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.701847 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 17 07:28:34.702215 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.702192 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 17 07:28:34.702308 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.702219 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 17 07:28:34.705757 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.705736 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-gjnlj\"" Sep 17 07:28:34.709627 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.709602 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 17 07:28:34.714166 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.714149 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-5kks2"] Sep 17 07:28:34.746068 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.746050 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj9ks\" (UniqueName: \"kubernetes.io/projected/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-kube-api-access-dj9ks\") pod \"cluster-monitoring-operator-6cd954b897-6tzzh\" (UID: \"7c06b4ad-93ab-43cc-a24a-9c758f3c957d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6tzzh" Sep 17 07:28:34.795055 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.794997 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmz74\" (UniqueName: \"kubernetes.io/projected/bb0ab5e4-50f2-430c-ac0a-5c451b8715b6-kube-api-access-tmz74\") pod \"volume-data-source-validator-86695b7cc7-cn25t\" (UID: \"bb0ab5e4-50f2-430c-ac0a-5c451b8715b6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-cn25t" Sep 17 07:28:34.795055 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.795032 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2455e65f-98bb-463d-a204-5dbd5d69b5f3-trusted-ca\") pod \"console-operator-696884fb9c-5kks2\" (UID: \"2455e65f-98bb-463d-a204-5dbd5d69b5f3\") " pod="openshift-console-operator/console-operator-696884fb9c-5kks2" Sep 17 07:28:34.795209 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.795101 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2455e65f-98bb-463d-a204-5dbd5d69b5f3-serving-cert\") pod \"console-operator-696884fb9c-5kks2\" (UID: \"2455e65f-98bb-463d-a204-5dbd5d69b5f3\") " pod="openshift-console-operator/console-operator-696884fb9c-5kks2" Sep 17 07:28:34.795209 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.795146 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s84rw\" (UniqueName: \"kubernetes.io/projected/2455e65f-98bb-463d-a204-5dbd5d69b5f3-kube-api-access-s84rw\") pod \"console-operator-696884fb9c-5kks2\" (UID: \"2455e65f-98bb-463d-a204-5dbd5d69b5f3\") " pod="openshift-console-operator/console-operator-696884fb9c-5kks2" Sep 17 07:28:34.795209 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.795189 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2455e65f-98bb-463d-a204-5dbd5d69b5f3-config\") pod \"console-operator-696884fb9c-5kks2\" (UID: \"2455e65f-98bb-463d-a204-5dbd5d69b5f3\") " pod="openshift-console-operator/console-operator-696884fb9c-5kks2" Sep 17 07:28:34.842364 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.842343 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmz74\" (UniqueName: \"kubernetes.io/projected/bb0ab5e4-50f2-430c-ac0a-5c451b8715b6-kube-api-access-tmz74\") pod \"volume-data-source-validator-86695b7cc7-cn25t\" (UID: \"bb0ab5e4-50f2-430c-ac0a-5c451b8715b6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-cn25t" Sep 17 07:28:34.879273 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.879252 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-cn25t" Sep 17 07:28:34.896546 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.896524 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2455e65f-98bb-463d-a204-5dbd5d69b5f3-trusted-ca\") pod \"console-operator-696884fb9c-5kks2\" (UID: \"2455e65f-98bb-463d-a204-5dbd5d69b5f3\") " pod="openshift-console-operator/console-operator-696884fb9c-5kks2" Sep 17 07:28:34.896634 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.896558 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2455e65f-98bb-463d-a204-5dbd5d69b5f3-serving-cert\") pod \"console-operator-696884fb9c-5kks2\" (UID: \"2455e65f-98bb-463d-a204-5dbd5d69b5f3\") " pod="openshift-console-operator/console-operator-696884fb9c-5kks2" Sep 17 07:28:34.896634 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.896574 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s84rw\" (UniqueName: \"kubernetes.io/projected/2455e65f-98bb-463d-a204-5dbd5d69b5f3-kube-api-access-s84rw\") pod \"console-operator-696884fb9c-5kks2\" (UID: \"2455e65f-98bb-463d-a204-5dbd5d69b5f3\") " pod="openshift-console-operator/console-operator-696884fb9c-5kks2" Sep 17 07:28:34.896634 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.896598 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2455e65f-98bb-463d-a204-5dbd5d69b5f3-config\") pod \"console-operator-696884fb9c-5kks2\" (UID: \"2455e65f-98bb-463d-a204-5dbd5d69b5f3\") " pod="openshift-console-operator/console-operator-696884fb9c-5kks2" Sep 17 07:28:34.897181 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.897153 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2455e65f-98bb-463d-a204-5dbd5d69b5f3-config\") pod \"console-operator-696884fb9c-5kks2\" (UID: \"2455e65f-98bb-463d-a204-5dbd5d69b5f3\") " pod="openshift-console-operator/console-operator-696884fb9c-5kks2" Sep 17 07:28:34.897261 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.897247 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2455e65f-98bb-463d-a204-5dbd5d69b5f3-trusted-ca\") pod \"console-operator-696884fb9c-5kks2\" (UID: \"2455e65f-98bb-463d-a204-5dbd5d69b5f3\") " pod="openshift-console-operator/console-operator-696884fb9c-5kks2" Sep 17 07:28:34.898654 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.898633 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2455e65f-98bb-463d-a204-5dbd5d69b5f3-serving-cert\") pod \"console-operator-696884fb9c-5kks2\" (UID: \"2455e65f-98bb-463d-a204-5dbd5d69b5f3\") " pod="openshift-console-operator/console-operator-696884fb9c-5kks2" Sep 17 07:28:34.918272 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.918253 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s84rw\" (UniqueName: \"kubernetes.io/projected/2455e65f-98bb-463d-a204-5dbd5d69b5f3-kube-api-access-s84rw\") pod \"console-operator-696884fb9c-5kks2\" (UID: \"2455e65f-98bb-463d-a204-5dbd5d69b5f3\") " pod="openshift-console-operator/console-operator-696884fb9c-5kks2" Sep 17 07:28:34.995485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:34.995456 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-5kks2" Sep 17 07:28:35.099478 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:35.099439 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:35.192072 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:35.192048 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-cn25t"] Sep 17 07:28:35.194991 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:28:35.194971 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb0ab5e4_50f2_430c_ac0a_5c451b8715b6.slice/crio-1014e60f329a4d192bc5f1fddc63b84728590293ba430df53892ebf5fae76524 WatchSource:0}: Error finding container 1014e60f329a4d192bc5f1fddc63b84728590293ba430df53892ebf5fae76524: Status 404 returned error can't find the container with id 1014e60f329a4d192bc5f1fddc63b84728590293ba430df53892ebf5fae76524 Sep 17 07:28:35.199209 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:35.199180 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-6tzzh\" (UID: \"7c06b4ad-93ab-43cc-a24a-9c758f3c957d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6tzzh" Sep 17 07:28:35.199392 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:35.199346 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 07:28:35.199431 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:35.199415 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-cluster-monitoring-operator-tls podName:7c06b4ad-93ab-43cc-a24a-9c758f3c957d nodeName:}" failed. No retries permitted until 2026-09-17 07:28:36.199396146 +0000 UTC m=+37.757580947 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-6tzzh" (UID: "7c06b4ad-93ab-43cc-a24a-9c758f3c957d") : secret "cluster-monitoring-operator-tls" not found Sep 17 07:28:35.224180 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:35.224156 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:35.258045 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:35.258013 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-cn25t" event={"ID":"bb0ab5e4-50f2-430c-ac0a-5c451b8715b6","Type":"ContainerStarted","Data":"1014e60f329a4d192bc5f1fddc63b84728590293ba430df53892ebf5fae76524"} Sep 17 07:28:35.260932 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:35.260912 2553 generic.go:356] "Generic (PLEG): container finished" podID="8af26520-8854-41c6-a082-58fd1e819408" containerID="9e9a66327843f0a020acd808958b4cc239b47dff499e7f7ac4600a2358a2cec0" exitCode=0 Sep 17 07:28:35.261031 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:35.260974 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-spnx4" event={"ID":"8af26520-8854-41c6-a082-58fd1e819408","Type":"ContainerDied","Data":"9e9a66327843f0a020acd808958b4cc239b47dff499e7f7ac4600a2358a2cec0"} Sep 17 07:28:35.316296 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:35.316277 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-5kks2"] Sep 17 07:28:36.040813 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:28:36.040775 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2455e65f_98bb_463d_a204_5dbd5d69b5f3.slice/crio-d2ea181eb0bdb43f83f6bc79ee7f6eb8f04b123c691d96bc88e6f1a157a365f0 WatchSource:0}: Error finding container d2ea181eb0bdb43f83f6bc79ee7f6eb8f04b123c691d96bc88e6f1a157a365f0: Status 404 returned error can't find the container with id d2ea181eb0bdb43f83f6bc79ee7f6eb8f04b123c691d96bc88e6f1a157a365f0 Sep 17 07:28:36.206033 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:36.205890 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-6tzzh\" (UID: \"7c06b4ad-93ab-43cc-a24a-9c758f3c957d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6tzzh" Sep 17 07:28:36.206141 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:36.206031 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 07:28:36.206141 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:36.206086 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-cluster-monitoring-operator-tls podName:7c06b4ad-93ab-43cc-a24a-9c758f3c957d nodeName:}" failed. No retries permitted until 2026-09-17 07:28:38.206069531 +0000 UTC m=+39.764254320 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-6tzzh" (UID: "7c06b4ad-93ab-43cc-a24a-9c758f3c957d") : secret "cluster-monitoring-operator-tls" not found Sep 17 07:28:36.264236 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:36.264201 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-5kks2" event={"ID":"2455e65f-98bb-463d-a204-5dbd5d69b5f3","Type":"ContainerStarted","Data":"d2ea181eb0bdb43f83f6bc79ee7f6eb8f04b123c691d96bc88e6f1a157a365f0"} Sep 17 07:28:36.267347 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:36.267322 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-spnx4" event={"ID":"8af26520-8854-41c6-a082-58fd1e819408","Type":"ContainerStarted","Data":"eb58fbeb3c30bed99ff93465cd86cef0c81c2a2eb3fd2fc3ee645879afb9c6f7"} Sep 17 07:28:36.591663 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:36.591582 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:36.609551 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:36.609516 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/688b5736-013f-4a52-8145-0fb44f6eb438-metrics-tls\") pod \"dns-default-ctfdd\" (UID: \"688b5736-013f-4a52-8145-0fb44f6eb438\") " pod="openshift-dns/dns-default-ctfdd" Sep 17 07:28:36.609661 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:36.609615 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-tls\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:36.609661 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:36.609634 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:28:36.609744 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:36.609695 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/688b5736-013f-4a52-8145-0fb44f6eb438-metrics-tls podName:688b5736-013f-4a52-8145-0fb44f6eb438 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:40.609678044 +0000 UTC m=+42.167862834 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/688b5736-013f-4a52-8145-0fb44f6eb438-metrics-tls") pod "dns-default-ctfdd" (UID: "688b5736-013f-4a52-8145-0fb44f6eb438") : secret "dns-default-metrics-tls" not found Sep 17 07:28:36.609789 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:36.609748 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 07:28:36.609789 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:36.609761 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7484fc99c6-dxxgt: secret "image-registry-tls" not found Sep 17 07:28:36.609936 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:36.609922 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-tls podName:dab30fd6-0467-4b87-83a4-104b2fa7d30d nodeName:}" failed. No retries permitted until 2026-09-17 07:28:40.609906618 +0000 UTC m=+42.168091423 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-tls") pod "image-registry-7484fc99c6-dxxgt" (UID: "dab30fd6-0467-4b87-83a4-104b2fa7d30d") : secret "image-registry-tls" not found Sep 17 07:28:36.699087 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:36.699059 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:37.271538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:37.271497 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qsklm" event={"ID":"fdc52c65-c729-4ef4-9dde-14b700bc0a46","Type":"ContainerStarted","Data":"5e8c222593153a855b608dd689bbad61b518dd36f7dbad99d08fedd7c5c1cc7d"} Sep 17 07:28:37.291669 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:37.291627 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qsklm" podStartSLOduration=2.564960042 podStartE2EDuration="5.291613786s" totalImagesPullingTime="2.726653744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:28:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:37.291056422 +0000 UTC m=+38.849241230" watchObservedRunningTime="2026-09-17 07:28:37.291613786 +0000 UTC m=+38.849798593" Sep 17 07:28:37.292198 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:37.292171 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-spnx4" podStartSLOduration=7.73908253 podStartE2EDuration="38.292159261s" totalImagesPullingTime="30.553076731s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:27:59 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:36.321582606 +0000 UTC m=+37.879767415" watchObservedRunningTime="2026-09-17 07:28:37.292159261 +0000 UTC m=+38.850344068" Sep 17 07:28:37.826479 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:37.826443 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:37.888503 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:37.888470 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:38.222645 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:38.222618 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-6tzzh\" (UID: \"7c06b4ad-93ab-43cc-a24a-9c758f3c957d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6tzzh" Sep 17 07:28:38.222772 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:38.222753 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 07:28:38.222820 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:38.222811 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-cluster-monitoring-operator-tls podName:7c06b4ad-93ab-43cc-a24a-9c758f3c957d nodeName:}" failed. No retries permitted until 2026-09-17 07:28:42.222794936 +0000 UTC m=+43.780979726 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-6tzzh" (UID: "7c06b4ad-93ab-43cc-a24a-9c758f3c957d") : secret "cluster-monitoring-operator-tls" not found Sep 17 07:28:38.275666 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:38.275638 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-cn25t" event={"ID":"bb0ab5e4-50f2-430c-ac0a-5c451b8715b6","Type":"ContainerStarted","Data":"002045c92f33901d296d9cfbe474dbc5c2fd2594eb6010025c7f10133cb36214"} Sep 17 07:28:38.296207 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:38.296163 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-cn25t" podStartSLOduration=2.307112636 podStartE2EDuration="4.296149388s" totalImagesPullingTime="1.989036752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:38.295529161 +0000 UTC m=+39.853713970" watchObservedRunningTime="2026-09-17 07:28:38.296149388 +0000 UTC m=+39.854334199" Sep 17 07:28:38.404488 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:38.404461 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:38.508613 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:38.508583 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:38.712453 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:38.712397 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:38.791375 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:38.791318 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:39.231494 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:39.231465 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:39.280256 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:39.280234 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-5kks2_2455e65f-98bb-463d-a204-5dbd5d69b5f3/console-operator/0.log" Sep 17 07:28:39.280619 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:39.280272 2553 generic.go:356] "Generic (PLEG): container finished" podID="2455e65f-98bb-463d-a204-5dbd5d69b5f3" containerID="2200239daf451b457cc23e9fd67b6291a96cf32cea2b1c8c87ad9a06fa7d554f" exitCode=255 Sep 17 07:28:39.280619 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:39.280342 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-5kks2" event={"ID":"2455e65f-98bb-463d-a204-5dbd5d69b5f3","Type":"ContainerDied","Data":"2200239daf451b457cc23e9fd67b6291a96cf32cea2b1c8c87ad9a06fa7d554f"} Sep 17 07:28:39.280619 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:39.280504 2553 scope.go:117] "RemoveContainer" containerID="2200239daf451b457cc23e9fd67b6291a96cf32cea2b1c8c87ad9a06fa7d554f" Sep 17 07:28:39.312171 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:39.312144 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:40.284057 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:40.284031 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-5kks2_2455e65f-98bb-463d-a204-5dbd5d69b5f3/console-operator/1.log" Sep 17 07:28:40.284479 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:40.284465 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-5kks2_2455e65f-98bb-463d-a204-5dbd5d69b5f3/console-operator/0.log" Sep 17 07:28:40.284519 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:40.284498 2553 generic.go:356] "Generic (PLEG): container finished" podID="2455e65f-98bb-463d-a204-5dbd5d69b5f3" containerID="58505ba8f8600da578e2d722620b024b24cbfdac62d31995bc29884e0e4be019" exitCode=255 Sep 17 07:28:40.284549 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:40.284525 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-5kks2" event={"ID":"2455e65f-98bb-463d-a204-5dbd5d69b5f3","Type":"ContainerDied","Data":"58505ba8f8600da578e2d722620b024b24cbfdac62d31995bc29884e0e4be019"} Sep 17 07:28:40.284614 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:40.284603 2553 scope.go:117] "RemoveContainer" containerID="2200239daf451b457cc23e9fd67b6291a96cf32cea2b1c8c87ad9a06fa7d554f" Sep 17 07:28:40.284782 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:40.284756 2553 scope.go:117] "RemoveContainer" containerID="58505ba8f8600da578e2d722620b024b24cbfdac62d31995bc29884e0e4be019" Sep 17 07:28:40.284981 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:40.284960 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-5kks2_openshift-console-operator(2455e65f-98bb-463d-a204-5dbd5d69b5f3)\"" pod="openshift-console-operator/console-operator-696884fb9c-5kks2" podUID="2455e65f-98bb-463d-a204-5dbd5d69b5f3" Sep 17 07:28:40.639060 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:40.639003 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-tls\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:40.639148 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:40.639059 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/688b5736-013f-4a52-8145-0fb44f6eb438-metrics-tls\") pod \"dns-default-ctfdd\" (UID: \"688b5736-013f-4a52-8145-0fb44f6eb438\") " pod="openshift-dns/dns-default-ctfdd" Sep 17 07:28:40.639148 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:40.639129 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 07:28:40.639148 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:40.639146 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7484fc99c6-dxxgt: secret "image-registry-tls" not found Sep 17 07:28:40.639245 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:40.639150 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:28:40.639245 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:40.639193 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-tls podName:dab30fd6-0467-4b87-83a4-104b2fa7d30d nodeName:}" failed. No retries permitted until 2026-09-17 07:28:48.639179499 +0000 UTC m=+50.197364285 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-tls") pod "image-registry-7484fc99c6-dxxgt" (UID: "dab30fd6-0467-4b87-83a4-104b2fa7d30d") : secret "image-registry-tls" not found Sep 17 07:28:40.639245 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:40.639206 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/688b5736-013f-4a52-8145-0fb44f6eb438-metrics-tls podName:688b5736-013f-4a52-8145-0fb44f6eb438 nodeName:}" failed. No retries permitted until 2026-09-17 07:28:48.639200457 +0000 UTC m=+50.197385243 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/688b5736-013f-4a52-8145-0fb44f6eb438-metrics-tls") pod "dns-default-ctfdd" (UID: "688b5736-013f-4a52-8145-0fb44f6eb438") : secret "dns-default-metrics-tls" not found Sep 17 07:28:41.289162 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:41.289138 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-5kks2_2455e65f-98bb-463d-a204-5dbd5d69b5f3/console-operator/1.log" Sep 17 07:28:41.289515 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:41.289474 2553 scope.go:117] "RemoveContainer" containerID="58505ba8f8600da578e2d722620b024b24cbfdac62d31995bc29884e0e4be019" Sep 17 07:28:41.289661 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:41.289639 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-5kks2_openshift-console-operator(2455e65f-98bb-463d-a204-5dbd5d69b5f3)\"" pod="openshift-console-operator/console-operator-696884fb9c-5kks2" podUID="2455e65f-98bb-463d-a204-5dbd5d69b5f3" Sep 17 07:28:41.782957 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:41.782920 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:42.250829 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:42.250794 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-6tzzh\" (UID: \"7c06b4ad-93ab-43cc-a24a-9c758f3c957d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6tzzh" Sep 17 07:28:42.251008 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:42.250944 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 07:28:42.251008 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:42.251008 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-cluster-monitoring-operator-tls podName:7c06b4ad-93ab-43cc-a24a-9c758f3c957d nodeName:}" failed. No retries permitted until 2026-09-17 07:28:50.250993104 +0000 UTC m=+51.809177894 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-6tzzh" (UID: "7c06b4ad-93ab-43cc-a24a-9c758f3c957d") : secret "cluster-monitoring-operator-tls" not found Sep 17 07:28:42.817761 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:42.817727 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:42.818535 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:42.818515 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nwctn"] Sep 17 07:28:42.853390 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:42.853367 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nwctn"] Sep 17 07:28:42.853490 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:42.853472 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:42.856130 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:42.856110 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 17 07:28:42.856130 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:42.856123 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4qrls\"" Sep 17 07:28:42.856333 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:42.856320 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 17 07:28:42.856407 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:42.856390 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 17 07:28:42.857058 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:42.857044 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 17 07:28:42.955290 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:42.955267 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2669c49a-7327-437d-8ed2-fcba617cc62e-crio-socket\") pod \"insights-runtime-extractor-nwctn\" (UID: \"2669c49a-7327-437d-8ed2-fcba617cc62e\") " pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:42.955369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:42.955292 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-884v8\" (UniqueName: \"kubernetes.io/projected/2669c49a-7327-437d-8ed2-fcba617cc62e-kube-api-access-884v8\") pod \"insights-runtime-extractor-nwctn\" (UID: \"2669c49a-7327-437d-8ed2-fcba617cc62e\") " pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:42.955369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:42.955334 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2669c49a-7327-437d-8ed2-fcba617cc62e-data-volume\") pod \"insights-runtime-extractor-nwctn\" (UID: \"2669c49a-7327-437d-8ed2-fcba617cc62e\") " pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:42.955431 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:42.955403 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2669c49a-7327-437d-8ed2-fcba617cc62e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nwctn\" (UID: \"2669c49a-7327-437d-8ed2-fcba617cc62e\") " pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:42.955462 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:42.955447 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2669c49a-7327-437d-8ed2-fcba617cc62e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nwctn\" (UID: \"2669c49a-7327-437d-8ed2-fcba617cc62e\") " pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:43.055706 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:43.055684 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2669c49a-7327-437d-8ed2-fcba617cc62e-crio-socket\") pod \"insights-runtime-extractor-nwctn\" (UID: \"2669c49a-7327-437d-8ed2-fcba617cc62e\") " pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:43.055760 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:43.055711 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-884v8\" (UniqueName: \"kubernetes.io/projected/2669c49a-7327-437d-8ed2-fcba617cc62e-kube-api-access-884v8\") pod \"insights-runtime-extractor-nwctn\" (UID: \"2669c49a-7327-437d-8ed2-fcba617cc62e\") " pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:43.055760 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:43.055750 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2669c49a-7327-437d-8ed2-fcba617cc62e-data-volume\") pod \"insights-runtime-extractor-nwctn\" (UID: \"2669c49a-7327-437d-8ed2-fcba617cc62e\") " pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:43.055826 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:43.055789 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2669c49a-7327-437d-8ed2-fcba617cc62e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nwctn\" (UID: \"2669c49a-7327-437d-8ed2-fcba617cc62e\") " pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:43.055862 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:43.055843 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2669c49a-7327-437d-8ed2-fcba617cc62e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nwctn\" (UID: \"2669c49a-7327-437d-8ed2-fcba617cc62e\") " pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:43.055937 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:43.055904 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2669c49a-7327-437d-8ed2-fcba617cc62e-crio-socket\") pod \"insights-runtime-extractor-nwctn\" (UID: \"2669c49a-7327-437d-8ed2-fcba617cc62e\") " pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:43.055937 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:43.055926 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 07:28:43.056031 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:43.055992 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2669c49a-7327-437d-8ed2-fcba617cc62e-kube-rbac-proxy-cm podName:2669c49a-7327-437d-8ed2-fcba617cc62e nodeName:}" failed. No retries permitted until 2026-09-17 07:28:43.555978368 +0000 UTC m=+45.114163170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2669c49a-7327-437d-8ed2-fcba617cc62e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nwctn" (UID: "2669c49a-7327-437d-8ed2-fcba617cc62e") : configmap "kube-rbac-proxy" not found Sep 17 07:28:43.056031 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:43.055999 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 07:28:43.056120 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:43.056059 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2669c49a-7327-437d-8ed2-fcba617cc62e-insights-runtime-extractor-tls podName:2669c49a-7327-437d-8ed2-fcba617cc62e nodeName:}" failed. No retries permitted until 2026-09-17 07:28:43.556040113 +0000 UTC m=+45.114224919 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2669c49a-7327-437d-8ed2-fcba617cc62e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nwctn" (UID: "2669c49a-7327-437d-8ed2-fcba617cc62e") : secret "insights-runtime-extractor-tls" not found Sep 17 07:28:43.056188 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:43.056167 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2669c49a-7327-437d-8ed2-fcba617cc62e-data-volume\") pod \"insights-runtime-extractor-nwctn\" (UID: \"2669c49a-7327-437d-8ed2-fcba617cc62e\") " pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:43.065762 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:43.065744 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-884v8\" (UniqueName: \"kubernetes.io/projected/2669c49a-7327-437d-8ed2-fcba617cc62e-kube-api-access-884v8\") pod \"insights-runtime-extractor-nwctn\" (UID: \"2669c49a-7327-437d-8ed2-fcba617cc62e\") " pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:43.560311 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:43.560276 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2669c49a-7327-437d-8ed2-fcba617cc62e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nwctn\" (UID: \"2669c49a-7327-437d-8ed2-fcba617cc62e\") " pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:43.560485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:43.560369 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2669c49a-7327-437d-8ed2-fcba617cc62e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nwctn\" (UID: \"2669c49a-7327-437d-8ed2-fcba617cc62e\") " pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:43.560485 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:43.560412 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 07:28:43.560485 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:43.560442 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 07:28:43.560485 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:43.560472 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2669c49a-7327-437d-8ed2-fcba617cc62e-insights-runtime-extractor-tls podName:2669c49a-7327-437d-8ed2-fcba617cc62e nodeName:}" failed. No retries permitted until 2026-09-17 07:28:44.560458448 +0000 UTC m=+46.118643238 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2669c49a-7327-437d-8ed2-fcba617cc62e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nwctn" (UID: "2669c49a-7327-437d-8ed2-fcba617cc62e") : secret "insights-runtime-extractor-tls" not found Sep 17 07:28:43.560485 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:43.560487 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2669c49a-7327-437d-8ed2-fcba617cc62e-kube-rbac-proxy-cm podName:2669c49a-7327-437d-8ed2-fcba617cc62e nodeName:}" failed. No retries permitted until 2026-09-17 07:28:44.560480615 +0000 UTC m=+46.118665401 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2669c49a-7327-437d-8ed2-fcba617cc62e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nwctn" (UID: "2669c49a-7327-437d-8ed2-fcba617cc62e") : configmap "kube-rbac-proxy" not found Sep 17 07:28:43.939654 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:43.939559 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:44.026702 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:44.026672 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:44.124953 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:44.124927 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:44.225177 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:44.225149 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:44.281616 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:44.281590 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:44.419689 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:44.419657 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:44.481296 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:44.481240 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:44.561676 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:44.561653 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ddqsw_6d54b36a-cd25-4e0e-8a1d-a0b1ce6de23e/dns-node-resolver/0.log" Sep 17 07:28:44.567959 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:44.567935 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2669c49a-7327-437d-8ed2-fcba617cc62e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nwctn\" (UID: \"2669c49a-7327-437d-8ed2-fcba617cc62e\") " pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:44.568016 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:44.567981 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2669c49a-7327-437d-8ed2-fcba617cc62e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nwctn\" (UID: \"2669c49a-7327-437d-8ed2-fcba617cc62e\") " pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:44.568079 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:44.568061 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 07:28:44.568125 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:44.568110 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 07:28:44.568172 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:44.568125 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2669c49a-7327-437d-8ed2-fcba617cc62e-kube-rbac-proxy-cm podName:2669c49a-7327-437d-8ed2-fcba617cc62e nodeName:}" failed. No retries permitted until 2026-09-17 07:28:46.568110015 +0000 UTC m=+48.126294802 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2669c49a-7327-437d-8ed2-fcba617cc62e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nwctn" (UID: "2669c49a-7327-437d-8ed2-fcba617cc62e") : configmap "kube-rbac-proxy" not found Sep 17 07:28:44.568172 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:44.568162 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2669c49a-7327-437d-8ed2-fcba617cc62e-insights-runtime-extractor-tls podName:2669c49a-7327-437d-8ed2-fcba617cc62e nodeName:}" failed. No retries permitted until 2026-09-17 07:28:46.568149164 +0000 UTC m=+48.126333956 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2669c49a-7327-437d-8ed2-fcba617cc62e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nwctn" (UID: "2669c49a-7327-437d-8ed2-fcba617cc62e") : secret "insights-runtime-extractor-tls" not found Sep 17 07:28:44.889078 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:44.889002 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:44.987275 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:44.987240 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:44.996102 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:44.996084 2553 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-5kks2" Sep 17 07:28:44.996165 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:44.996124 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-5kks2" Sep 17 07:28:44.996391 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:44.996381 2553 scope.go:117] "RemoveContainer" containerID="58505ba8f8600da578e2d722620b024b24cbfdac62d31995bc29884e0e4be019" Sep 17 07:28:44.996521 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:44.996506 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-5kks2_openshift-console-operator(2455e65f-98bb-463d-a204-5dbd5d69b5f3)\"" pod="openshift-console-operator/console-operator-696884fb9c-5kks2" podUID="2455e65f-98bb-463d-a204-5dbd5d69b5f3" Sep 17 07:28:45.559061 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:45.559031 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7l8hw_b388be8d-ade7-4e8d-b182-c27d6b5c5b1a/node-ca/0.log" Sep 17 07:28:46.361683 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:46.361655 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qsklm_fdc52c65-c729-4ef4-9dde-14b700bc0a46/serve-healthcheck-canary/0.log" Sep 17 07:28:46.495369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:46.495333 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-6dc7s"] Sep 17 07:28:46.519440 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:46.519415 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-6dc7s"] Sep 17 07:28:46.519561 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:46.519520 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-6dc7s" Sep 17 07:28:46.522459 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:46.522434 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-cxclr\"" Sep 17 07:28:46.522574 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:46.522460 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 17 07:28:46.522574 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:46.522434 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 17 07:28:46.523580 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:46.523557 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 17 07:28:46.523697 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:46.523685 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 17 07:28:46.524793 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:46.524759 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:46.583154 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:46.583129 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2669c49a-7327-437d-8ed2-fcba617cc62e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nwctn\" (UID: \"2669c49a-7327-437d-8ed2-fcba617cc62e\") " pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:46.583240 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:46.583173 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2669c49a-7327-437d-8ed2-fcba617cc62e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nwctn\" (UID: \"2669c49a-7327-437d-8ed2-fcba617cc62e\") " pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:46.583285 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:46.583246 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 07:28:46.583285 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:46.583256 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 07:28:46.583340 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:46.583296 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2669c49a-7327-437d-8ed2-fcba617cc62e-kube-rbac-proxy-cm podName:2669c49a-7327-437d-8ed2-fcba617cc62e nodeName:}" failed. No retries permitted until 2026-09-17 07:28:50.583282847 +0000 UTC m=+52.141467638 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2669c49a-7327-437d-8ed2-fcba617cc62e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nwctn" (UID: "2669c49a-7327-437d-8ed2-fcba617cc62e") : configmap "kube-rbac-proxy" not found Sep 17 07:28:46.583380 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:46.583369 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2669c49a-7327-437d-8ed2-fcba617cc62e-insights-runtime-extractor-tls podName:2669c49a-7327-437d-8ed2-fcba617cc62e nodeName:}" failed. No retries permitted until 2026-09-17 07:28:50.583350443 +0000 UTC m=+52.141535233 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2669c49a-7327-437d-8ed2-fcba617cc62e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nwctn" (UID: "2669c49a-7327-437d-8ed2-fcba617cc62e") : secret "insights-runtime-extractor-tls" not found Sep 17 07:28:46.624212 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:46.624149 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:46.684230 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:46.684206 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a9d9441b-306e-4a3b-a426-60d9a56c40fd-signing-cabundle\") pod \"service-ca-568869fdc5-6dc7s\" (UID: \"a9d9441b-306e-4a3b-a426-60d9a56c40fd\") " pod="openshift-service-ca/service-ca-568869fdc5-6dc7s" Sep 17 07:28:46.684303 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:46.684235 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxs54\" (UniqueName: \"kubernetes.io/projected/a9d9441b-306e-4a3b-a426-60d9a56c40fd-kube-api-access-mxs54\") pod \"service-ca-568869fdc5-6dc7s\" (UID: \"a9d9441b-306e-4a3b-a426-60d9a56c40fd\") " pod="openshift-service-ca/service-ca-568869fdc5-6dc7s" Sep 17 07:28:46.684303 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:46.684254 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a9d9441b-306e-4a3b-a426-60d9a56c40fd-signing-key\") pod \"service-ca-568869fdc5-6dc7s\" (UID: \"a9d9441b-306e-4a3b-a426-60d9a56c40fd\") " pod="openshift-service-ca/service-ca-568869fdc5-6dc7s" Sep 17 07:28:46.725628 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:46.725605 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:46.785198 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:46.785171 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a9d9441b-306e-4a3b-a426-60d9a56c40fd-signing-cabundle\") pod \"service-ca-568869fdc5-6dc7s\" (UID: \"a9d9441b-306e-4a3b-a426-60d9a56c40fd\") " pod="openshift-service-ca/service-ca-568869fdc5-6dc7s" Sep 17 07:28:46.785291 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:46.785206 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxs54\" (UniqueName: \"kubernetes.io/projected/a9d9441b-306e-4a3b-a426-60d9a56c40fd-kube-api-access-mxs54\") pod \"service-ca-568869fdc5-6dc7s\" (UID: \"a9d9441b-306e-4a3b-a426-60d9a56c40fd\") " pod="openshift-service-ca/service-ca-568869fdc5-6dc7s" Sep 17 07:28:46.785291 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:46.785236 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a9d9441b-306e-4a3b-a426-60d9a56c40fd-signing-key\") pod \"service-ca-568869fdc5-6dc7s\" (UID: \"a9d9441b-306e-4a3b-a426-60d9a56c40fd\") " pod="openshift-service-ca/service-ca-568869fdc5-6dc7s" Sep 17 07:28:46.785821 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:46.785803 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a9d9441b-306e-4a3b-a426-60d9a56c40fd-signing-cabundle\") pod \"service-ca-568869fdc5-6dc7s\" (UID: \"a9d9441b-306e-4a3b-a426-60d9a56c40fd\") " pod="openshift-service-ca/service-ca-568869fdc5-6dc7s" Sep 17 07:28:46.787577 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:46.787555 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a9d9441b-306e-4a3b-a426-60d9a56c40fd-signing-key\") pod \"service-ca-568869fdc5-6dc7s\" (UID: \"a9d9441b-306e-4a3b-a426-60d9a56c40fd\") " pod="openshift-service-ca/service-ca-568869fdc5-6dc7s" Sep 17 07:28:46.795547 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:46.795525 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxs54\" (UniqueName: \"kubernetes.io/projected/a9d9441b-306e-4a3b-a426-60d9a56c40fd-kube-api-access-mxs54\") pod \"service-ca-568869fdc5-6dc7s\" (UID: \"a9d9441b-306e-4a3b-a426-60d9a56c40fd\") " pod="openshift-service-ca/service-ca-568869fdc5-6dc7s" Sep 17 07:28:46.829806 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:46.829788 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-6dc7s" Sep 17 07:28:47.130015 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:47.129987 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-6dc7s"] Sep 17 07:28:47.132655 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:28:47.132630 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9d9441b_306e_4a3b_a426_60d9a56c40fd.slice/crio-4019da49865a82f13ee1749673f4e67843a7dfb16f53467b9a509646144e81de WatchSource:0}: Error finding container 4019da49865a82f13ee1749673f4e67843a7dfb16f53467b9a509646144e81de: Status 404 returned error can't find the container with id 4019da49865a82f13ee1749673f4e67843a7dfb16f53467b9a509646144e81de Sep 17 07:28:47.304154 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:47.304122 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-6dc7s" event={"ID":"a9d9441b-306e-4a3b-a426-60d9a56c40fd","Type":"ContainerStarted","Data":"4019da49865a82f13ee1749673f4e67843a7dfb16f53467b9a509646144e81de"} Sep 17 07:28:47.702026 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:47.702000 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:48.701910 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:48.701864 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-tls\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:28:48.702159 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:48.701929 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/688b5736-013f-4a52-8145-0fb44f6eb438-metrics-tls\") pod \"dns-default-ctfdd\" (UID: \"688b5736-013f-4a52-8145-0fb44f6eb438\") " pod="openshift-dns/dns-default-ctfdd" Sep 17 07:28:48.702159 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:48.702120 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:28:48.702159 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:48.702125 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 07:28:48.702159 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:48.702147 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7484fc99c6-dxxgt: secret "image-registry-tls" not found Sep 17 07:28:48.702553 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:48.702185 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/688b5736-013f-4a52-8145-0fb44f6eb438-metrics-tls podName:688b5736-013f-4a52-8145-0fb44f6eb438 nodeName:}" failed. No retries permitted until 2026-09-17 07:29:04.702166406 +0000 UTC m=+66.260351210 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/688b5736-013f-4a52-8145-0fb44f6eb438-metrics-tls") pod "dns-default-ctfdd" (UID: "688b5736-013f-4a52-8145-0fb44f6eb438") : secret "dns-default-metrics-tls" not found Sep 17 07:28:48.702553 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:48.702202 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-tls podName:dab30fd6-0467-4b87-83a4-104b2fa7d30d nodeName:}" failed. No retries permitted until 2026-09-17 07:29:04.702195186 +0000 UTC m=+66.260379975 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-tls") pod "image-registry-7484fc99c6-dxxgt" (UID: "dab30fd6-0467-4b87-83a4-104b2fa7d30d") : secret "image-registry-tls" not found Sep 17 07:28:49.010627 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:49.010540 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:50.158922 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:50.158866 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:50.253844 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:50.253804 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:50.314457 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:50.314424 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-6dc7s" event={"ID":"a9d9441b-306e-4a3b-a426-60d9a56c40fd","Type":"ContainerStarted","Data":"bbbc55c2f18b4bba3de74427ecd4839111910f77c10feaec1147048ead7a753f"} Sep 17 07:28:50.315488 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:50.315460 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-6tzzh\" (UID: \"7c06b4ad-93ab-43cc-a24a-9c758f3c957d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6tzzh" Sep 17 07:28:50.316702 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:50.315729 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 07:28:50.316943 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:50.316872 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-cluster-monitoring-operator-tls podName:7c06b4ad-93ab-43cc-a24a-9c758f3c957d nodeName:}" failed. No retries permitted until 2026-09-17 07:29:06.316851477 +0000 UTC m=+67.875036273 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-6tzzh" (UID: "7c06b4ad-93ab-43cc-a24a-9c758f3c957d") : secret "cluster-monitoring-operator-tls" not found Sep 17 07:28:50.372531 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:50.372500 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:50.481548 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:50.481508 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:50.575480 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:50.575439 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:50.619648 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:50.619616 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2669c49a-7327-437d-8ed2-fcba617cc62e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nwctn\" (UID: \"2669c49a-7327-437d-8ed2-fcba617cc62e\") " pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:50.619806 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:50.619688 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2669c49a-7327-437d-8ed2-fcba617cc62e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nwctn\" (UID: \"2669c49a-7327-437d-8ed2-fcba617cc62e\") " pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:50.619865 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:50.619811 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 07:28:50.619943 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:28:50.619888 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2669c49a-7327-437d-8ed2-fcba617cc62e-insights-runtime-extractor-tls podName:2669c49a-7327-437d-8ed2-fcba617cc62e nodeName:}" failed. No retries permitted until 2026-09-17 07:28:58.6198592 +0000 UTC m=+60.178043991 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2669c49a-7327-437d-8ed2-fcba617cc62e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nwctn" (UID: "2669c49a-7327-437d-8ed2-fcba617cc62e") : secret "insights-runtime-extractor-tls" not found Sep 17 07:28:50.620242 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:50.620216 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2669c49a-7327-437d-8ed2-fcba617cc62e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nwctn\" (UID: \"2669c49a-7327-437d-8ed2-fcba617cc62e\") " pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:50.876813 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:50.876736 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:50.957539 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:50.957512 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:51.631979 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:51.631941 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:52.048515 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:52.048471 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:52.425669 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:52.425586 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:52.821440 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:52.821399 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:53.227715 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:53.227683 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:53.627990 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:53.627917 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:53.830787 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:53.830758 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:53.947303 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:53.947223 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:54.215940 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:54.215913 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:54.427467 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:54.427443 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:54.830010 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:54.829986 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:54.912984 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:54.912959 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:55.056772 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:55.056746 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:55.428374 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:55.428348 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:55.850064 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:55.850035 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:56.411693 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:56.411663 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:56.625857 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:56.625828 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:56.827679 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:56.827658 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:57.020998 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:57.020977 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:57.237080 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:57.237052 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:58.238578 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:58.238538 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:58.654797 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:58.654718 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:58.685465 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:58.685438 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2669c49a-7327-437d-8ed2-fcba617cc62e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nwctn\" (UID: \"2669c49a-7327-437d-8ed2-fcba617cc62e\") " pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:58.687601 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:58.687576 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2669c49a-7327-437d-8ed2-fcba617cc62e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nwctn\" (UID: \"2669c49a-7327-437d-8ed2-fcba617cc62e\") " pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:58.761977 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:58.761953 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nwctn" Sep 17 07:28:58.837198 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:58.837163 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:59.062839 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:59.062794 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-6dc7s" podStartSLOduration=10.917048819 podStartE2EDuration="13.062778258s" totalImagesPullingTime="2.145729439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:28:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:28:50.380181943 +0000 UTC m=+51.938366752" watchObservedRunningTime="2026-09-17 07:28:59.062778258 +0000 UTC m=+60.620963066" Sep 17 07:28:59.063177 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:59.063161 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nwctn"] Sep 17 07:28:59.064902 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:28:59.064854 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2669c49a_7327_437d_8ed2_fcba617cc62e.slice/crio-137ded3856bfb52f3e8824041ddb9cf08b1751cc734c9b27748c3c2aa8a5b768 WatchSource:0}: Error finding container 137ded3856bfb52f3e8824041ddb9cf08b1751cc734c9b27748c3c2aa8a5b768: Status 404 returned error can't find the container with id 137ded3856bfb52f3e8824041ddb9cf08b1751cc734c9b27748c3c2aa8a5b768 Sep 17 07:28:59.114923 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:59.114871 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:59.243455 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:59.243416 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:59.250107 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:59.250089 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zhw5d" Sep 17 07:28:59.313288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:59.313232 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:59.340003 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:59.339978 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nwctn" event={"ID":"2669c49a-7327-437d-8ed2-fcba617cc62e","Type":"ContainerStarted","Data":"d622d21afaf593840c29cae06b40603c5971d074190ce9ed011dde6a626999a8"} Sep 17 07:28:59.340084 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:59.340049 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nwctn" event={"ID":"2669c49a-7327-437d-8ed2-fcba617cc62e","Type":"ContainerStarted","Data":"137ded3856bfb52f3e8824041ddb9cf08b1751cc734c9b27748c3c2aa8a5b768"} Sep 17 07:28:59.724953 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:59.724918 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:28:59.928792 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:28:59.928772 2553 scope.go:117] "RemoveContainer" containerID="58505ba8f8600da578e2d722620b024b24cbfdac62d31995bc29884e0e4be019" Sep 17 07:29:00.256816 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:00.256771 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:00.347137 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:00.347098 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-5kks2_2455e65f-98bb-463d-a204-5dbd5d69b5f3/console-operator/1.log" Sep 17 07:29:00.347291 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:00.347189 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-5kks2" event={"ID":"2455e65f-98bb-463d-a204-5dbd5d69b5f3","Type":"ContainerStarted","Data":"60c82a21108b262f5a5efec2b936ffdbf3e436539e80f17c720b6a9972f6d1ee"} Sep 17 07:29:00.347691 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:00.347670 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-5kks2" Sep 17 07:29:00.350210 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:00.350175 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nwctn" event={"ID":"2669c49a-7327-437d-8ed2-fcba617cc62e","Type":"ContainerStarted","Data":"a8b2563bc39dfe3ba156f20280520805d0e34262d99596f7721b4152f152d910"} Sep 17 07:29:00.353938 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:00.353648 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-5kks2" Sep 17 07:29:00.405369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:00.405323 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-5kks2" podStartSLOduration=24.095975531 podStartE2EDuration="26.40530524s" totalImagesPullingTime="2.309329709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:00.403564329 +0000 UTC m=+61.961749128" watchObservedRunningTime="2026-09-17 07:29:00.40530524 +0000 UTC m=+61.963490047" Sep 17 07:29:00.457377 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:00.457342 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:00.527541 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:00.527462 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:00.654838 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:00.654790 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:00.723232 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:00.723204 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:00.854339 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:00.854257 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:00.929355 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:00.929314 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:01.080433 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:01.080401 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:01.145217 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:01.145140 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:01.355291 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:01.355264 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nwctn" event={"ID":"2669c49a-7327-437d-8ed2-fcba617cc62e","Type":"ContainerStarted","Data":"1baec340a2d1b1793ca9fc99f8cc65178d0b31d9ad14db094bba9935eb422aed"} Sep 17 07:29:01.378730 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:01.378688 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nwctn" podStartSLOduration=17.315797611 podStartE2EDuration="19.378673359s" totalImagesPullingTime="2.062875748s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:28:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:01.377809356 +0000 UTC m=+62.935994177" watchObservedRunningTime="2026-09-17 07:29:01.378673359 +0000 UTC m=+62.936858208" Sep 17 07:29:01.893765 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:01.893735 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:02.185673 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:02.185585 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:02.285987 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:02.285958 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:03.728426 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:03.728391 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f68f4c1a-7946-43c1-80ca-b472129b2bfe-original-pull-secret\") pod \"global-pull-secret-syncer-gh9zp\" (UID: \"f68f4c1a-7946-43c1-80ca-b472129b2bfe\") " pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:29:03.728426 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:03.728428 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c35e7d45-1111-40a8-acf5-666b633e8180-metrics-certs\") pod \"network-metrics-daemon-xh2n2\" (UID: \"c35e7d45-1111-40a8-acf5-666b633e8180\") " pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:29:03.731936 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:03.731916 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 17 07:29:03.732058 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:03.731958 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 17 07:29:03.740715 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:03.740681 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c35e7d45-1111-40a8-acf5-666b633e8180-metrics-certs\") pod \"network-metrics-daemon-xh2n2\" (UID: \"c35e7d45-1111-40a8-acf5-666b633e8180\") " pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:29:03.740809 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:03.740691 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f68f4c1a-7946-43c1-80ca-b472129b2bfe-original-pull-secret\") pod \"global-pull-secret-syncer-gh9zp\" (UID: \"f68f4c1a-7946-43c1-80ca-b472129b2bfe\") " pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:29:03.829717 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:03.829692 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vc7c\" (UniqueName: \"kubernetes.io/projected/6a96dd04-5687-4059-bf3f-ee1864ed8e40-kube-api-access-5vc7c\") pod \"network-check-target-tsc9z\" (UID: \"6a96dd04-5687-4059-bf3f-ee1864ed8e40\") " pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:29:03.834592 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:03.834572 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 17 07:29:03.843421 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:03.843400 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 17 07:29:03.850144 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:03.850127 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-c888k\"" Sep 17 07:29:03.853208 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:03.853185 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vc7c\" (UniqueName: \"kubernetes.io/projected/6a96dd04-5687-4059-bf3f-ee1864ed8e40-kube-api-access-5vc7c\") pod \"network-check-target-tsc9z\" (UID: \"6a96dd04-5687-4059-bf3f-ee1864ed8e40\") " pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:29:03.854953 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:03.854930 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xh2n2" Sep 17 07:29:03.939440 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:03.939409 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gh9zp" Sep 17 07:29:04.135077 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:04.135051 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xh2n2"] Sep 17 07:29:04.138743 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:29:04.138719 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc35e7d45_1111_40a8_acf5_666b633e8180.slice/crio-f11b56b1776832025fb339f44bae73bf64df90df5ae097f68c056a27523e5aff WatchSource:0}: Error finding container f11b56b1776832025fb339f44bae73bf64df90df5ae097f68c056a27523e5aff: Status 404 returned error can't find the container with id f11b56b1776832025fb339f44bae73bf64df90df5ae097f68c056a27523e5aff Sep 17 07:29:04.142575 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:04.142557 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nzmmj\"" Sep 17 07:29:04.149940 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:04.149924 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:29:04.221793 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:04.221635 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gh9zp"] Sep 17 07:29:04.365918 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:04.365851 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gh9zp" event={"ID":"f68f4c1a-7946-43c1-80ca-b472129b2bfe","Type":"ContainerStarted","Data":"f2ba1a579fc374d0479b112cf74cb8bc6f3953a9cae46d639078a8d8522b6c52"} Sep 17 07:29:04.366986 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:04.366966 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xh2n2" event={"ID":"c35e7d45-1111-40a8-acf5-666b633e8180","Type":"ContainerStarted","Data":"f11b56b1776832025fb339f44bae73bf64df90df5ae097f68c056a27523e5aff"} Sep 17 07:29:04.442963 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:04.442944 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tsc9z"] Sep 17 07:29:04.445081 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:29:04.445059 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a96dd04_5687_4059_bf3f_ee1864ed8e40.slice/crio-894b227cec35123e4ae2456b1f64bdd0cb443ebc5672261ae95ba1015e7b3b9d WatchSource:0}: Error finding container 894b227cec35123e4ae2456b1f64bdd0cb443ebc5672261ae95ba1015e7b3b9d: Status 404 returned error can't find the container with id 894b227cec35123e4ae2456b1f64bdd0cb443ebc5672261ae95ba1015e7b3b9d Sep 17 07:29:04.736856 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:04.736829 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-tls\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:29:04.737241 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:04.736873 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/688b5736-013f-4a52-8145-0fb44f6eb438-metrics-tls\") pod \"dns-default-ctfdd\" (UID: \"688b5736-013f-4a52-8145-0fb44f6eb438\") " pod="openshift-dns/dns-default-ctfdd" Sep 17 07:29:04.739198 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:04.739178 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/688b5736-013f-4a52-8145-0fb44f6eb438-metrics-tls\") pod \"dns-default-ctfdd\" (UID: \"688b5736-013f-4a52-8145-0fb44f6eb438\") " pod="openshift-dns/dns-default-ctfdd" Sep 17 07:29:04.739259 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:04.739189 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-tls\") pod \"image-registry-7484fc99c6-dxxgt\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:29:04.911988 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:04.911959 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:04.926380 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:04.926357 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cbg6t\"" Sep 17 07:29:04.934683 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:04.934665 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:29:04.958750 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:04.958729 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2jzn2\"" Sep 17 07:29:04.966187 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:04.966173 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ctfdd" Sep 17 07:29:05.243340 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:05.243308 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7484fc99c6-dxxgt"] Sep 17 07:29:05.247274 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:29:05.247242 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddab30fd6_0467_4b87_83a4_104b2fa7d30d.slice/crio-7f26e02adaa3a0f1f0b3172c7f6544cda949376b27479ad36d448c8b66c8d349 WatchSource:0}: Error finding container 7f26e02adaa3a0f1f0b3172c7f6544cda949376b27479ad36d448c8b66c8d349: Status 404 returned error can't find the container with id 7f26e02adaa3a0f1f0b3172c7f6544cda949376b27479ad36d448c8b66c8d349 Sep 17 07:29:05.269297 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:05.269274 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ctfdd"] Sep 17 07:29:05.273181 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:29:05.273140 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod688b5736_013f_4a52_8145_0fb44f6eb438.slice/crio-ea1efdd315073b77e95c939e1abcfb252dd224b4843e116b6aab7295760ccd93 WatchSource:0}: Error finding container ea1efdd315073b77e95c939e1abcfb252dd224b4843e116b6aab7295760ccd93: Status 404 returned error can't find the container with id ea1efdd315073b77e95c939e1abcfb252dd224b4843e116b6aab7295760ccd93 Sep 17 07:29:05.372377 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:05.372344 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tsc9z" event={"ID":"6a96dd04-5687-4059-bf3f-ee1864ed8e40","Type":"ContainerStarted","Data":"894b227cec35123e4ae2456b1f64bdd0cb443ebc5672261ae95ba1015e7b3b9d"} Sep 17 07:29:05.374208 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:05.374182 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ctfdd" event={"ID":"688b5736-013f-4a52-8145-0fb44f6eb438","Type":"ContainerStarted","Data":"ea1efdd315073b77e95c939e1abcfb252dd224b4843e116b6aab7295760ccd93"} Sep 17 07:29:05.376452 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:05.376386 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" event={"ID":"dab30fd6-0467-4b87-83a4-104b2fa7d30d","Type":"ContainerStarted","Data":"a8466143a3de36a0eedf94326d1f92ed66dc77bd1ce7aa17fe7dd50f9edf9e1b"} Sep 17 07:29:05.376452 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:05.376424 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" event={"ID":"dab30fd6-0467-4b87-83a4-104b2fa7d30d","Type":"ContainerStarted","Data":"7f26e02adaa3a0f1f0b3172c7f6544cda949376b27479ad36d448c8b66c8d349"} Sep 17 07:29:05.376689 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:05.376649 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:29:05.401190 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:05.401142 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" podStartSLOduration=44.401124747 podStartE2EDuration="44.401124747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:28:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:05.400179821 +0000 UTC m=+66.958364631" watchObservedRunningTime="2026-09-17 07:29:05.401124747 +0000 UTC m=+66.959309556" Sep 17 07:29:06.351892 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.351330 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-6tzzh\" (UID: \"7c06b4ad-93ab-43cc-a24a-9c758f3c957d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6tzzh" Sep 17 07:29:06.355008 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.354980 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c06b4ad-93ab-43cc-a24a-9c758f3c957d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-6tzzh\" (UID: \"7c06b4ad-93ab-43cc-a24a-9c758f3c957d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6tzzh" Sep 17 07:29:06.385909 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.385784 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xh2n2" event={"ID":"c35e7d45-1111-40a8-acf5-666b633e8180","Type":"ContainerStarted","Data":"2c6adb36d356a12200377e57a508515ac6f7e31330f00cce403437bcd35d95d7"} Sep 17 07:29:06.385909 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.385841 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xh2n2" event={"ID":"c35e7d45-1111-40a8-acf5-666b633e8180","Type":"ContainerStarted","Data":"9d03c40c35a33d48f851e598120e33b49e510b3af8d61e9cfca3607d89be7409"} Sep 17 07:29:06.406560 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.406502 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xh2n2" podStartSLOduration=65.781502055 podStartE2EDuration="1m7.406484128s" totalImagesPullingTime="1.624982073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:27:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:06.405670485 +0000 UTC m=+67.963855294" watchObservedRunningTime="2026-09-17 07:29:06.406484128 +0000 UTC m=+67.964668933" Sep 17 07:29:06.603590 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.603514 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-57fdcd854-xl4g6"] Sep 17 07:29:06.605745 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.605721 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57fdcd854-xl4g6" Sep 17 07:29:06.608374 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.608317 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 17 07:29:06.609354 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.609331 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 17 07:29:06.610965 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.610947 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 17 07:29:06.611896 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.611849 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zrnzm\"" Sep 17 07:29:06.618849 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.618820 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6tzzh" Sep 17 07:29:06.620257 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.620209 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 17 07:29:06.631014 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.630921 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-57fdcd854-xl4g6"] Sep 17 07:29:06.654540 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.654418 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f8c3e643-12a6-4dd1-90c2-947ae1e7a6f8-klusterlet-config\") pod \"klusterlet-addon-workmgr-57fdcd854-xl4g6\" (UID: \"f8c3e643-12a6-4dd1-90c2-947ae1e7a6f8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57fdcd854-xl4g6" Sep 17 07:29:06.654540 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.654483 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj557\" (UniqueName: \"kubernetes.io/projected/f8c3e643-12a6-4dd1-90c2-947ae1e7a6f8-kube-api-access-xj557\") pod \"klusterlet-addon-workmgr-57fdcd854-xl4g6\" (UID: \"f8c3e643-12a6-4dd1-90c2-947ae1e7a6f8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57fdcd854-xl4g6" Sep 17 07:29:06.654727 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.654556 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8c3e643-12a6-4dd1-90c2-947ae1e7a6f8-tmp\") pod \"klusterlet-addon-workmgr-57fdcd854-xl4g6\" (UID: \"f8c3e643-12a6-4dd1-90c2-947ae1e7a6f8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57fdcd854-xl4g6" Sep 17 07:29:06.755136 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.755104 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj557\" (UniqueName: \"kubernetes.io/projected/f8c3e643-12a6-4dd1-90c2-947ae1e7a6f8-kube-api-access-xj557\") pod \"klusterlet-addon-workmgr-57fdcd854-xl4g6\" (UID: \"f8c3e643-12a6-4dd1-90c2-947ae1e7a6f8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57fdcd854-xl4g6" Sep 17 07:29:06.755286 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.755167 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8c3e643-12a6-4dd1-90c2-947ae1e7a6f8-tmp\") pod \"klusterlet-addon-workmgr-57fdcd854-xl4g6\" (UID: \"f8c3e643-12a6-4dd1-90c2-947ae1e7a6f8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57fdcd854-xl4g6" Sep 17 07:29:06.755286 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.755217 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f8c3e643-12a6-4dd1-90c2-947ae1e7a6f8-klusterlet-config\") pod \"klusterlet-addon-workmgr-57fdcd854-xl4g6\" (UID: \"f8c3e643-12a6-4dd1-90c2-947ae1e7a6f8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57fdcd854-xl4g6" Sep 17 07:29:06.755680 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.755652 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8c3e643-12a6-4dd1-90c2-947ae1e7a6f8-tmp\") pod \"klusterlet-addon-workmgr-57fdcd854-xl4g6\" (UID: \"f8c3e643-12a6-4dd1-90c2-947ae1e7a6f8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57fdcd854-xl4g6" Sep 17 07:29:06.757971 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.757907 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f8c3e643-12a6-4dd1-90c2-947ae1e7a6f8-klusterlet-config\") pod \"klusterlet-addon-workmgr-57fdcd854-xl4g6\" (UID: \"f8c3e643-12a6-4dd1-90c2-947ae1e7a6f8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57fdcd854-xl4g6" Sep 17 07:29:06.783341 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.782282 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-jbx97"] Sep 17 07:29:06.785790 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.785772 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-jbx97" Sep 17 07:29:06.791490 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:29:06.791422 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-141-54.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-141-54.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Sep 17 07:29:06.791588 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:29:06.791548 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-141-54.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-141-54.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Sep 17 07:29:06.791660 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:29:06.791604 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-rqx59\" is forbidden: User \"system:node:ip-10-0-141-54.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-141-54.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"default-dockercfg-rqx59\"" type="*v1.Secret" Sep 17 07:29:06.791660 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:29:06.791622 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"downloads-76f99584f4-jbx97\" is forbidden: User \"system:node:ip-10-0-141-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-141-54.ec2.internal' and this object" podUID="5541ff6e-1169-489f-adf9-fe21aac2cd9a" pod="openshift-console/downloads-76f99584f4-jbx97" Sep 17 07:29:06.806491 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.806426 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-jbx97"] Sep 17 07:29:06.810801 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.810701 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj557\" (UniqueName: \"kubernetes.io/projected/f8c3e643-12a6-4dd1-90c2-947ae1e7a6f8-kube-api-access-xj557\") pod \"klusterlet-addon-workmgr-57fdcd854-xl4g6\" (UID: \"f8c3e643-12a6-4dd1-90c2-947ae1e7a6f8\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57fdcd854-xl4g6" Sep 17 07:29:06.855921 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.855855 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqx7n\" (UniqueName: \"kubernetes.io/projected/5541ff6e-1169-489f-adf9-fe21aac2cd9a-kube-api-access-nqx7n\") pod \"downloads-76f99584f4-jbx97\" (UID: \"5541ff6e-1169-489f-adf9-fe21aac2cd9a\") " pod="openshift-console/downloads-76f99584f4-jbx97" Sep 17 07:29:06.892867 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.892778 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:06.917914 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.917843 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57fdcd854-xl4g6" Sep 17 07:29:06.954547 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.954522 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-6tzzh"] Sep 17 07:29:06.956955 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:06.956924 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqx7n\" (UniqueName: \"kubernetes.io/projected/5541ff6e-1169-489f-adf9-fe21aac2cd9a-kube-api-access-nqx7n\") pod \"downloads-76f99584f4-jbx97\" (UID: \"5541ff6e-1169-489f-adf9-fe21aac2cd9a\") " pod="openshift-console/downloads-76f99584f4-jbx97" Sep 17 07:29:07.013487 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:07.013432 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:07.094551 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:07.094474 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:07.201507 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:07.201467 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:07.455739 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:29:07.455644 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c06b4ad_93ab_43cc_a24a_9c758f3c957d.slice/crio-181cf4e7aa5816ac5c54bfd4fe663908d094267cd3b8859b295c6fc3d608be98 WatchSource:0}: Error finding container 181cf4e7aa5816ac5c54bfd4fe663908d094267cd3b8859b295c6fc3d608be98: Status 404 returned error can't find the container with id 181cf4e7aa5816ac5c54bfd4fe663908d094267cd3b8859b295c6fc3d608be98 Sep 17 07:29:07.965137 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:29:07.965098 2553 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition Sep 17 07:29:07.975423 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:07.975387 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:07.993074 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:07.993051 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-rqx59\"" Sep 17 07:29:08.029913 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:08.029892 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 17 07:29:08.036126 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:29:08.036103 2553 projected.go:196] Error preparing data for projected volume kube-api-access-nqx7n for pod openshift-console/downloads-76f99584f4-jbx97: failed to sync configmap cache: timed out waiting for the condition Sep 17 07:29:08.036222 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:29:08.036177 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5541ff6e-1169-489f-adf9-fe21aac2cd9a-kube-api-access-nqx7n podName:5541ff6e-1169-489f-adf9-fe21aac2cd9a nodeName:}" failed. No retries permitted until 2026-09-17 07:29:08.536155546 +0000 UTC m=+70.094340403 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-nqx7n" (UniqueName: "kubernetes.io/projected/5541ff6e-1169-489f-adf9-fe21aac2cd9a-kube-api-access-nqx7n") pod "downloads-76f99584f4-jbx97" (UID: "5541ff6e-1169-489f-adf9-fe21aac2cd9a") : failed to sync configmap cache: timed out waiting for the condition Sep 17 07:29:08.215035 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:08.214989 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:08.241973 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:08.241911 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 17 07:29:08.277763 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:08.277728 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:08.395548 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:08.395515 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6tzzh" event={"ID":"7c06b4ad-93ab-43cc-a24a-9c758f3c957d","Type":"ContainerStarted","Data":"181cf4e7aa5816ac5c54bfd4fe663908d094267cd3b8859b295c6fc3d608be98"} Sep 17 07:29:08.571682 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:08.571589 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqx7n\" (UniqueName: \"kubernetes.io/projected/5541ff6e-1169-489f-adf9-fe21aac2cd9a-kube-api-access-nqx7n\") pod \"downloads-76f99584f4-jbx97\" (UID: \"5541ff6e-1169-489f-adf9-fe21aac2cd9a\") " pod="openshift-console/downloads-76f99584f4-jbx97" Sep 17 07:29:08.574698 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:08.574649 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqx7n\" (UniqueName: \"kubernetes.io/projected/5541ff6e-1169-489f-adf9-fe21aac2cd9a-kube-api-access-nqx7n\") pod \"downloads-76f99584f4-jbx97\" (UID: \"5541ff6e-1169-489f-adf9-fe21aac2cd9a\") " pod="openshift-console/downloads-76f99584f4-jbx97" Sep 17 07:29:08.598819 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:08.598757 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-jbx97" Sep 17 07:29:08.617547 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:08.617518 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:09.784464 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:09.784426 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:09.872540 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:09.872506 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:09.983592 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:09.983560 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:10.175739 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:10.175654 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-57fdcd854-xl4g6"] Sep 17 07:29:10.177209 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:10.177179 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-jbx97"] Sep 17 07:29:10.389292 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:29:10.389256 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5541ff6e_1169_489f_adf9_fe21aac2cd9a.slice/crio-bfeec2f1347a6f007af57e379311e0af964f3b84c1546b720492c4d9f31b1c11 WatchSource:0}: Error finding container bfeec2f1347a6f007af57e379311e0af964f3b84c1546b720492c4d9f31b1c11: Status 404 returned error can't find the container with id bfeec2f1347a6f007af57e379311e0af964f3b84c1546b720492c4d9f31b1c11 Sep 17 07:29:10.390459 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:29:10.390427 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8c3e643_12a6_4dd1_90c2_947ae1e7a6f8.slice/crio-760a5c7309d7ccfc65a95d830acf9a0abb8e17acf0d20271a9539057947fde80 WatchSource:0}: Error finding container 760a5c7309d7ccfc65a95d830acf9a0abb8e17acf0d20271a9539057947fde80: Status 404 returned error can't find the container with id 760a5c7309d7ccfc65a95d830acf9a0abb8e17acf0d20271a9539057947fde80 Sep 17 07:29:10.404498 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:10.404464 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-jbx97" event={"ID":"5541ff6e-1169-489f-adf9-fe21aac2cd9a","Type":"ContainerStarted","Data":"bfeec2f1347a6f007af57e379311e0af964f3b84c1546b720492c4d9f31b1c11"} Sep 17 07:29:10.405679 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:10.405659 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57fdcd854-xl4g6" event={"ID":"f8c3e643-12a6-4dd1-90c2-947ae1e7a6f8","Type":"ContainerStarted","Data":"760a5c7309d7ccfc65a95d830acf9a0abb8e17acf0d20271a9539057947fde80"} Sep 17 07:29:10.843388 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:10.843345 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:10.956423 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:10.956377 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:11.107605 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:11.107473 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:11.125095 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:11.125064 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-6rfjp"] Sep 17 07:29:11.142386 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:11.142337 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-6rfjp" Sep 17 07:29:11.142969 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:11.142390 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-6rfjp"] Sep 17 07:29:11.145990 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:11.145966 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 17 07:29:11.146478 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:11.146459 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-jvzlt\"" Sep 17 07:29:11.193362 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:11.193336 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c71c7342-234e-4b40-ac3c-f7579ee3ffc5-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-6rfjp\" (UID: \"c71c7342-234e-4b40-ac3c-f7579ee3ffc5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-6rfjp" Sep 17 07:29:11.294408 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:11.294382 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c71c7342-234e-4b40-ac3c-f7579ee3ffc5-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-6rfjp\" (UID: \"c71c7342-234e-4b40-ac3c-f7579ee3ffc5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-6rfjp" Sep 17 07:29:11.294546 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:29:11.294531 2553 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Sep 17 07:29:11.294608 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:29:11.294598 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c71c7342-234e-4b40-ac3c-f7579ee3ffc5-tls-certificates podName:c71c7342-234e-4b40-ac3c-f7579ee3ffc5 nodeName:}" failed. No retries permitted until 2026-09-17 07:29:11.794578112 +0000 UTC m=+73.352762906 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/c71c7342-234e-4b40-ac3c-f7579ee3ffc5-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-6rfjp" (UID: "c71c7342-234e-4b40-ac3c-f7579ee3ffc5") : secret "prometheus-operator-admission-webhook-tls" not found Sep 17 07:29:11.413652 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:11.413580 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gh9zp" event={"ID":"f68f4c1a-7946-43c1-80ca-b472129b2bfe","Type":"ContainerStarted","Data":"856a920bde3bffebb8e92685f8db15787a3615258ffcab7f81b77ed645fec1dd"} Sep 17 07:29:11.416316 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:11.416283 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tsc9z" event={"ID":"6a96dd04-5687-4059-bf3f-ee1864ed8e40","Type":"ContainerStarted","Data":"2fcdbe734af338433f63e6250b6642519ad1058aae1147cc84464751d3465648"} Sep 17 07:29:11.416726 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:11.416705 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:29:11.421760 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:11.421254 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ctfdd" event={"ID":"688b5736-013f-4a52-8145-0fb44f6eb438","Type":"ContainerStarted","Data":"dc0ba45eb6417c22f893c0e84f8e223d56de7eb9e6d78749c579619781e92eb9"} Sep 17 07:29:11.421760 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:11.421298 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ctfdd" event={"ID":"688b5736-013f-4a52-8145-0fb44f6eb438","Type":"ContainerStarted","Data":"28ef8e40095b9624063c980de12b1dc59b1dab3a1ca20b2a7c3082aea6f5d966"} Sep 17 07:29:11.421760 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:11.421712 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ctfdd" Sep 17 07:29:11.424133 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:11.423725 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6tzzh" event={"ID":"7c06b4ad-93ab-43cc-a24a-9c758f3c957d","Type":"ContainerStarted","Data":"ae9539107701aaa607ead26cb410aacd27f5ae860e86cb7f9869c106e7c77b09"} Sep 17 07:29:11.433451 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:11.431966 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-gh9zp" podStartSLOduration=65.277135757 podStartE2EDuration="1m11.43194905s" totalImagesPullingTime="6.154813293s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:28:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:11.431636045 +0000 UTC m=+72.989820852" watchObservedRunningTime="2026-09-17 07:29:11.43194905 +0000 UTC m=+72.990133859" Sep 17 07:29:11.451395 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:11.449588 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-tsc9z" podStartSLOduration=65.505462083 podStartE2EDuration="1m11.449571385s" totalImagesPullingTime="5.944109302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:28:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:11.449500689 +0000 UTC m=+73.007685498" watchObservedRunningTime="2026-09-17 07:29:11.449571385 +0000 UTC m=+73.007756222" Sep 17 07:29:11.469528 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:11.469477 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ctfdd" podStartSLOduration=34.851634326 podStartE2EDuration="39.469460932s" totalImagesPullingTime="4.617826606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:28:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:11.469164338 +0000 UTC m=+73.027349147" watchObservedRunningTime="2026-09-17 07:29:11.469460932 +0000 UTC m=+73.027645742" Sep 17 07:29:11.493369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:11.492993 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6tzzh" podStartSLOduration=34.527510808 podStartE2EDuration="37.492974846s" totalImagesPullingTime="2.965464038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:11.492153285 +0000 UTC m=+73.050338094" watchObservedRunningTime="2026-09-17 07:29:11.492974846 +0000 UTC m=+73.051159656" Sep 17 07:29:11.683470 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:11.683293 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:11.798197 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:11.798092 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c71c7342-234e-4b40-ac3c-f7579ee3ffc5-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-6rfjp\" (UID: \"c71c7342-234e-4b40-ac3c-f7579ee3ffc5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-6rfjp" Sep 17 07:29:11.801135 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:11.801093 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c71c7342-234e-4b40-ac3c-f7579ee3ffc5-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-6rfjp\" (UID: \"c71c7342-234e-4b40-ac3c-f7579ee3ffc5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-6rfjp" Sep 17 07:29:12.054077 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:12.054042 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-6rfjp" Sep 17 07:29:12.367450 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:12.367175 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-6rfjp"] Sep 17 07:29:13.141341 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:29:13.141300 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc71c7342_234e_4b40_ac3c_f7579ee3ffc5.slice/crio-b09bb9d95c7c84498848e2a27431b1afbbe5a76bf9077414791ec20d02a24af8 WatchSource:0}: Error finding container b09bb9d95c7c84498848e2a27431b1afbbe5a76bf9077414791ec20d02a24af8: Status 404 returned error can't find the container with id b09bb9d95c7c84498848e2a27431b1afbbe5a76bf9077414791ec20d02a24af8 Sep 17 07:29:13.439423 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:13.439341 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-6rfjp" event={"ID":"c71c7342-234e-4b40-ac3c-f7579ee3ffc5","Type":"ContainerStarted","Data":"b09bb9d95c7c84498848e2a27431b1afbbe5a76bf9077414791ec20d02a24af8"} Sep 17 07:29:14.229035 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:14.228992 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:14.328139 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:14.328097 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:15.451287 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:15.451190 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57fdcd854-xl4g6" event={"ID":"f8c3e643-12a6-4dd1-90c2-947ae1e7a6f8","Type":"ContainerStarted","Data":"329e7cc50d1b5135be5ba9db871ebde368f3f598563f3873d0b970170a5b48eb"} Sep 17 07:29:15.451769 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:15.451474 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57fdcd854-xl4g6" Sep 17 07:29:15.453562 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:15.453539 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57fdcd854-xl4g6" Sep 17 07:29:15.509802 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:15.509764 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57fdcd854-xl4g6" podStartSLOduration=5.051457575 podStartE2EDuration="9.509747982s" totalImagesPullingTime="4.458290407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:29:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:15.474936431 +0000 UTC m=+77.033121239" watchObservedRunningTime="2026-09-17 07:29:15.509747982 +0000 UTC m=+77.067932789" Sep 17 07:29:16.457336 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.457299 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-6rfjp" event={"ID":"c71c7342-234e-4b40-ac3c-f7579ee3ffc5","Type":"ContainerStarted","Data":"550950eac72f4473e09ad37c3e38fae6ee1db66d1987966b1926e3b26f6cea46"} Sep 17 07:29:16.457800 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.457597 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-6rfjp" Sep 17 07:29:16.463098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.463064 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-6rfjp" Sep 17 07:29:16.480861 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.480804 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-6rfjp" podStartSLOduration=3.117494187 podStartE2EDuration="5.480788127s" totalImagesPullingTime="2.36329394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:29:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:16.479162281 +0000 UTC m=+78.037347091" watchObservedRunningTime="2026-09-17 07:29:16.480788127 +0000 UTC m=+78.038972932" Sep 17 07:29:16.524726 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.524698 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:16.553644 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.553618 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-65d5f4bdbb-6xr4f"] Sep 17 07:29:16.555773 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.555756 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65d5f4bdbb-6xr4f" Sep 17 07:29:16.558506 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.558483 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 17 07:29:16.558619 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.558585 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 17 07:29:16.558696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.558631 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 17 07:29:16.558808 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.558790 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 17 07:29:16.558949 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.558794 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 17 07:29:16.558949 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.558856 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-sqrn6\"" Sep 17 07:29:16.570832 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.570813 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65d5f4bdbb-6xr4f"] Sep 17 07:29:16.596482 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.596455 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:16.634867 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.634842 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/945229a2-9595-4ac3-901c-bfa0576f6140-oauth-serving-cert\") pod \"console-65d5f4bdbb-6xr4f\" (UID: \"945229a2-9595-4ac3-901c-bfa0576f6140\") " pod="openshift-console/console-65d5f4bdbb-6xr4f" Sep 17 07:29:16.634947 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.634892 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/945229a2-9595-4ac3-901c-bfa0576f6140-console-oauth-config\") pod \"console-65d5f4bdbb-6xr4f\" (UID: \"945229a2-9595-4ac3-901c-bfa0576f6140\") " pod="openshift-console/console-65d5f4bdbb-6xr4f" Sep 17 07:29:16.634947 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.634909 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-286mz\" (UniqueName: \"kubernetes.io/projected/945229a2-9595-4ac3-901c-bfa0576f6140-kube-api-access-286mz\") pod \"console-65d5f4bdbb-6xr4f\" (UID: \"945229a2-9595-4ac3-901c-bfa0576f6140\") " pod="openshift-console/console-65d5f4bdbb-6xr4f" Sep 17 07:29:16.635025 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.634988 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/945229a2-9595-4ac3-901c-bfa0576f6140-console-serving-cert\") pod \"console-65d5f4bdbb-6xr4f\" (UID: \"945229a2-9595-4ac3-901c-bfa0576f6140\") " pod="openshift-console/console-65d5f4bdbb-6xr4f" Sep 17 07:29:16.635025 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.635015 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/945229a2-9595-4ac3-901c-bfa0576f6140-console-config\") pod \"console-65d5f4bdbb-6xr4f\" (UID: \"945229a2-9595-4ac3-901c-bfa0576f6140\") " pod="openshift-console/console-65d5f4bdbb-6xr4f" Sep 17 07:29:16.635087 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.635038 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/945229a2-9595-4ac3-901c-bfa0576f6140-service-ca\") pod \"console-65d5f4bdbb-6xr4f\" (UID: \"945229a2-9595-4ac3-901c-bfa0576f6140\") " pod="openshift-console/console-65d5f4bdbb-6xr4f" Sep 17 07:29:16.735463 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.735394 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/945229a2-9595-4ac3-901c-bfa0576f6140-oauth-serving-cert\") pod \"console-65d5f4bdbb-6xr4f\" (UID: \"945229a2-9595-4ac3-901c-bfa0576f6140\") " pod="openshift-console/console-65d5f4bdbb-6xr4f" Sep 17 07:29:16.735463 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.735456 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/945229a2-9595-4ac3-901c-bfa0576f6140-console-oauth-config\") pod \"console-65d5f4bdbb-6xr4f\" (UID: \"945229a2-9595-4ac3-901c-bfa0576f6140\") " pod="openshift-console/console-65d5f4bdbb-6xr4f" Sep 17 07:29:16.735616 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.735484 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-286mz\" (UniqueName: \"kubernetes.io/projected/945229a2-9595-4ac3-901c-bfa0576f6140-kube-api-access-286mz\") pod \"console-65d5f4bdbb-6xr4f\" (UID: \"945229a2-9595-4ac3-901c-bfa0576f6140\") " pod="openshift-console/console-65d5f4bdbb-6xr4f" Sep 17 07:29:16.735616 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.735529 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/945229a2-9595-4ac3-901c-bfa0576f6140-console-serving-cert\") pod \"console-65d5f4bdbb-6xr4f\" (UID: \"945229a2-9595-4ac3-901c-bfa0576f6140\") " pod="openshift-console/console-65d5f4bdbb-6xr4f" Sep 17 07:29:16.735616 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.735560 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/945229a2-9595-4ac3-901c-bfa0576f6140-console-config\") pod \"console-65d5f4bdbb-6xr4f\" (UID: \"945229a2-9595-4ac3-901c-bfa0576f6140\") " pod="openshift-console/console-65d5f4bdbb-6xr4f" Sep 17 07:29:16.735616 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.735584 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/945229a2-9595-4ac3-901c-bfa0576f6140-service-ca\") pod \"console-65d5f4bdbb-6xr4f\" (UID: \"945229a2-9595-4ac3-901c-bfa0576f6140\") " pod="openshift-console/console-65d5f4bdbb-6xr4f" Sep 17 07:29:16.736276 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.736253 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/945229a2-9595-4ac3-901c-bfa0576f6140-oauth-serving-cert\") pod \"console-65d5f4bdbb-6xr4f\" (UID: \"945229a2-9595-4ac3-901c-bfa0576f6140\") " pod="openshift-console/console-65d5f4bdbb-6xr4f" Sep 17 07:29:16.736276 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.736267 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/945229a2-9595-4ac3-901c-bfa0576f6140-service-ca\") pod \"console-65d5f4bdbb-6xr4f\" (UID: \"945229a2-9595-4ac3-901c-bfa0576f6140\") " pod="openshift-console/console-65d5f4bdbb-6xr4f" Sep 17 07:29:16.736457 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.736433 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/945229a2-9595-4ac3-901c-bfa0576f6140-console-config\") pod \"console-65d5f4bdbb-6xr4f\" (UID: \"945229a2-9595-4ac3-901c-bfa0576f6140\") " pod="openshift-console/console-65d5f4bdbb-6xr4f" Sep 17 07:29:16.738408 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.738388 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/945229a2-9595-4ac3-901c-bfa0576f6140-console-oauth-config\") pod \"console-65d5f4bdbb-6xr4f\" (UID: \"945229a2-9595-4ac3-901c-bfa0576f6140\") " pod="openshift-console/console-65d5f4bdbb-6xr4f" Sep 17 07:29:16.738595 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.738575 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/945229a2-9595-4ac3-901c-bfa0576f6140-console-serving-cert\") pod \"console-65d5f4bdbb-6xr4f\" (UID: \"945229a2-9595-4ac3-901c-bfa0576f6140\") " pod="openshift-console/console-65d5f4bdbb-6xr4f" Sep 17 07:29:16.752620 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.752562 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-286mz\" (UniqueName: \"kubernetes.io/projected/945229a2-9595-4ac3-901c-bfa0576f6140-kube-api-access-286mz\") pod \"console-65d5f4bdbb-6xr4f\" (UID: \"945229a2-9595-4ac3-901c-bfa0576f6140\") " pod="openshift-console/console-65d5f4bdbb-6xr4f" Sep 17 07:29:16.776098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.776059 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:16.865611 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.865589 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65d5f4bdbb-6xr4f" Sep 17 07:29:16.874827 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.874799 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:16.934063 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:16.934030 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:17.156507 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.156466 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65d5f4bdbb-6xr4f"] Sep 17 07:29:17.157327 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:29:17.157299 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod945229a2_9595_4ac3_901c_bfa0576f6140.slice/crio-6646076dd7dd8cf21f9a62a3b4bd7397c26b92a308b0ec765a73feaf6723e154 WatchSource:0}: Error finding container 6646076dd7dd8cf21f9a62a3b4bd7397c26b92a308b0ec765a73feaf6723e154: Status 404 returned error can't find the container with id 6646076dd7dd8cf21f9a62a3b4bd7397c26b92a308b0ec765a73feaf6723e154 Sep 17 07:29:17.196896 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.196855 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:17.269253 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.269226 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-xngr7"] Sep 17 07:29:17.272324 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.272301 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-xngr7" Sep 17 07:29:17.278645 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.278622 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 17 07:29:17.278765 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.278623 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-rrcg8\"" Sep 17 07:29:17.278828 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.278785 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 17 07:29:17.278942 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.278926 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 17 07:29:17.295133 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.295105 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-xngr7"] Sep 17 07:29:17.314019 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.313927 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:17.341817 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.341793 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8jzw\" (UniqueName: \"kubernetes.io/projected/ff6d3938-9c3b-4629-82ed-83fe088836d8-kube-api-access-m8jzw\") pod \"prometheus-operator-68bcd7f987-xngr7\" (UID: \"ff6d3938-9c3b-4629-82ed-83fe088836d8\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-xngr7" Sep 17 07:29:17.341954 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.341840 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff6d3938-9c3b-4629-82ed-83fe088836d8-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-xngr7\" (UID: \"ff6d3938-9c3b-4629-82ed-83fe088836d8\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-xngr7" Sep 17 07:29:17.341954 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.341868 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff6d3938-9c3b-4629-82ed-83fe088836d8-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-xngr7\" (UID: \"ff6d3938-9c3b-4629-82ed-83fe088836d8\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-xngr7" Sep 17 07:29:17.342083 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.341991 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff6d3938-9c3b-4629-82ed-83fe088836d8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-xngr7\" (UID: \"ff6d3938-9c3b-4629-82ed-83fe088836d8\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-xngr7" Sep 17 07:29:17.442996 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.442930 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff6d3938-9c3b-4629-82ed-83fe088836d8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-xngr7\" (UID: \"ff6d3938-9c3b-4629-82ed-83fe088836d8\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-xngr7" Sep 17 07:29:17.442996 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.442985 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8jzw\" (UniqueName: \"kubernetes.io/projected/ff6d3938-9c3b-4629-82ed-83fe088836d8-kube-api-access-m8jzw\") pod \"prometheus-operator-68bcd7f987-xngr7\" (UID: \"ff6d3938-9c3b-4629-82ed-83fe088836d8\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-xngr7" Sep 17 07:29:17.443162 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.443021 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff6d3938-9c3b-4629-82ed-83fe088836d8-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-xngr7\" (UID: \"ff6d3938-9c3b-4629-82ed-83fe088836d8\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-xngr7" Sep 17 07:29:17.443162 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.443041 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff6d3938-9c3b-4629-82ed-83fe088836d8-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-xngr7\" (UID: \"ff6d3938-9c3b-4629-82ed-83fe088836d8\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-xngr7" Sep 17 07:29:17.443377 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:29:17.443358 2553 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Sep 17 07:29:17.443458 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:29:17.443440 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff6d3938-9c3b-4629-82ed-83fe088836d8-prometheus-operator-tls podName:ff6d3938-9c3b-4629-82ed-83fe088836d8 nodeName:}" failed. No retries permitted until 2026-09-17 07:29:17.943420211 +0000 UTC m=+79.501605001 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/ff6d3938-9c3b-4629-82ed-83fe088836d8-prometheus-operator-tls") pod "prometheus-operator-68bcd7f987-xngr7" (UID: "ff6d3938-9c3b-4629-82ed-83fe088836d8") : secret "prometheus-operator-tls" not found Sep 17 07:29:17.443761 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.443740 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff6d3938-9c3b-4629-82ed-83fe088836d8-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-xngr7\" (UID: \"ff6d3938-9c3b-4629-82ed-83fe088836d8\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-xngr7" Sep 17 07:29:17.445927 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.445904 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff6d3938-9c3b-4629-82ed-83fe088836d8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-xngr7\" (UID: \"ff6d3938-9c3b-4629-82ed-83fe088836d8\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-xngr7" Sep 17 07:29:17.452936 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.452912 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8jzw\" (UniqueName: \"kubernetes.io/projected/ff6d3938-9c3b-4629-82ed-83fe088836d8-kube-api-access-m8jzw\") pod \"prometheus-operator-68bcd7f987-xngr7\" (UID: \"ff6d3938-9c3b-4629-82ed-83fe088836d8\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-xngr7" Sep 17 07:29:17.462849 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.462819 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65d5f4bdbb-6xr4f" event={"ID":"945229a2-9595-4ac3-901c-bfa0576f6140","Type":"ContainerStarted","Data":"6646076dd7dd8cf21f9a62a3b4bd7397c26b92a308b0ec765a73feaf6723e154"} Sep 17 07:29:17.588817 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.588782 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:17.946737 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.946705 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff6d3938-9c3b-4629-82ed-83fe088836d8-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-xngr7\" (UID: \"ff6d3938-9c3b-4629-82ed-83fe088836d8\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-xngr7" Sep 17 07:29:17.949330 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:17.949303 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff6d3938-9c3b-4629-82ed-83fe088836d8-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-xngr7\" (UID: \"ff6d3938-9c3b-4629-82ed-83fe088836d8\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-xngr7" Sep 17 07:29:18.184074 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.184040 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-xngr7" Sep 17 07:29:18.495491 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.495441 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-xngr7"] Sep 17 07:29:18.498555 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:29:18.498526 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff6d3938_9c3b_4629_82ed_83fe088836d8.slice/crio-f3babb3091f9faa072b599bf63a74829ea9800b4b06f7229f0420b6c8591dd1e WatchSource:0}: Error finding container f3babb3091f9faa072b599bf63a74829ea9800b4b06f7229f0420b6c8591dd1e: Status 404 returned error can't find the container with id f3babb3091f9faa072b599bf63a74829ea9800b4b06f7229f0420b6c8591dd1e Sep 17 07:29:18.728050 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.727975 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:18.756472 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.756404 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-fbfb846df-w65xg"] Sep 17 07:29:18.759903 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.759690 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:18.774736 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.774626 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 17 07:29:18.777193 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.777168 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fbfb846df-w65xg"] Sep 17 07:29:18.854125 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.854098 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-console-oauth-config\") pod \"console-fbfb846df-w65xg\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:18.854261 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.854138 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dhmw\" (UniqueName: \"kubernetes.io/projected/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-kube-api-access-5dhmw\") pod \"console-fbfb846df-w65xg\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:18.854261 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.854171 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-console-serving-cert\") pod \"console-fbfb846df-w65xg\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:18.854261 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.854241 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-oauth-serving-cert\") pod \"console-fbfb846df-w65xg\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:18.854422 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.854316 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-console-config\") pod \"console-fbfb846df-w65xg\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:18.854422 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.854348 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-trusted-ca-bundle\") pod \"console-fbfb846df-w65xg\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:18.854422 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.854380 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-service-ca\") pod \"console-fbfb846df-w65xg\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:18.955583 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.955555 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-console-oauth-config\") pod \"console-fbfb846df-w65xg\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:18.955731 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.955596 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dhmw\" (UniqueName: \"kubernetes.io/projected/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-kube-api-access-5dhmw\") pod \"console-fbfb846df-w65xg\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:18.955731 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.955623 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-console-serving-cert\") pod \"console-fbfb846df-w65xg\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:18.955731 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.955647 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-oauth-serving-cert\") pod \"console-fbfb846df-w65xg\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:18.955731 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.955693 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-console-config\") pod \"console-fbfb846df-w65xg\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:18.955731 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.955718 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-trusted-ca-bundle\") pod \"console-fbfb846df-w65xg\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:18.955995 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.955748 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-service-ca\") pod \"console-fbfb846df-w65xg\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:18.957910 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.956503 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-service-ca\") pod \"console-fbfb846df-w65xg\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:18.957910 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.957072 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-console-config\") pod \"console-fbfb846df-w65xg\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:18.957910 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.957389 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-trusted-ca-bundle\") pod \"console-fbfb846df-w65xg\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:18.957910 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.957606 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-oauth-serving-cert\") pod \"console-fbfb846df-w65xg\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:18.960167 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.960147 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-console-serving-cert\") pod \"console-fbfb846df-w65xg\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:18.960376 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.960353 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-console-oauth-config\") pod \"console-fbfb846df-w65xg\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:18.968101 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:18.968080 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dhmw\" (UniqueName: \"kubernetes.io/projected/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-kube-api-access-5dhmw\") pod \"console-fbfb846df-w65xg\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:19.073369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:19.073309 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:19.113420 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:19.113374 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:19.215516 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:19.215464 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:19.363268 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:19.363153 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fbfb846df-w65xg"] Sep 17 07:29:19.474673 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:19.474639 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fbfb846df-w65xg" event={"ID":"082ae202-9e8d-4e87-9ab7-1216e78a5f5c","Type":"ContainerStarted","Data":"b358b2ef4f5ace213a5c2fb92ad1331806c91fe89b3ba67c5da536da3a6d6a7c"} Sep 17 07:29:19.476451 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:19.476423 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-xngr7" event={"ID":"ff6d3938-9c3b-4629-82ed-83fe088836d8","Type":"ContainerStarted","Data":"f3babb3091f9faa072b599bf63a74829ea9800b4b06f7229f0420b6c8591dd1e"} Sep 17 07:29:22.442515 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:22.442397 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ctfdd" Sep 17 07:29:23.308929 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:23.308894 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:24.939959 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:24.939914 2553 patch_prober.go:28] interesting pod/image-registry-7484fc99c6-dxxgt container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 17 07:29:24.940391 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:24.939985 2553 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" podUID="dab30fd6-0467-4b87-83a4-104b2fa7d30d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 07:29:26.390664 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:26.390634 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:29:26.639171 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:26.639134 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:26.741226 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:26.741181 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:28.840738 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:28.840622 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:28.892655 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:28.892622 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7484fc99c6-dxxgt"] Sep 17 07:29:29.017757 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:29.017722 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:29.079369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:29.079290 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:29.517897 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:29.517828 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65d5f4bdbb-6xr4f" event={"ID":"945229a2-9595-4ac3-901c-bfa0576f6140","Type":"ContainerStarted","Data":"ceccb0e9f95e8e8fff0c465e8376780e079dae1ab7faee8769b47cb41a51f50d"} Sep 17 07:29:29.519990 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:29.519958 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fbfb846df-w65xg" event={"ID":"082ae202-9e8d-4e87-9ab7-1216e78a5f5c","Type":"ContainerStarted","Data":"afdca5c29c79bfd2740e21aa5df446046b41de5ac6a042fa3429e6fbd7df33f1"} Sep 17 07:29:29.522313 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:29.522286 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-xngr7" event={"ID":"ff6d3938-9c3b-4629-82ed-83fe088836d8","Type":"ContainerStarted","Data":"2196921348c9cade327381497abe1a84afab62ae551b02ca1dd1d56d66112543"} Sep 17 07:29:29.522444 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:29.522334 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-xngr7" event={"ID":"ff6d3938-9c3b-4629-82ed-83fe088836d8","Type":"ContainerStarted","Data":"94dbebc364755a34c0c347fabe016ff0b22c85a1485d0e14f9ce4845ebc20298"} Sep 17 07:29:29.524341 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:29.524313 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-jbx97" event={"ID":"5541ff6e-1169-489f-adf9-fe21aac2cd9a","Type":"ContainerStarted","Data":"222f410a99eb36fe0f597fbdb519e0e209140998111ba102d2d6c5ed9fd22d57"} Sep 17 07:29:29.524527 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:29.524499 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-jbx97" Sep 17 07:29:29.539224 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:29.539204 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-jbx97" Sep 17 07:29:29.553383 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:29.553335 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-65d5f4bdbb-6xr4f" podStartSLOduration=1.923231776 podStartE2EDuration="13.553316789s" totalImagesPullingTime="11.630085013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:29.551795502 +0000 UTC m=+91.109980311" watchObservedRunningTime="2026-09-17 07:29:29.553316789 +0000 UTC m=+91.111501598" Sep 17 07:29:29.575298 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:29.575248 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-jbx97" podStartSLOduration=5.42519737 podStartE2EDuration="23.575230438s" totalImagesPullingTime="18.150033068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:29:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:29.573341182 +0000 UTC m=+91.131525983" watchObservedRunningTime="2026-09-17 07:29:29.575230438 +0000 UTC m=+91.133415267" Sep 17 07:29:29.599050 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:29.598760 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-xngr7" podStartSLOduration=2.6041964909999997 podStartE2EDuration="12.598742239s" totalImagesPullingTime="9.994545748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:29:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:29.597388019 +0000 UTC m=+91.155572828" watchObservedRunningTime="2026-09-17 07:29:29.598742239 +0000 UTC m=+91.156927051" Sep 17 07:29:29.623551 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:29.623503 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-fbfb846df-w65xg" podStartSLOduration=2.28674856 podStartE2EDuration="11.623484305s" totalImagesPullingTime="9.336735745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:29:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:29.623463051 +0000 UTC m=+91.181647861" watchObservedRunningTime="2026-09-17 07:29:29.623484305 +0000 UTC m=+91.181669114" Sep 17 07:29:30.135417 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:30.135374 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:31.424475 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.424444 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:31.681805 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.681720 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:31.753549 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.753520 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wkvwb"] Sep 17 07:29:31.777308 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.777282 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:31.778100 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.778075 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.781974 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.781947 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 17 07:29:31.782771 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.782754 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 17 07:29:31.783226 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.783103 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 17 07:29:31.783984 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.783155 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gnwph\"" Sep 17 07:29:31.871166 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.871135 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ee598350-452c-4087-9df4-729094cfd671-node-exporter-textfile\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.871304 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.871276 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ee598350-452c-4087-9df4-729094cfd671-root\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.871371 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.871307 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lkhb\" (UniqueName: \"kubernetes.io/projected/ee598350-452c-4087-9df4-729094cfd671-kube-api-access-5lkhb\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.871371 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.871360 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee598350-452c-4087-9df4-729094cfd671-sys\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.871474 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.871402 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee598350-452c-4087-9df4-729094cfd671-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.871474 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.871433 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ee598350-452c-4087-9df4-729094cfd671-node-exporter-wtmp\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.871474 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.871467 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ee598350-452c-4087-9df4-729094cfd671-node-exporter-tls\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.871623 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.871509 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee598350-452c-4087-9df4-729094cfd671-metrics-client-ca\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.871623 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.871533 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ee598350-452c-4087-9df4-729094cfd671-node-exporter-accelerators-collector-config\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.972871 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.972834 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee598350-452c-4087-9df4-729094cfd671-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.973166 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.973148 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ee598350-452c-4087-9df4-729094cfd671-node-exporter-wtmp\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.973300 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.973287 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ee598350-452c-4087-9df4-729094cfd671-node-exporter-tls\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.973425 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.973413 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee598350-452c-4087-9df4-729094cfd671-metrics-client-ca\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.973530 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.973517 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ee598350-452c-4087-9df4-729094cfd671-node-exporter-accelerators-collector-config\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.973646 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.973634 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ee598350-452c-4087-9df4-729094cfd671-node-exporter-textfile\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.973756 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.973743 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ee598350-452c-4087-9df4-729094cfd671-root\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.973854 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.973843 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lkhb\" (UniqueName: \"kubernetes.io/projected/ee598350-452c-4087-9df4-729094cfd671-kube-api-access-5lkhb\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.974016 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.973993 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee598350-452c-4087-9df4-729094cfd671-sys\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.974193 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.974181 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee598350-452c-4087-9df4-729094cfd671-sys\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.974906 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.974858 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ee598350-452c-4087-9df4-729094cfd671-root\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.975459 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.975146 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ee598350-452c-4087-9df4-729094cfd671-node-exporter-wtmp\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.975902 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.975873 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ee598350-452c-4087-9df4-729094cfd671-node-exporter-textfile\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.976968 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.976943 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee598350-452c-4087-9df4-729094cfd671-metrics-client-ca\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.977588 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.977570 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ee598350-452c-4087-9df4-729094cfd671-node-exporter-accelerators-collector-config\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.979620 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.979601 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee598350-452c-4087-9df4-729094cfd671-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:31.988537 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:31.988513 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ee598350-452c-4087-9df4-729094cfd671-node-exporter-tls\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:32.001336 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:32.001317 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lkhb\" (UniqueName: \"kubernetes.io/projected/ee598350-452c-4087-9df4-729094cfd671-kube-api-access-5lkhb\") pod \"node-exporter-wkvwb\" (UID: \"ee598350-452c-4087-9df4-729094cfd671\") " pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:32.016681 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:32.016654 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:32.092693 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:32.092669 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wkvwb" Sep 17 07:29:32.112992 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:32.112968 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:32.137690 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:29:32.137660 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee598350_452c_4087_9df4_729094cfd671.slice/crio-eae25690524cfc34a5af10424061c8c7a9ebeb69bbeee395caaee1b0d4a0c147 WatchSource:0}: Error finding container eae25690524cfc34a5af10424061c8c7a9ebeb69bbeee395caaee1b0d4a0c147: Status 404 returned error can't find the container with id eae25690524cfc34a5af10424061c8c7a9ebeb69bbeee395caaee1b0d4a0c147 Sep 17 07:29:32.209123 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:32.209089 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:32.313577 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:32.313492 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:32.410372 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:32.410301 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:32.501918 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:32.501868 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:32.539207 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:32.539173 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wkvwb" event={"ID":"ee598350-452c-4087-9df4-729094cfd671","Type":"ContainerStarted","Data":"eae25690524cfc34a5af10424061c8c7a9ebeb69bbeee395caaee1b0d4a0c147"} Sep 17 07:29:32.568507 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:32.568439 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:32.685456 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:32.684475 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:32.811354 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:32.811088 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:33.417620 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:33.417579 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:33.517533 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:33.517498 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:34.549349 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:34.549317 2553 generic.go:356] "Generic (PLEG): container finished" podID="ee598350-452c-4087-9df4-729094cfd671" containerID="ea465a48abea60fc282909a233bdd7b686af1fdedddcc26c5b7fa1a1759225cd" exitCode=0 Sep 17 07:29:34.549867 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:34.549412 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wkvwb" event={"ID":"ee598350-452c-4087-9df4-729094cfd671","Type":"ContainerDied","Data":"ea465a48abea60fc282909a233bdd7b686af1fdedddcc26c5b7fa1a1759225cd"} Sep 17 07:29:34.723156 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:34.723131 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:34.826367 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:34.826326 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:34.899575 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:34.899542 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:35.131135 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:35.131053 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:35.556430 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:35.556395 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wkvwb" event={"ID":"ee598350-452c-4087-9df4-729094cfd671","Type":"ContainerStarted","Data":"fc1ec21196060185a302c82bac04035170da20f635ed7431538ca10f3da2de14"} Sep 17 07:29:35.556828 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:35.556445 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wkvwb" event={"ID":"ee598350-452c-4087-9df4-729094cfd671","Type":"ContainerStarted","Data":"9a61701fb393e53797d962681f67a922e251001434d9b2fd7f444e17618b942e"} Sep 17 07:29:35.580146 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:35.580098 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wkvwb" podStartSLOduration=2.806836092 podStartE2EDuration="4.58008341s" totalImagesPullingTime="1.684438194s" totalInitContainerRuntime="88.809124ms" isStatefulPod=true podCreationTimestamp="2026-09-17 07:29:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:35.578209639 +0000 UTC m=+97.136394459" watchObservedRunningTime="2026-09-17 07:29:35.58008341 +0000 UTC m=+97.138268218" Sep 17 07:29:36.000232 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.000200 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:36.061895 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.061849 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:36.453820 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.453703 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:36.462664 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.462627 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65d5f4bdbb-6xr4f"] Sep 17 07:29:36.499056 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.499030 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-955b7476f-ssrp8"] Sep 17 07:29:36.545493 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.545462 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-955b7476f-ssrp8"] Sep 17 07:29:36.545645 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.545570 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:29:36.546109 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.546083 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:36.711396 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.711317 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/952cf7c3-4861-43ab-b200-860136cd2ba7-console-serving-cert\") pod \"console-955b7476f-ssrp8\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:29:36.711396 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.711365 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/952cf7c3-4861-43ab-b200-860136cd2ba7-service-ca\") pod \"console-955b7476f-ssrp8\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:29:36.711857 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.711630 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/952cf7c3-4861-43ab-b200-860136cd2ba7-oauth-serving-cert\") pod \"console-955b7476f-ssrp8\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:29:36.711857 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.711704 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/952cf7c3-4861-43ab-b200-860136cd2ba7-trusted-ca-bundle\") pod \"console-955b7476f-ssrp8\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:29:36.711857 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.711730 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frqzn\" (UniqueName: \"kubernetes.io/projected/952cf7c3-4861-43ab-b200-860136cd2ba7-kube-api-access-frqzn\") pod \"console-955b7476f-ssrp8\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:29:36.711857 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.711804 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/952cf7c3-4861-43ab-b200-860136cd2ba7-console-oauth-config\") pod \"console-955b7476f-ssrp8\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:29:36.712077 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.711855 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/952cf7c3-4861-43ab-b200-860136cd2ba7-console-config\") pod \"console-955b7476f-ssrp8\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:29:36.812562 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.812524 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/952cf7c3-4861-43ab-b200-860136cd2ba7-trusted-ca-bundle\") pod \"console-955b7476f-ssrp8\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:29:36.812711 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.812565 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frqzn\" (UniqueName: \"kubernetes.io/projected/952cf7c3-4861-43ab-b200-860136cd2ba7-kube-api-access-frqzn\") pod \"console-955b7476f-ssrp8\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:29:36.812711 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.812603 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/952cf7c3-4861-43ab-b200-860136cd2ba7-console-oauth-config\") pod \"console-955b7476f-ssrp8\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:29:36.812711 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.812627 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/952cf7c3-4861-43ab-b200-860136cd2ba7-console-config\") pod \"console-955b7476f-ssrp8\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:29:36.812711 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.812664 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/952cf7c3-4861-43ab-b200-860136cd2ba7-console-serving-cert\") pod \"console-955b7476f-ssrp8\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:29:36.812711 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.812692 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/952cf7c3-4861-43ab-b200-860136cd2ba7-service-ca\") pod \"console-955b7476f-ssrp8\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:29:36.813018 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.812735 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/952cf7c3-4861-43ab-b200-860136cd2ba7-oauth-serving-cert\") pod \"console-955b7476f-ssrp8\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:29:36.813805 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.813455 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/952cf7c3-4861-43ab-b200-860136cd2ba7-oauth-serving-cert\") pod \"console-955b7476f-ssrp8\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:29:36.813805 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.813590 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/952cf7c3-4861-43ab-b200-860136cd2ba7-trusted-ca-bundle\") pod \"console-955b7476f-ssrp8\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:29:36.813805 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.813762 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/952cf7c3-4861-43ab-b200-860136cd2ba7-service-ca\") pod \"console-955b7476f-ssrp8\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:29:36.814471 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.814451 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/952cf7c3-4861-43ab-b200-860136cd2ba7-console-config\") pod \"console-955b7476f-ssrp8\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:29:36.816118 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.816096 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/952cf7c3-4861-43ab-b200-860136cd2ba7-console-serving-cert\") pod \"console-955b7476f-ssrp8\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:29:36.816255 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.816235 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/952cf7c3-4861-43ab-b200-860136cd2ba7-console-oauth-config\") pod \"console-955b7476f-ssrp8\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:29:36.817197 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.817170 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:36.822932 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.822910 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frqzn\" (UniqueName: \"kubernetes.io/projected/952cf7c3-4861-43ab-b200-860136cd2ba7-kube-api-access-frqzn\") pod \"console-955b7476f-ssrp8\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:29:36.856633 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.856612 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:29:36.866697 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:36.866675 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65d5f4bdbb-6xr4f" Sep 17 07:29:37.173249 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.173226 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7b49b5b569-4nw84"] Sep 17 07:29:37.174273 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:29:37.174225 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod952cf7c3_4861_43ab_b200_860136cd2ba7.slice/crio-9091b19a94177e9767e790abacacbf57cda10264467f587ab5b4b2cea4968623 WatchSource:0}: Error finding container 9091b19a94177e9767e790abacacbf57cda10264467f587ab5b4b2cea4968623: Status 404 returned error can't find the container with id 9091b19a94177e9767e790abacacbf57cda10264467f587ab5b4b2cea4968623 Sep 17 07:29:37.204360 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.204340 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-955b7476f-ssrp8"] Sep 17 07:29:37.204447 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.204369 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7b49b5b569-4nw84"] Sep 17 07:29:37.204488 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.204448 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.207506 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.207485 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 17 07:29:37.207996 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.207975 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-qknn6\"" Sep 17 07:29:37.208075 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.207998 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 17 07:29:37.208075 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.208013 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 17 07:29:37.208182 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.208070 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 17 07:29:37.208182 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.208087 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 17 07:29:37.213610 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.213589 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 17 07:29:37.318153 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.318127 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/aec98c2a-2693-44a2-a6a4-6b53e2bca7ed-federate-client-tls\") pod \"telemeter-client-7b49b5b569-4nw84\" (UID: \"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed\") " pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.318264 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.318171 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aec98c2a-2693-44a2-a6a4-6b53e2bca7ed-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b49b5b569-4nw84\" (UID: \"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed\") " pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.318264 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.318193 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/aec98c2a-2693-44a2-a6a4-6b53e2bca7ed-telemeter-client-tls\") pod \"telemeter-client-7b49b5b569-4nw84\" (UID: \"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed\") " pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.318408 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.318271 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aec98c2a-2693-44a2-a6a4-6b53e2bca7ed-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b49b5b569-4nw84\" (UID: \"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed\") " pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.318408 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.318303 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aec98c2a-2693-44a2-a6a4-6b53e2bca7ed-serving-certs-ca-bundle\") pod \"telemeter-client-7b49b5b569-4nw84\" (UID: \"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed\") " pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.318408 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.318325 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aec98c2a-2693-44a2-a6a4-6b53e2bca7ed-metrics-client-ca\") pod \"telemeter-client-7b49b5b569-4nw84\" (UID: \"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed\") " pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.318408 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.318351 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwh4v\" (UniqueName: \"kubernetes.io/projected/aec98c2a-2693-44a2-a6a4-6b53e2bca7ed-kube-api-access-bwh4v\") pod \"telemeter-client-7b49b5b569-4nw84\" (UID: \"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed\") " pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.318408 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.318371 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/aec98c2a-2693-44a2-a6a4-6b53e2bca7ed-secret-telemeter-client\") pod \"telemeter-client-7b49b5b569-4nw84\" (UID: \"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed\") " pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.419075 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.419047 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aec98c2a-2693-44a2-a6a4-6b53e2bca7ed-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b49b5b569-4nw84\" (UID: \"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed\") " pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.419219 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.419083 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/aec98c2a-2693-44a2-a6a4-6b53e2bca7ed-telemeter-client-tls\") pod \"telemeter-client-7b49b5b569-4nw84\" (UID: \"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed\") " pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.419219 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.419103 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aec98c2a-2693-44a2-a6a4-6b53e2bca7ed-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b49b5b569-4nw84\" (UID: \"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed\") " pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.419219 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.419153 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aec98c2a-2693-44a2-a6a4-6b53e2bca7ed-serving-certs-ca-bundle\") pod \"telemeter-client-7b49b5b569-4nw84\" (UID: \"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed\") " pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.419219 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.419186 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aec98c2a-2693-44a2-a6a4-6b53e2bca7ed-metrics-client-ca\") pod \"telemeter-client-7b49b5b569-4nw84\" (UID: \"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed\") " pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.419448 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.419222 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwh4v\" (UniqueName: \"kubernetes.io/projected/aec98c2a-2693-44a2-a6a4-6b53e2bca7ed-kube-api-access-bwh4v\") pod \"telemeter-client-7b49b5b569-4nw84\" (UID: \"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed\") " pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.419448 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.419254 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/aec98c2a-2693-44a2-a6a4-6b53e2bca7ed-secret-telemeter-client\") pod \"telemeter-client-7b49b5b569-4nw84\" (UID: \"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed\") " pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.419448 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.419331 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/aec98c2a-2693-44a2-a6a4-6b53e2bca7ed-federate-client-tls\") pod \"telemeter-client-7b49b5b569-4nw84\" (UID: \"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed\") " pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.420069 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.420035 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aec98c2a-2693-44a2-a6a4-6b53e2bca7ed-serving-certs-ca-bundle\") pod \"telemeter-client-7b49b5b569-4nw84\" (UID: \"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed\") " pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.420181 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.420040 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aec98c2a-2693-44a2-a6a4-6b53e2bca7ed-metrics-client-ca\") pod \"telemeter-client-7b49b5b569-4nw84\" (UID: \"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed\") " pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.420352 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.420331 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aec98c2a-2693-44a2-a6a4-6b53e2bca7ed-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b49b5b569-4nw84\" (UID: \"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed\") " pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.421803 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.421751 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aec98c2a-2693-44a2-a6a4-6b53e2bca7ed-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b49b5b569-4nw84\" (UID: \"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed\") " pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.422236 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.422215 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/aec98c2a-2693-44a2-a6a4-6b53e2bca7ed-federate-client-tls\") pod \"telemeter-client-7b49b5b569-4nw84\" (UID: \"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed\") " pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.422317 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.422258 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/aec98c2a-2693-44a2-a6a4-6b53e2bca7ed-secret-telemeter-client\") pod \"telemeter-client-7b49b5b569-4nw84\" (UID: \"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed\") " pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.422317 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.422285 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/aec98c2a-2693-44a2-a6a4-6b53e2bca7ed-telemeter-client-tls\") pod \"telemeter-client-7b49b5b569-4nw84\" (UID: \"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed\") " pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.428328 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.428307 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwh4v\" (UniqueName: \"kubernetes.io/projected/aec98c2a-2693-44a2-a6a4-6b53e2bca7ed-kube-api-access-bwh4v\") pod \"telemeter-client-7b49b5b569-4nw84\" (UID: \"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed\") " pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.514321 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.514293 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" Sep 17 07:29:37.566519 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.566492 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-955b7476f-ssrp8" event={"ID":"952cf7c3-4861-43ab-b200-860136cd2ba7","Type":"ContainerStarted","Data":"5f285da0e5330b36b12a221c8b79eccb51a54cb107d4ec24f6e27bb3b260afc3"} Sep 17 07:29:37.566640 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.566548 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-955b7476f-ssrp8" event={"ID":"952cf7c3-4861-43ab-b200-860136cd2ba7","Type":"ContainerStarted","Data":"9091b19a94177e9767e790abacacbf57cda10264467f587ab5b4b2cea4968623"} Sep 17 07:29:37.573849 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.573820 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:37.592300 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.592080 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-955b7476f-ssrp8" podStartSLOduration=1.592056544 podStartE2EDuration="1.592056544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:29:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:37.58950889 +0000 UTC m=+99.147693704" watchObservedRunningTime="2026-09-17 07:29:37.592056544 +0000 UTC m=+99.150241353" Sep 17 07:29:37.675678 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.675603 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:37.774015 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.773975 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:37.816484 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:37.816454 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7b49b5b569-4nw84"] Sep 17 07:29:37.819129 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:29:37.819106 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaec98c2a_2693_44a2_a6a4_6b53e2bca7ed.slice/crio-22a20c0ec9e3f52ef6f0bb4494250ddc97d3fcfa07733cc4d70ac7c09f359a94 WatchSource:0}: Error finding container 22a20c0ec9e3f52ef6f0bb4494250ddc97d3fcfa07733cc4d70ac7c09f359a94: Status 404 returned error can't find the container with id 22a20c0ec9e3f52ef6f0bb4494250ddc97d3fcfa07733cc4d70ac7c09f359a94 Sep 17 07:29:38.085532 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:38.085489 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:38.202519 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:38.202490 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:38.330325 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:38.330291 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:38.417341 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:38.417273 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:38.573149 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:38.573116 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" event={"ID":"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed","Type":"ContainerStarted","Data":"22a20c0ec9e3f52ef6f0bb4494250ddc97d3fcfa07733cc4d70ac7c09f359a94"} Sep 17 07:29:39.073533 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:39.073497 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:39.074025 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:39.073557 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:39.078573 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:39.078550 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:39.583472 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:39.583442 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:29:39.801737 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:39.801690 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:39.902070 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:39.901996 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:40.583906 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:40.583799 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" event={"ID":"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed","Type":"ContainerStarted","Data":"2beff73245f0253e9564e24099f6126d3178497db8aa8e11a1bccf90c088432f"} Sep 17 07:29:40.823306 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:40.823266 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:40.938810 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:40.938741 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:42.592722 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.592684 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" event={"ID":"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed","Type":"ContainerStarted","Data":"fb9c5caef6ad77cc4b989b9606b17c48b59de1bc8bfb1c761bc0842c0a988e82"} Sep 17 07:29:42.593135 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.592785 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" event={"ID":"aec98c2a-2693-44a2-a6a4-6b53e2bca7ed","Type":"ContainerStarted","Data":"210522109d5fe92e993397425bd0af0915631fdc633ec59120ac0238cf0070f9"} Sep 17 07:29:42.623487 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.623433 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7b49b5b569-4nw84" podStartSLOduration=1.645483539 podStartE2EDuration="5.623413545s" totalImagesPullingTime="3.977930006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:29:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:42.621169451 +0000 UTC m=+104.179354283" watchObservedRunningTime="2026-09-17 07:29:42.623413545 +0000 UTC m=+104.181598382" Sep 17 07:29:42.750124 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.750081 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-955b7476f-ssrp8"] Sep 17 07:29:42.801376 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.801336 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5947c9574b-f9h8j"] Sep 17 07:29:42.819159 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.819118 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5947c9574b-f9h8j"] Sep 17 07:29:42.819331 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.819210 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:42.863614 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.863539 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/293f041b-5ce3-4e35-8e3c-a42072574d10-console-oauth-config\") pod \"console-5947c9574b-f9h8j\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:42.863614 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.863570 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/293f041b-5ce3-4e35-8e3c-a42072574d10-oauth-serving-cert\") pod \"console-5947c9574b-f9h8j\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:42.863614 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.863590 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/293f041b-5ce3-4e35-8e3c-a42072574d10-console-config\") pod \"console-5947c9574b-f9h8j\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:42.863847 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.863628 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/293f041b-5ce3-4e35-8e3c-a42072574d10-trusted-ca-bundle\") pod \"console-5947c9574b-f9h8j\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:42.863847 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.863650 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/293f041b-5ce3-4e35-8e3c-a42072574d10-service-ca\") pod \"console-5947c9574b-f9h8j\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:42.863847 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.863678 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snvbn\" (UniqueName: \"kubernetes.io/projected/293f041b-5ce3-4e35-8e3c-a42072574d10-kube-api-access-snvbn\") pod \"console-5947c9574b-f9h8j\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:42.863847 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.863693 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/293f041b-5ce3-4e35-8e3c-a42072574d10-console-serving-cert\") pod \"console-5947c9574b-f9h8j\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:42.964874 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.964842 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/293f041b-5ce3-4e35-8e3c-a42072574d10-console-oauth-config\") pod \"console-5947c9574b-f9h8j\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:42.964874 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.964904 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/293f041b-5ce3-4e35-8e3c-a42072574d10-oauth-serving-cert\") pod \"console-5947c9574b-f9h8j\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:42.965141 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.964925 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/293f041b-5ce3-4e35-8e3c-a42072574d10-console-config\") pod \"console-5947c9574b-f9h8j\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:42.965141 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.964963 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/293f041b-5ce3-4e35-8e3c-a42072574d10-trusted-ca-bundle\") pod \"console-5947c9574b-f9h8j\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:42.965141 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.964990 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/293f041b-5ce3-4e35-8e3c-a42072574d10-service-ca\") pod \"console-5947c9574b-f9h8j\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:42.965141 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.965040 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snvbn\" (UniqueName: \"kubernetes.io/projected/293f041b-5ce3-4e35-8e3c-a42072574d10-kube-api-access-snvbn\") pod \"console-5947c9574b-f9h8j\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:42.965141 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.965090 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/293f041b-5ce3-4e35-8e3c-a42072574d10-console-serving-cert\") pod \"console-5947c9574b-f9h8j\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:42.965788 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.965753 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/293f041b-5ce3-4e35-8e3c-a42072574d10-console-config\") pod \"console-5947c9574b-f9h8j\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:42.965912 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.965787 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/293f041b-5ce3-4e35-8e3c-a42072574d10-service-ca\") pod \"console-5947c9574b-f9h8j\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:42.965912 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.965863 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/293f041b-5ce3-4e35-8e3c-a42072574d10-oauth-serving-cert\") pod \"console-5947c9574b-f9h8j\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:42.965989 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.965918 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/293f041b-5ce3-4e35-8e3c-a42072574d10-trusted-ca-bundle\") pod \"console-5947c9574b-f9h8j\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:42.967565 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.967544 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/293f041b-5ce3-4e35-8e3c-a42072574d10-console-serving-cert\") pod \"console-5947c9574b-f9h8j\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:42.967739 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.967719 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/293f041b-5ce3-4e35-8e3c-a42072574d10-console-oauth-config\") pod \"console-5947c9574b-f9h8j\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:42.994833 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:42.994799 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snvbn\" (UniqueName: \"kubernetes.io/projected/293f041b-5ce3-4e35-8e3c-a42072574d10-kube-api-access-snvbn\") pod \"console-5947c9574b-f9h8j\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:43.130674 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:43.130580 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:43.153340 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:43.153305 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:43.264647 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:43.264610 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:43.441971 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:43.441868 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-tsc9z" Sep 17 07:29:43.445115 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:43.445095 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5947c9574b-f9h8j"] Sep 17 07:29:43.446686 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:29:43.446665 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod293f041b_5ce3_4e35_8e3c_a42072574d10.slice/crio-a103eb087ebd0ce363b69e3fe34a678d3c393a057a17165c2d1dd5d1127296fd WatchSource:0}: Error finding container a103eb087ebd0ce363b69e3fe34a678d3c393a057a17165c2d1dd5d1127296fd: Status 404 returned error can't find the container with id a103eb087ebd0ce363b69e3fe34a678d3c393a057a17165c2d1dd5d1127296fd Sep 17 07:29:43.599213 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:43.599117 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5947c9574b-f9h8j" event={"ID":"293f041b-5ce3-4e35-8e3c-a42072574d10","Type":"ContainerStarted","Data":"169a66ba3cb0ed5d0c1da79af1994b384cbb91e84fe8281faa9ec66a316180ac"} Sep 17 07:29:43.599213 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:43.599177 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5947c9574b-f9h8j" event={"ID":"293f041b-5ce3-4e35-8e3c-a42072574d10","Type":"ContainerStarted","Data":"a103eb087ebd0ce363b69e3fe34a678d3c393a057a17165c2d1dd5d1127296fd"} Sep 17 07:29:43.624545 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:43.624493 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5947c9574b-f9h8j" podStartSLOduration=1.624473522 podStartE2EDuration="1.624473522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:29:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:43.623833674 +0000 UTC m=+105.182018483" watchObservedRunningTime="2026-09-17 07:29:43.624473522 +0000 UTC m=+105.182658331" Sep 17 07:29:43.768496 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:43.768464 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:44.233365 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:44.233326 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:44.296211 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:44.296183 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:45.097341 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:45.097297 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:45.205055 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:45.205023 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:46.857341 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:46.857310 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:29:53.130732 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:53.130697 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:53.131345 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:53.130766 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:53.135233 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:53.135209 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:53.642078 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:53.642053 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:29:53.693270 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:53.693241 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fbfb846df-w65xg"] Sep 17 07:29:53.803119 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:53.803085 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:53.904207 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:53.904121 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:53.917025 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:53.916994 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" podUID="dab30fd6-0467-4b87-83a4-104b2fa7d30d" containerName="registry" containerID="cri-o://a8466143a3de36a0eedf94326d1f92ed66dc77bd1ce7aa17fe7dd50f9edf9e1b" gracePeriod=30 Sep 17 07:29:53.963197 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:53.963175 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:54.194159 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.194129 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:29:54.248957 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.248932 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dab30fd6-0467-4b87-83a4-104b2fa7d30d-trusted-ca\") pod \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " Sep 17 07:29:54.249083 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.248973 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dab30fd6-0467-4b87-83a4-104b2fa7d30d-image-registry-private-configuration\") pod \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " Sep 17 07:29:54.249083 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.249020 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnjsb\" (UniqueName: \"kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-kube-api-access-hnjsb\") pod \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " Sep 17 07:29:54.249083 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.249077 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dab30fd6-0467-4b87-83a4-104b2fa7d30d-ca-trust-extracted\") pod \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " Sep 17 07:29:54.249247 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.249121 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dab30fd6-0467-4b87-83a4-104b2fa7d30d-installation-pull-secrets\") pod \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " Sep 17 07:29:54.249247 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.249150 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-certificates\") pod \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " Sep 17 07:29:54.249247 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.249194 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-tls\") pod \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " Sep 17 07:29:54.249247 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.249222 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-bound-sa-token\") pod \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\" (UID: \"dab30fd6-0467-4b87-83a4-104b2fa7d30d\") " Sep 17 07:29:54.249479 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.249450 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dab30fd6-0467-4b87-83a4-104b2fa7d30d-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "dab30fd6-0467-4b87-83a4-104b2fa7d30d" (UID: "dab30fd6-0467-4b87-83a4-104b2fa7d30d"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:29:54.249688 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.249662 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "dab30fd6-0467-4b87-83a4-104b2fa7d30d" (UID: "dab30fd6-0467-4b87-83a4-104b2fa7d30d"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:29:54.251714 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.251668 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "dab30fd6-0467-4b87-83a4-104b2fa7d30d" (UID: "dab30fd6-0467-4b87-83a4-104b2fa7d30d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:29:54.251814 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.251744 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-kube-api-access-hnjsb" (OuterVolumeSpecName: "kube-api-access-hnjsb") pod "dab30fd6-0467-4b87-83a4-104b2fa7d30d" (UID: "dab30fd6-0467-4b87-83a4-104b2fa7d30d"). InnerVolumeSpecName "kube-api-access-hnjsb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:29:54.251814 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.251776 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dab30fd6-0467-4b87-83a4-104b2fa7d30d-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "dab30fd6-0467-4b87-83a4-104b2fa7d30d" (UID: "dab30fd6-0467-4b87-83a4-104b2fa7d30d"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:29:54.251951 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.251808 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "dab30fd6-0467-4b87-83a4-104b2fa7d30d" (UID: "dab30fd6-0467-4b87-83a4-104b2fa7d30d"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:29:54.251951 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.251910 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dab30fd6-0467-4b87-83a4-104b2fa7d30d-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "dab30fd6-0467-4b87-83a4-104b2fa7d30d" (UID: "dab30fd6-0467-4b87-83a4-104b2fa7d30d"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:29:54.257720 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.257695 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dab30fd6-0467-4b87-83a4-104b2fa7d30d-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "dab30fd6-0467-4b87-83a4-104b2fa7d30d" (UID: "dab30fd6-0467-4b87-83a4-104b2fa7d30d"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:29:54.350022 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.350002 2553 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dab30fd6-0467-4b87-83a4-104b2fa7d30d-ca-trust-extracted\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:29:54.350022 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.350020 2553 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dab30fd6-0467-4b87-83a4-104b2fa7d30d-installation-pull-secrets\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:29:54.350131 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.350029 2553 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-certificates\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:29:54.350131 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.350039 2553 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-registry-tls\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:29:54.350131 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.350048 2553 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-bound-sa-token\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:29:54.350131 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.350055 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dab30fd6-0467-4b87-83a4-104b2fa7d30d-trusted-ca\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:29:54.350131 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.350063 2553 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dab30fd6-0467-4b87-83a4-104b2fa7d30d-image-registry-private-configuration\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:29:54.350131 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.350072 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnjsb\" (UniqueName: \"kubernetes.io/projected/dab30fd6-0467-4b87-83a4-104b2fa7d30d-kube-api-access-hnjsb\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:29:54.643559 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.643530 2553 generic.go:356] "Generic (PLEG): container finished" podID="dab30fd6-0467-4b87-83a4-104b2fa7d30d" containerID="a8466143a3de36a0eedf94326d1f92ed66dc77bd1ce7aa17fe7dd50f9edf9e1b" exitCode=0 Sep 17 07:29:54.643680 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.643602 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" Sep 17 07:29:54.643680 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.643624 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" event={"ID":"dab30fd6-0467-4b87-83a4-104b2fa7d30d","Type":"ContainerDied","Data":"a8466143a3de36a0eedf94326d1f92ed66dc77bd1ce7aa17fe7dd50f9edf9e1b"} Sep 17 07:29:54.643794 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.643706 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7484fc99c6-dxxgt" event={"ID":"dab30fd6-0467-4b87-83a4-104b2fa7d30d","Type":"ContainerDied","Data":"7f26e02adaa3a0f1f0b3172c7f6544cda949376b27479ad36d448c8b66c8d349"} Sep 17 07:29:54.643794 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.643729 2553 scope.go:117] "RemoveContainer" containerID="a8466143a3de36a0eedf94326d1f92ed66dc77bd1ce7aa17fe7dd50f9edf9e1b" Sep 17 07:29:54.653034 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.653016 2553 scope.go:117] "RemoveContainer" containerID="a8466143a3de36a0eedf94326d1f92ed66dc77bd1ce7aa17fe7dd50f9edf9e1b" Sep 17 07:29:54.653310 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:29:54.653290 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8466143a3de36a0eedf94326d1f92ed66dc77bd1ce7aa17fe7dd50f9edf9e1b\": container with ID starting with a8466143a3de36a0eedf94326d1f92ed66dc77bd1ce7aa17fe7dd50f9edf9e1b not found: ID does not exist" containerID="a8466143a3de36a0eedf94326d1f92ed66dc77bd1ce7aa17fe7dd50f9edf9e1b" Sep 17 07:29:54.653362 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.653318 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8466143a3de36a0eedf94326d1f92ed66dc77bd1ce7aa17fe7dd50f9edf9e1b"} err="failed to get container status \"a8466143a3de36a0eedf94326d1f92ed66dc77bd1ce7aa17fe7dd50f9edf9e1b\": rpc error: code = NotFound desc = could not find container \"a8466143a3de36a0eedf94326d1f92ed66dc77bd1ce7aa17fe7dd50f9edf9e1b\": container with ID starting with a8466143a3de36a0eedf94326d1f92ed66dc77bd1ce7aa17fe7dd50f9edf9e1b not found: ID does not exist" Sep 17 07:29:54.668060 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.668037 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7484fc99c6-dxxgt"] Sep 17 07:29:54.671764 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.671745 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7484fc99c6-dxxgt"] Sep 17 07:29:54.933565 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:54.933511 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dab30fd6-0467-4b87-83a4-104b2fa7d30d" path="/var/lib/kubelet/pods/dab30fd6-0467-4b87-83a4-104b2fa7d30d/volumes" Sep 17 07:29:59.251271 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:59.251238 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:29:59.328123 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:29:59.328094 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:30:01.489432 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:01.489369 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-65d5f4bdbb-6xr4f" podUID="945229a2-9595-4ac3-901c-bfa0576f6140" containerName="console" containerID="cri-o://ceccb0e9f95e8e8fff0c465e8376780e079dae1ab7faee8769b47cb41a51f50d" gracePeriod=15 Sep 17 07:30:01.674258 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:01.674224 2553 generic.go:356] "Generic (PLEG): container finished" podID="945229a2-9595-4ac3-901c-bfa0576f6140" containerID="ceccb0e9f95e8e8fff0c465e8376780e079dae1ab7faee8769b47cb41a51f50d" exitCode=0 Sep 17 07:30:01.674414 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:01.674299 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65d5f4bdbb-6xr4f" event={"ID":"945229a2-9595-4ac3-901c-bfa0576f6140","Type":"ContainerDied","Data":"ceccb0e9f95e8e8fff0c465e8376780e079dae1ab7faee8769b47cb41a51f50d"} Sep 17 07:30:01.766404 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:01.766351 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65d5f4bdbb-6xr4f" Sep 17 07:30:01.804375 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:01.804353 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/945229a2-9595-4ac3-901c-bfa0576f6140-oauth-serving-cert\") pod \"945229a2-9595-4ac3-901c-bfa0576f6140\" (UID: \"945229a2-9595-4ac3-901c-bfa0576f6140\") " Sep 17 07:30:01.804505 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:01.804389 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/945229a2-9595-4ac3-901c-bfa0576f6140-console-oauth-config\") pod \"945229a2-9595-4ac3-901c-bfa0576f6140\" (UID: \"945229a2-9595-4ac3-901c-bfa0576f6140\") " Sep 17 07:30:01.804505 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:01.804406 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/945229a2-9595-4ac3-901c-bfa0576f6140-console-config\") pod \"945229a2-9595-4ac3-901c-bfa0576f6140\" (UID: \"945229a2-9595-4ac3-901c-bfa0576f6140\") " Sep 17 07:30:01.804505 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:01.804431 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/945229a2-9595-4ac3-901c-bfa0576f6140-service-ca\") pod \"945229a2-9595-4ac3-901c-bfa0576f6140\" (UID: \"945229a2-9595-4ac3-901c-bfa0576f6140\") " Sep 17 07:30:01.804663 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:01.804603 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/945229a2-9595-4ac3-901c-bfa0576f6140-console-serving-cert\") pod \"945229a2-9595-4ac3-901c-bfa0576f6140\" (UID: \"945229a2-9595-4ac3-901c-bfa0576f6140\") " Sep 17 07:30:01.804719 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:01.804708 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-286mz\" (UniqueName: \"kubernetes.io/projected/945229a2-9595-4ac3-901c-bfa0576f6140-kube-api-access-286mz\") pod \"945229a2-9595-4ac3-901c-bfa0576f6140\" (UID: \"945229a2-9595-4ac3-901c-bfa0576f6140\") " Sep 17 07:30:01.804800 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:01.804780 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/945229a2-9595-4ac3-901c-bfa0576f6140-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "945229a2-9595-4ac3-901c-bfa0576f6140" (UID: "945229a2-9595-4ac3-901c-bfa0576f6140"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:30:01.804800 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:01.804789 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/945229a2-9595-4ac3-901c-bfa0576f6140-service-ca" (OuterVolumeSpecName: "service-ca") pod "945229a2-9595-4ac3-901c-bfa0576f6140" (UID: "945229a2-9595-4ac3-901c-bfa0576f6140"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:30:01.804933 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:01.804798 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/945229a2-9595-4ac3-901c-bfa0576f6140-console-config" (OuterVolumeSpecName: "console-config") pod "945229a2-9595-4ac3-901c-bfa0576f6140" (UID: "945229a2-9595-4ac3-901c-bfa0576f6140"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:30:01.805007 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:01.804990 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/945229a2-9595-4ac3-901c-bfa0576f6140-oauth-serving-cert\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:30:01.805055 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:01.805013 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/945229a2-9595-4ac3-901c-bfa0576f6140-console-config\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:30:01.805055 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:01.805027 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/945229a2-9595-4ac3-901c-bfa0576f6140-service-ca\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:30:01.806604 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:01.806578 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/945229a2-9595-4ac3-901c-bfa0576f6140-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "945229a2-9595-4ac3-901c-bfa0576f6140" (UID: "945229a2-9595-4ac3-901c-bfa0576f6140"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:30:01.806698 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:01.806651 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/945229a2-9595-4ac3-901c-bfa0576f6140-kube-api-access-286mz" (OuterVolumeSpecName: "kube-api-access-286mz") pod "945229a2-9595-4ac3-901c-bfa0576f6140" (UID: "945229a2-9595-4ac3-901c-bfa0576f6140"). InnerVolumeSpecName "kube-api-access-286mz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:30:01.807086 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:01.807069 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/945229a2-9595-4ac3-901c-bfa0576f6140-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "945229a2-9595-4ac3-901c-bfa0576f6140" (UID: "945229a2-9595-4ac3-901c-bfa0576f6140"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:30:01.906226 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:01.906204 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-286mz\" (UniqueName: \"kubernetes.io/projected/945229a2-9595-4ac3-901c-bfa0576f6140-kube-api-access-286mz\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:30:01.906226 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:01.906224 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/945229a2-9595-4ac3-901c-bfa0576f6140-console-oauth-config\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:30:01.906350 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:01.906233 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/945229a2-9595-4ac3-901c-bfa0576f6140-console-serving-cert\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:30:02.679457 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:02.679426 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65d5f4bdbb-6xr4f" Sep 17 07:30:02.679816 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:02.679459 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65d5f4bdbb-6xr4f" event={"ID":"945229a2-9595-4ac3-901c-bfa0576f6140","Type":"ContainerDied","Data":"6646076dd7dd8cf21f9a62a3b4bd7397c26b92a308b0ec765a73feaf6723e154"} Sep 17 07:30:02.679816 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:02.679523 2553 scope.go:117] "RemoveContainer" containerID="ceccb0e9f95e8e8fff0c465e8376780e079dae1ab7faee8769b47cb41a51f50d" Sep 17 07:30:02.713223 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:02.713191 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65d5f4bdbb-6xr4f"] Sep 17 07:30:02.717824 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:02.717804 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-65d5f4bdbb-6xr4f"] Sep 17 07:30:02.934120 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:02.934072 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="945229a2-9595-4ac3-901c-bfa0576f6140" path="/var/lib/kubelet/pods/945229a2-9595-4ac3-901c-bfa0576f6140/volumes" Sep 17 07:30:07.774574 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:07.774530 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-955b7476f-ssrp8" podUID="952cf7c3-4861-43ab-b200-860136cd2ba7" containerName="console" containerID="cri-o://5f285da0e5330b36b12a221c8b79eccb51a54cb107d4ec24f6e27bb3b260afc3" gracePeriod=15 Sep 17 07:30:08.049059 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.048997 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:30:08.151961 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.151934 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/952cf7c3-4861-43ab-b200-860136cd2ba7-service-ca\") pod \"952cf7c3-4861-43ab-b200-860136cd2ba7\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " Sep 17 07:30:08.152064 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.151981 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/952cf7c3-4861-43ab-b200-860136cd2ba7-oauth-serving-cert\") pod \"952cf7c3-4861-43ab-b200-860136cd2ba7\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " Sep 17 07:30:08.152064 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.151998 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/952cf7c3-4861-43ab-b200-860136cd2ba7-trusted-ca-bundle\") pod \"952cf7c3-4861-43ab-b200-860136cd2ba7\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " Sep 17 07:30:08.152064 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.152017 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/952cf7c3-4861-43ab-b200-860136cd2ba7-console-oauth-config\") pod \"952cf7c3-4861-43ab-b200-860136cd2ba7\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " Sep 17 07:30:08.152064 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.152053 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/952cf7c3-4861-43ab-b200-860136cd2ba7-console-config\") pod \"952cf7c3-4861-43ab-b200-860136cd2ba7\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " Sep 17 07:30:08.152309 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.152074 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/952cf7c3-4861-43ab-b200-860136cd2ba7-console-serving-cert\") pod \"952cf7c3-4861-43ab-b200-860136cd2ba7\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " Sep 17 07:30:08.152309 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.152104 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frqzn\" (UniqueName: \"kubernetes.io/projected/952cf7c3-4861-43ab-b200-860136cd2ba7-kube-api-access-frqzn\") pod \"952cf7c3-4861-43ab-b200-860136cd2ba7\" (UID: \"952cf7c3-4861-43ab-b200-860136cd2ba7\") " Sep 17 07:30:08.152453 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.152425 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/952cf7c3-4861-43ab-b200-860136cd2ba7-service-ca" (OuterVolumeSpecName: "service-ca") pod "952cf7c3-4861-43ab-b200-860136cd2ba7" (UID: "952cf7c3-4861-43ab-b200-860136cd2ba7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:30:08.152538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.152451 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/952cf7c3-4861-43ab-b200-860136cd2ba7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "952cf7c3-4861-43ab-b200-860136cd2ba7" (UID: "952cf7c3-4861-43ab-b200-860136cd2ba7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:30:08.152593 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.152554 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/952cf7c3-4861-43ab-b200-860136cd2ba7-console-config" (OuterVolumeSpecName: "console-config") pod "952cf7c3-4861-43ab-b200-860136cd2ba7" (UID: "952cf7c3-4861-43ab-b200-860136cd2ba7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:30:08.152635 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.152582 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/952cf7c3-4861-43ab-b200-860136cd2ba7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "952cf7c3-4861-43ab-b200-860136cd2ba7" (UID: "952cf7c3-4861-43ab-b200-860136cd2ba7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:30:08.154307 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.154275 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/952cf7c3-4861-43ab-b200-860136cd2ba7-kube-api-access-frqzn" (OuterVolumeSpecName: "kube-api-access-frqzn") pod "952cf7c3-4861-43ab-b200-860136cd2ba7" (UID: "952cf7c3-4861-43ab-b200-860136cd2ba7"). InnerVolumeSpecName "kube-api-access-frqzn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:30:08.154307 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.154284 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/952cf7c3-4861-43ab-b200-860136cd2ba7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "952cf7c3-4861-43ab-b200-860136cd2ba7" (UID: "952cf7c3-4861-43ab-b200-860136cd2ba7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:30:08.154439 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.154302 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/952cf7c3-4861-43ab-b200-860136cd2ba7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "952cf7c3-4861-43ab-b200-860136cd2ba7" (UID: "952cf7c3-4861-43ab-b200-860136cd2ba7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:30:08.253272 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.253249 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/952cf7c3-4861-43ab-b200-860136cd2ba7-oauth-serving-cert\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:30:08.253272 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.253269 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/952cf7c3-4861-43ab-b200-860136cd2ba7-trusted-ca-bundle\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:30:08.253413 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.253278 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/952cf7c3-4861-43ab-b200-860136cd2ba7-console-oauth-config\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:30:08.253413 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.253286 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/952cf7c3-4861-43ab-b200-860136cd2ba7-console-config\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:30:08.253413 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.253295 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/952cf7c3-4861-43ab-b200-860136cd2ba7-console-serving-cert\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:30:08.253413 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.253304 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frqzn\" (UniqueName: \"kubernetes.io/projected/952cf7c3-4861-43ab-b200-860136cd2ba7-kube-api-access-frqzn\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:30:08.253413 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.253313 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/952cf7c3-4861-43ab-b200-860136cd2ba7-service-ca\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:30:08.707753 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.707719 2553 generic.go:356] "Generic (PLEG): container finished" podID="952cf7c3-4861-43ab-b200-860136cd2ba7" containerID="5f285da0e5330b36b12a221c8b79eccb51a54cb107d4ec24f6e27bb3b260afc3" exitCode=0 Sep 17 07:30:08.707919 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.707772 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-955b7476f-ssrp8" Sep 17 07:30:08.707919 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.707814 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-955b7476f-ssrp8" event={"ID":"952cf7c3-4861-43ab-b200-860136cd2ba7","Type":"ContainerDied","Data":"5f285da0e5330b36b12a221c8b79eccb51a54cb107d4ec24f6e27bb3b260afc3"} Sep 17 07:30:08.708014 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.707922 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-955b7476f-ssrp8" event={"ID":"952cf7c3-4861-43ab-b200-860136cd2ba7","Type":"ContainerDied","Data":"9091b19a94177e9767e790abacacbf57cda10264467f587ab5b4b2cea4968623"} Sep 17 07:30:08.708014 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.707949 2553 scope.go:117] "RemoveContainer" containerID="5f285da0e5330b36b12a221c8b79eccb51a54cb107d4ec24f6e27bb3b260afc3" Sep 17 07:30:08.717221 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.717203 2553 scope.go:117] "RemoveContainer" containerID="5f285da0e5330b36b12a221c8b79eccb51a54cb107d4ec24f6e27bb3b260afc3" Sep 17 07:30:08.717511 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:30:08.717487 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f285da0e5330b36b12a221c8b79eccb51a54cb107d4ec24f6e27bb3b260afc3\": container with ID starting with 5f285da0e5330b36b12a221c8b79eccb51a54cb107d4ec24f6e27bb3b260afc3 not found: ID does not exist" containerID="5f285da0e5330b36b12a221c8b79eccb51a54cb107d4ec24f6e27bb3b260afc3" Sep 17 07:30:08.717613 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.717515 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f285da0e5330b36b12a221c8b79eccb51a54cb107d4ec24f6e27bb3b260afc3"} err="failed to get container status \"5f285da0e5330b36b12a221c8b79eccb51a54cb107d4ec24f6e27bb3b260afc3\": rpc error: code = NotFound desc = could not find container \"5f285da0e5330b36b12a221c8b79eccb51a54cb107d4ec24f6e27bb3b260afc3\": container with ID starting with 5f285da0e5330b36b12a221c8b79eccb51a54cb107d4ec24f6e27bb3b260afc3 not found: ID does not exist" Sep 17 07:30:08.729050 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.729027 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-955b7476f-ssrp8"] Sep 17 07:30:08.736009 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.735987 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-955b7476f-ssrp8"] Sep 17 07:30:08.933081 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:08.933052 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="952cf7c3-4861-43ab-b200-860136cd2ba7" path="/var/lib/kubelet/pods/952cf7c3-4861-43ab-b200-860136cd2ba7/volumes" Sep 17 07:30:18.714154 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:18.714113 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-fbfb846df-w65xg" podUID="082ae202-9e8d-4e87-9ab7-1216e78a5f5c" containerName="console" containerID="cri-o://afdca5c29c79bfd2740e21aa5df446046b41de5ac6a042fa3429e6fbd7df33f1" gracePeriod=15 Sep 17 07:30:18.954620 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:18.954596 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:30:19.025375 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.025324 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-console-oauth-config\") pod \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " Sep 17 07:30:19.025375 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.025362 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-console-config\") pod \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " Sep 17 07:30:19.025567 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.025383 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-oauth-serving-cert\") pod \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " Sep 17 07:30:19.025567 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.025403 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-trusted-ca-bundle\") pod \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " Sep 17 07:30:19.025567 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.025428 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-console-serving-cert\") pod \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " Sep 17 07:30:19.025567 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.025451 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-service-ca\") pod \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " Sep 17 07:30:19.025567 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.025525 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dhmw\" (UniqueName: \"kubernetes.io/projected/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-kube-api-access-5dhmw\") pod \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\" (UID: \"082ae202-9e8d-4e87-9ab7-1216e78a5f5c\") " Sep 17 07:30:19.025873 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.025844 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "082ae202-9e8d-4e87-9ab7-1216e78a5f5c" (UID: "082ae202-9e8d-4e87-9ab7-1216e78a5f5c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:30:19.025873 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.025859 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-console-config" (OuterVolumeSpecName: "console-config") pod "082ae202-9e8d-4e87-9ab7-1216e78a5f5c" (UID: "082ae202-9e8d-4e87-9ab7-1216e78a5f5c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:30:19.025996 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.025853 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "082ae202-9e8d-4e87-9ab7-1216e78a5f5c" (UID: "082ae202-9e8d-4e87-9ab7-1216e78a5f5c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:30:19.025996 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.025910 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-service-ca" (OuterVolumeSpecName: "service-ca") pod "082ae202-9e8d-4e87-9ab7-1216e78a5f5c" (UID: "082ae202-9e8d-4e87-9ab7-1216e78a5f5c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:30:19.027591 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.027569 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-kube-api-access-5dhmw" (OuterVolumeSpecName: "kube-api-access-5dhmw") pod "082ae202-9e8d-4e87-9ab7-1216e78a5f5c" (UID: "082ae202-9e8d-4e87-9ab7-1216e78a5f5c"). InnerVolumeSpecName "kube-api-access-5dhmw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:30:19.027923 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.027870 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "082ae202-9e8d-4e87-9ab7-1216e78a5f5c" (UID: "082ae202-9e8d-4e87-9ab7-1216e78a5f5c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:30:19.028000 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.027961 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "082ae202-9e8d-4e87-9ab7-1216e78a5f5c" (UID: "082ae202-9e8d-4e87-9ab7-1216e78a5f5c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:30:19.126475 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.126451 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-console-oauth-config\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:30:19.126475 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.126472 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-console-config\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:30:19.126602 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.126481 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-oauth-serving-cert\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:30:19.126602 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.126489 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-trusted-ca-bundle\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:30:19.126602 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.126499 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-console-serving-cert\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:30:19.126602 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.126507 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-service-ca\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:30:19.126602 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.126515 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dhmw\" (UniqueName: \"kubernetes.io/projected/082ae202-9e8d-4e87-9ab7-1216e78a5f5c-kube-api-access-5dhmw\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:30:19.757855 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.757824 2553 generic.go:356] "Generic (PLEG): container finished" podID="082ae202-9e8d-4e87-9ab7-1216e78a5f5c" containerID="afdca5c29c79bfd2740e21aa5df446046b41de5ac6a042fa3429e6fbd7df33f1" exitCode=0 Sep 17 07:30:19.758285 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.757871 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fbfb846df-w65xg" Sep 17 07:30:19.758285 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.757897 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fbfb846df-w65xg" event={"ID":"082ae202-9e8d-4e87-9ab7-1216e78a5f5c","Type":"ContainerDied","Data":"afdca5c29c79bfd2740e21aa5df446046b41de5ac6a042fa3429e6fbd7df33f1"} Sep 17 07:30:19.758285 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.757942 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fbfb846df-w65xg" event={"ID":"082ae202-9e8d-4e87-9ab7-1216e78a5f5c","Type":"ContainerDied","Data":"b358b2ef4f5ace213a5c2fb92ad1331806c91fe89b3ba67c5da536da3a6d6a7c"} Sep 17 07:30:19.758285 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.757965 2553 scope.go:117] "RemoveContainer" containerID="afdca5c29c79bfd2740e21aa5df446046b41de5ac6a042fa3429e6fbd7df33f1" Sep 17 07:30:19.767619 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.767597 2553 scope.go:117] "RemoveContainer" containerID="afdca5c29c79bfd2740e21aa5df446046b41de5ac6a042fa3429e6fbd7df33f1" Sep 17 07:30:19.767927 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:30:19.767909 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afdca5c29c79bfd2740e21aa5df446046b41de5ac6a042fa3429e6fbd7df33f1\": container with ID starting with afdca5c29c79bfd2740e21aa5df446046b41de5ac6a042fa3429e6fbd7df33f1 not found: ID does not exist" containerID="afdca5c29c79bfd2740e21aa5df446046b41de5ac6a042fa3429e6fbd7df33f1" Sep 17 07:30:19.767974 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.767934 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afdca5c29c79bfd2740e21aa5df446046b41de5ac6a042fa3429e6fbd7df33f1"} err="failed to get container status \"afdca5c29c79bfd2740e21aa5df446046b41de5ac6a042fa3429e6fbd7df33f1\": rpc error: code = NotFound desc = could not find container \"afdca5c29c79bfd2740e21aa5df446046b41de5ac6a042fa3429e6fbd7df33f1\": container with ID starting with afdca5c29c79bfd2740e21aa5df446046b41de5ac6a042fa3429e6fbd7df33f1 not found: ID does not exist" Sep 17 07:30:19.782061 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.782041 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fbfb846df-w65xg"] Sep 17 07:30:19.789404 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:19.789379 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-fbfb846df-w65xg"] Sep 17 07:30:20.933811 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:20.933770 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="082ae202-9e8d-4e87-9ab7-1216e78a5f5c" path="/var/lib/kubelet/pods/082ae202-9e8d-4e87-9ab7-1216e78a5f5c/volumes" Sep 17 07:30:47.908283 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:47.908250 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:30:48.005977 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:48.005948 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:30:54.407720 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.407684 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 17 07:30:54.408199 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.407980 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="945229a2-9595-4ac3-901c-bfa0576f6140" containerName="console" Sep 17 07:30:54.408199 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.407993 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="945229a2-9595-4ac3-901c-bfa0576f6140" containerName="console" Sep 17 07:30:54.408199 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.408002 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dab30fd6-0467-4b87-83a4-104b2fa7d30d" containerName="registry" Sep 17 07:30:54.408199 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.408007 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="dab30fd6-0467-4b87-83a4-104b2fa7d30d" containerName="registry" Sep 17 07:30:54.408199 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.408019 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="082ae202-9e8d-4e87-9ab7-1216e78a5f5c" containerName="console" Sep 17 07:30:54.408199 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.408024 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="082ae202-9e8d-4e87-9ab7-1216e78a5f5c" containerName="console" Sep 17 07:30:54.408199 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.408031 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="952cf7c3-4861-43ab-b200-860136cd2ba7" containerName="console" Sep 17 07:30:54.408199 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.408037 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="952cf7c3-4861-43ab-b200-860136cd2ba7" containerName="console" Sep 17 07:30:54.408199 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.408088 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="082ae202-9e8d-4e87-9ab7-1216e78a5f5c" containerName="console" Sep 17 07:30:54.408199 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.408097 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="945229a2-9595-4ac3-901c-bfa0576f6140" containerName="console" Sep 17 07:30:54.408199 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.408103 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="dab30fd6-0467-4b87-83a4-104b2fa7d30d" containerName="registry" Sep 17 07:30:54.408199 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.408108 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="952cf7c3-4861-43ab-b200-860136cd2ba7" containerName="console" Sep 17 07:30:54.410489 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.410466 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.416740 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.416712 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-x2wvj\"" Sep 17 07:30:54.416871 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.416764 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 17 07:30:54.417693 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.417669 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 17 07:30:54.418166 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.418145 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 17 07:30:54.418854 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.418823 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 17 07:30:54.418854 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.418848 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 17 07:30:54.419004 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.418822 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 17 07:30:54.419004 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.418822 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 17 07:30:54.419198 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.419177 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 17 07:30:54.423355 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.423335 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 17 07:30:54.436870 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.436846 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 17 07:30:54.560845 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.560819 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.560966 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.560848 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.560966 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.560869 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.560966 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.560942 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.561070 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.560965 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.561070 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.561011 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.561070 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.561048 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-config-volume\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.561158 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.561072 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-config-out\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.561158 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.561089 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.561158 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.561105 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-web-config\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.561158 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.561126 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.561367 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.561163 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d95jb\" (UniqueName: \"kubernetes.io/projected/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-kube-api-access-d95jb\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.561367 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.561193 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.662036 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.661978 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-config-volume\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.662036 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.662010 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-config-out\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.662036 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.662029 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.662198 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.662053 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-web-config\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.662198 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.662075 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.662198 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.662097 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d95jb\" (UniqueName: \"kubernetes.io/projected/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-kube-api-access-d95jb\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.662198 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.662117 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.662198 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.662147 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.662198 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.662170 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.662198 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.662195 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.662538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.662232 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.662538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.662258 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.662538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.662291 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.662538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.662507 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.663146 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.663121 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.665362 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.665058 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-config-out\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.665362 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.665202 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-config-volume\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.665362 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.665216 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.665362 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.665293 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.665595 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.665399 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-web-config\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.665679 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.665657 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.665838 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.665817 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.665995 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.665974 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.666044 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.665996 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.667137 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.667121 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.675177 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.675150 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d95jb\" (UniqueName: \"kubernetes.io/projected/f749cdd2-2dc4-475f-be93-a7dc4d12bfc0-kube-api-access-d95jb\") pod \"alertmanager-main-0\" (UID: \"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0\") " pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:54.720345 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:54.720324 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 17 07:30:55.020598 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.020568 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 17 07:30:55.024039 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:30:55.024007 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf749cdd2_2dc4_475f_be93_a7dc4d12bfc0.slice/crio-62767b14d5def9062be557b0a887e9f97124ef8c6cf247d913b9d81481d1c43d WatchSource:0}: Error finding container 62767b14d5def9062be557b0a887e9f97124ef8c6cf247d913b9d81481d1c43d: Status 404 returned error can't find the container with id 62767b14d5def9062be557b0a887e9f97124ef8c6cf247d913b9d81481d1c43d Sep 17 07:30:55.710754 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.710718 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c655bd8c-xvl6f"] Sep 17 07:30:55.713071 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.713049 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:30:55.735695 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.735669 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c655bd8c-xvl6f"] Sep 17 07:30:55.804577 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.804540 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:30:55.872606 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.872579 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/137a96f3-3205-431b-9034-de0e2fd7b359-console-config\") pod \"console-5c655bd8c-xvl6f\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:30:55.872698 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.872610 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/137a96f3-3205-431b-9034-de0e2fd7b359-oauth-serving-cert\") pod \"console-5c655bd8c-xvl6f\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:30:55.872698 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.872633 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/137a96f3-3205-431b-9034-de0e2fd7b359-trusted-ca-bundle\") pod \"console-5c655bd8c-xvl6f\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:30:55.872791 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.872698 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/137a96f3-3205-431b-9034-de0e2fd7b359-console-oauth-config\") pod \"console-5c655bd8c-xvl6f\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:30:55.872791 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.872736 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/137a96f3-3205-431b-9034-de0e2fd7b359-service-ca\") pod \"console-5c655bd8c-xvl6f\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:30:55.872791 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.872761 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q4z5\" (UniqueName: \"kubernetes.io/projected/137a96f3-3205-431b-9034-de0e2fd7b359-kube-api-access-2q4z5\") pod \"console-5c655bd8c-xvl6f\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:30:55.872791 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.872780 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/137a96f3-3205-431b-9034-de0e2fd7b359-console-serving-cert\") pod \"console-5c655bd8c-xvl6f\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:30:55.906305 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.906280 2553 generic.go:356] "Generic (PLEG): container finished" podID="f749cdd2-2dc4-475f-be93-a7dc4d12bfc0" containerID="cb98755de10ece4e2c4dc703832504bb7d3a03f19f0598ff4429df01d97f5f73" exitCode=0 Sep 17 07:30:55.906392 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.906310 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0","Type":"ContainerDied","Data":"cb98755de10ece4e2c4dc703832504bb7d3a03f19f0598ff4429df01d97f5f73"} Sep 17 07:30:55.906392 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.906355 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0","Type":"ContainerStarted","Data":"62767b14d5def9062be557b0a887e9f97124ef8c6cf247d913b9d81481d1c43d"} Sep 17 07:30:55.973989 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.973937 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/137a96f3-3205-431b-9034-de0e2fd7b359-console-serving-cert\") pod \"console-5c655bd8c-xvl6f\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:30:55.973989 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.973976 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/137a96f3-3205-431b-9034-de0e2fd7b359-console-config\") pod \"console-5c655bd8c-xvl6f\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:30:55.974102 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.973994 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/137a96f3-3205-431b-9034-de0e2fd7b359-oauth-serving-cert\") pod \"console-5c655bd8c-xvl6f\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:30:55.974102 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.974018 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/137a96f3-3205-431b-9034-de0e2fd7b359-trusted-ca-bundle\") pod \"console-5c655bd8c-xvl6f\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:30:55.974102 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.974055 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/137a96f3-3205-431b-9034-de0e2fd7b359-console-oauth-config\") pod \"console-5c655bd8c-xvl6f\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:30:55.974102 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.974078 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/137a96f3-3205-431b-9034-de0e2fd7b359-service-ca\") pod \"console-5c655bd8c-xvl6f\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:30:55.974245 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.974159 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q4z5\" (UniqueName: \"kubernetes.io/projected/137a96f3-3205-431b-9034-de0e2fd7b359-kube-api-access-2q4z5\") pod \"console-5c655bd8c-xvl6f\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:30:55.974680 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.974661 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/137a96f3-3205-431b-9034-de0e2fd7b359-console-config\") pod \"console-5c655bd8c-xvl6f\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:30:55.974776 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.974742 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/137a96f3-3205-431b-9034-de0e2fd7b359-service-ca\") pod \"console-5c655bd8c-xvl6f\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:30:55.974846 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.974742 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/137a96f3-3205-431b-9034-de0e2fd7b359-oauth-serving-cert\") pod \"console-5c655bd8c-xvl6f\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:30:55.975407 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.975383 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/137a96f3-3205-431b-9034-de0e2fd7b359-trusted-ca-bundle\") pod \"console-5c655bd8c-xvl6f\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:30:55.976515 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.976494 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/137a96f3-3205-431b-9034-de0e2fd7b359-console-oauth-config\") pod \"console-5c655bd8c-xvl6f\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:30:55.976628 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.976610 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/137a96f3-3205-431b-9034-de0e2fd7b359-console-serving-cert\") pod \"console-5c655bd8c-xvl6f\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:30:55.984498 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:55.984479 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q4z5\" (UniqueName: \"kubernetes.io/projected/137a96f3-3205-431b-9034-de0e2fd7b359-kube-api-access-2q4z5\") pod \"console-5c655bd8c-xvl6f\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:30:56.021459 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:56.021434 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:30:56.302704 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:56.302623 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c655bd8c-xvl6f"] Sep 17 07:30:56.305631 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:30:56.305606 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod137a96f3_3205_431b_9034_de0e2fd7b359.slice/crio-879aae759813c65625c6d091209b719f755122183dfe2f3b091e828b0a6a77d4 WatchSource:0}: Error finding container 879aae759813c65625c6d091209b719f755122183dfe2f3b091e828b0a6a77d4: Status 404 returned error can't find the container with id 879aae759813c65625c6d091209b719f755122183dfe2f3b091e828b0a6a77d4 Sep 17 07:30:56.914471 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:56.914421 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c655bd8c-xvl6f" event={"ID":"137a96f3-3205-431b-9034-de0e2fd7b359","Type":"ContainerStarted","Data":"95fd5e439284abcd84f1877bfa08a87474b1d7bacd1e72e0988471fdd0bd2060"} Sep 17 07:30:56.914942 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:56.914490 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c655bd8c-xvl6f" event={"ID":"137a96f3-3205-431b-9034-de0e2fd7b359","Type":"ContainerStarted","Data":"879aae759813c65625c6d091209b719f755122183dfe2f3b091e828b0a6a77d4"} Sep 17 07:30:56.962893 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:56.962827 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c655bd8c-xvl6f" podStartSLOduration=1.962810757 podStartE2EDuration="1.962810757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:30:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:30:56.961640978 +0000 UTC m=+178.519825814" watchObservedRunningTime="2026-09-17 07:30:56.962810757 +0000 UTC m=+178.520995565" Sep 17 07:30:57.927048 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:57.927013 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0","Type":"ContainerStarted","Data":"ef0c407f4b6f90a0ef84d98f4bda5ce36a0273c375f4cb3bda283b0ca92bcb83"} Sep 17 07:30:57.927492 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:57.927125 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0","Type":"ContainerStarted","Data":"4f78ffe72eb3ac50d6c7d9e8ff7e6bfd5beeccc05dc453d7f26c78fd303e4fa5"} Sep 17 07:30:57.927492 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:57.927174 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0","Type":"ContainerStarted","Data":"301f9153362b58ea169cfe89e8b6643cc447766132c80204e41d3e0539c668bd"} Sep 17 07:30:57.927492 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:57.927220 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0","Type":"ContainerStarted","Data":"bd327b61bdc815ca37eea2ff245b90533e33401ed7ac31aaa3a70f124a5d0150"} Sep 17 07:30:57.927492 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:57.927258 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0","Type":"ContainerStarted","Data":"743ad4d7797a920331711f7bbfe258c290baae8bf9ecaf463d7ac679d85cf854"} Sep 17 07:30:58.933984 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:58.933960 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f749cdd2-2dc4-475f-be93-a7dc4d12bfc0","Type":"ContainerStarted","Data":"745c3898be035153990eb5c9dd2d6ecdc43805b48f6d33a71f54b75a78f2506e"} Sep 17 07:30:58.972026 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:58.971985 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.811534698 podStartE2EDuration="4.971968515s" totalImagesPullingTime="2.14591458s" totalInitContainerRuntime="14.519237ms" isStatefulPod=false podCreationTimestamp="2026-09-17 07:30:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:30:58.971521 +0000 UTC m=+180.529705807" watchObservedRunningTime="2026-09-17 07:30:58.971968515 +0000 UTC m=+180.530153323" Sep 17 07:30:59.883152 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:59.883116 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:30:59.958855 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:30:59.958815 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:31:05.354748 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:05.354701 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:31:05.448109 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:05.448079 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:31:05.542637 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:05.542609 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:31:06.021853 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:06.021827 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:31:06.022016 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:06.021873 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:31:06.026373 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:06.026352 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:31:06.970148 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:06.970114 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:31:07.023027 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:07.022997 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5947c9574b-f9h8j"] Sep 17 07:31:07.138150 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:07.138115 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:31:07.239603 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:07.239534 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:31:07.293818 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:07.293788 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:31:18.025337 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:18.025295 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:31:23.817461 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:23.817412 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:31:23.910369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:23.910338 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:31:32.047553 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:32.047483 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5947c9574b-f9h8j" podUID="293f041b-5ce3-4e35-8e3c-a42072574d10" containerName="console" containerID="cri-o://169a66ba3cb0ed5d0c1da79af1994b384cbb91e84fe8281faa9ec66a316180ac" gracePeriod=15 Sep 17 07:31:32.286695 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:32.286672 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:31:32.341149 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:32.341094 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/293f041b-5ce3-4e35-8e3c-a42072574d10-console-oauth-config\") pod \"293f041b-5ce3-4e35-8e3c-a42072574d10\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " Sep 17 07:31:32.341149 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:32.341133 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/293f041b-5ce3-4e35-8e3c-a42072574d10-oauth-serving-cert\") pod \"293f041b-5ce3-4e35-8e3c-a42072574d10\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " Sep 17 07:31:32.341328 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:32.341151 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/293f041b-5ce3-4e35-8e3c-a42072574d10-trusted-ca-bundle\") pod \"293f041b-5ce3-4e35-8e3c-a42072574d10\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " Sep 17 07:31:32.341328 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:32.341182 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-snvbn\" (UniqueName: \"kubernetes.io/projected/293f041b-5ce3-4e35-8e3c-a42072574d10-kube-api-access-snvbn\") pod \"293f041b-5ce3-4e35-8e3c-a42072574d10\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " Sep 17 07:31:32.341328 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:32.341207 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/293f041b-5ce3-4e35-8e3c-a42072574d10-service-ca\") pod \"293f041b-5ce3-4e35-8e3c-a42072574d10\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " Sep 17 07:31:32.341328 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:32.341235 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/293f041b-5ce3-4e35-8e3c-a42072574d10-console-config\") pod \"293f041b-5ce3-4e35-8e3c-a42072574d10\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " Sep 17 07:31:32.341530 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:32.341343 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/293f041b-5ce3-4e35-8e3c-a42072574d10-console-serving-cert\") pod \"293f041b-5ce3-4e35-8e3c-a42072574d10\" (UID: \"293f041b-5ce3-4e35-8e3c-a42072574d10\") " Sep 17 07:31:32.341648 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:32.341624 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/293f041b-5ce3-4e35-8e3c-a42072574d10-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "293f041b-5ce3-4e35-8e3c-a42072574d10" (UID: "293f041b-5ce3-4e35-8e3c-a42072574d10"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:31:32.341707 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:32.341630 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/293f041b-5ce3-4e35-8e3c-a42072574d10-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "293f041b-5ce3-4e35-8e3c-a42072574d10" (UID: "293f041b-5ce3-4e35-8e3c-a42072574d10"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:31:32.341707 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:32.341668 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/293f041b-5ce3-4e35-8e3c-a42072574d10-service-ca" (OuterVolumeSpecName: "service-ca") pod "293f041b-5ce3-4e35-8e3c-a42072574d10" (UID: "293f041b-5ce3-4e35-8e3c-a42072574d10"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:31:32.341707 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:32.341675 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/293f041b-5ce3-4e35-8e3c-a42072574d10-console-config" (OuterVolumeSpecName: "console-config") pod "293f041b-5ce3-4e35-8e3c-a42072574d10" (UID: "293f041b-5ce3-4e35-8e3c-a42072574d10"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:31:32.343263 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:32.343242 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/293f041b-5ce3-4e35-8e3c-a42072574d10-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "293f041b-5ce3-4e35-8e3c-a42072574d10" (UID: "293f041b-5ce3-4e35-8e3c-a42072574d10"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:31:32.343494 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:32.343472 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/293f041b-5ce3-4e35-8e3c-a42072574d10-kube-api-access-snvbn" (OuterVolumeSpecName: "kube-api-access-snvbn") pod "293f041b-5ce3-4e35-8e3c-a42072574d10" (UID: "293f041b-5ce3-4e35-8e3c-a42072574d10"). InnerVolumeSpecName "kube-api-access-snvbn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:31:32.343494 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:32.343470 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/293f041b-5ce3-4e35-8e3c-a42072574d10-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "293f041b-5ce3-4e35-8e3c-a42072574d10" (UID: "293f041b-5ce3-4e35-8e3c-a42072574d10"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:31:32.441971 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:32.441936 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/293f041b-5ce3-4e35-8e3c-a42072574d10-oauth-serving-cert\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:31:32.441971 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:32.441968 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/293f041b-5ce3-4e35-8e3c-a42072574d10-trusted-ca-bundle\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:31:32.441971 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:32.441978 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snvbn\" (UniqueName: \"kubernetes.io/projected/293f041b-5ce3-4e35-8e3c-a42072574d10-kube-api-access-snvbn\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:31:32.442137 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:32.441987 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/293f041b-5ce3-4e35-8e3c-a42072574d10-service-ca\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:31:32.442137 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:32.441996 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/293f041b-5ce3-4e35-8e3c-a42072574d10-console-config\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:31:32.442137 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:32.442003 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/293f041b-5ce3-4e35-8e3c-a42072574d10-console-serving-cert\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:31:32.442137 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:32.442011 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/293f041b-5ce3-4e35-8e3c-a42072574d10-console-oauth-config\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:31:33.064735 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:33.064706 2553 generic.go:356] "Generic (PLEG): container finished" podID="293f041b-5ce3-4e35-8e3c-a42072574d10" containerID="169a66ba3cb0ed5d0c1da79af1994b384cbb91e84fe8281faa9ec66a316180ac" exitCode=0 Sep 17 07:31:33.065148 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:33.064753 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5947c9574b-f9h8j" Sep 17 07:31:33.065148 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:33.064803 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5947c9574b-f9h8j" event={"ID":"293f041b-5ce3-4e35-8e3c-a42072574d10","Type":"ContainerDied","Data":"169a66ba3cb0ed5d0c1da79af1994b384cbb91e84fe8281faa9ec66a316180ac"} Sep 17 07:31:33.065148 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:33.064857 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5947c9574b-f9h8j" event={"ID":"293f041b-5ce3-4e35-8e3c-a42072574d10","Type":"ContainerDied","Data":"a103eb087ebd0ce363b69e3fe34a678d3c393a057a17165c2d1dd5d1127296fd"} Sep 17 07:31:33.065148 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:33.064895 2553 scope.go:117] "RemoveContainer" containerID="169a66ba3cb0ed5d0c1da79af1994b384cbb91e84fe8281faa9ec66a316180ac" Sep 17 07:31:33.073221 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:33.073211 2553 scope.go:117] "RemoveContainer" containerID="169a66ba3cb0ed5d0c1da79af1994b384cbb91e84fe8281faa9ec66a316180ac" Sep 17 07:31:33.073495 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:31:33.073479 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"169a66ba3cb0ed5d0c1da79af1994b384cbb91e84fe8281faa9ec66a316180ac\": container with ID starting with 169a66ba3cb0ed5d0c1da79af1994b384cbb91e84fe8281faa9ec66a316180ac not found: ID does not exist" containerID="169a66ba3cb0ed5d0c1da79af1994b384cbb91e84fe8281faa9ec66a316180ac" Sep 17 07:31:33.073543 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:33.073501 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"169a66ba3cb0ed5d0c1da79af1994b384cbb91e84fe8281faa9ec66a316180ac"} err="failed to get container status \"169a66ba3cb0ed5d0c1da79af1994b384cbb91e84fe8281faa9ec66a316180ac\": rpc error: code = NotFound desc = could not find container \"169a66ba3cb0ed5d0c1da79af1994b384cbb91e84fe8281faa9ec66a316180ac\": container with ID starting with 169a66ba3cb0ed5d0c1da79af1994b384cbb91e84fe8281faa9ec66a316180ac not found: ID does not exist" Sep 17 07:31:33.081654 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:33.081633 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5947c9574b-f9h8j"] Sep 17 07:31:33.086508 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:33.086396 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5947c9574b-f9h8j"] Sep 17 07:31:34.933576 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:31:34.933537 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="293f041b-5ce3-4e35-8e3c-a42072574d10" path="/var/lib/kubelet/pods/293f041b-5ce3-4e35-8e3c-a42072574d10/volumes" Sep 17 07:32:58.810951 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:32:58.810921 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-5kks2_2455e65f-98bb-463d-a204-5dbd5d69b5f3/console-operator/1.log" Sep 17 07:32:58.818295 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:32:58.818262 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-5kks2_2455e65f-98bb-463d-a204-5dbd5d69b5f3/console-operator/1.log" Sep 17 07:32:58.822047 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:32:58.822024 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/ovn-acl-logging/0.log" Sep 17 07:32:58.825064 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:32:58.825043 2553 kubelet.go:1648] "Image garbage collection succeeded" Sep 17 07:32:58.825844 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:32:58.825828 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/ovn-acl-logging/0.log" Sep 17 07:33:15.895380 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:15.895347 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:33:16.720847 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:16.720800 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-757744b5df-mvlhw"] Sep 17 07:33:16.721163 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:16.721148 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="293f041b-5ce3-4e35-8e3c-a42072574d10" containerName="console" Sep 17 07:33:16.721222 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:16.721165 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="293f041b-5ce3-4e35-8e3c-a42072574d10" containerName="console" Sep 17 07:33:16.721296 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:16.721229 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="293f041b-5ce3-4e35-8e3c-a42072574d10" containerName="console" Sep 17 07:33:16.724014 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:16.723998 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-757744b5df-mvlhw" Sep 17 07:33:16.726800 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:16.726778 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 17 07:33:16.726957 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:16.726846 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 17 07:33:16.727894 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:16.727861 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-52p6b\"" Sep 17 07:33:16.733057 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:16.733032 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-757744b5df-mvlhw"] Sep 17 07:33:16.794800 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:16.794776 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jswdg\" (UniqueName: \"kubernetes.io/projected/5b6736ce-f9b4-4f30-823c-701eaf30331d-kube-api-access-jswdg\") pod \"feast-operator-controller-manager-757744b5df-mvlhw\" (UID: \"5b6736ce-f9b4-4f30-823c-701eaf30331d\") " pod="feast-operator-system/feast-operator-controller-manager-757744b5df-mvlhw" Sep 17 07:33:16.895513 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:16.895481 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jswdg\" (UniqueName: \"kubernetes.io/projected/5b6736ce-f9b4-4f30-823c-701eaf30331d-kube-api-access-jswdg\") pod \"feast-operator-controller-manager-757744b5df-mvlhw\" (UID: \"5b6736ce-f9b4-4f30-823c-701eaf30331d\") " pod="feast-operator-system/feast-operator-controller-manager-757744b5df-mvlhw" Sep 17 07:33:16.904836 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:16.904811 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jswdg\" (UniqueName: \"kubernetes.io/projected/5b6736ce-f9b4-4f30-823c-701eaf30331d-kube-api-access-jswdg\") pod \"feast-operator-controller-manager-757744b5df-mvlhw\" (UID: \"5b6736ce-f9b4-4f30-823c-701eaf30331d\") " pod="feast-operator-system/feast-operator-controller-manager-757744b5df-mvlhw" Sep 17 07:33:17.035204 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:17.035151 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-757744b5df-mvlhw" Sep 17 07:33:17.304205 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:17.304141 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-757744b5df-mvlhw"] Sep 17 07:33:17.306920 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:33:17.306890 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b6736ce_f9b4_4f30_823c_701eaf30331d.slice/crio-54cf8d314b1761a28b0e7cfc1cb53e2d970f0e7d99bff6d7cf208e8016fb84da WatchSource:0}: Error finding container 54cf8d314b1761a28b0e7cfc1cb53e2d970f0e7d99bff6d7cf208e8016fb84da: Status 404 returned error can't find the container with id 54cf8d314b1761a28b0e7cfc1cb53e2d970f0e7d99bff6d7cf208e8016fb84da Sep 17 07:33:17.308840 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:17.308817 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 07:33:17.461874 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:17.461847 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-757744b5df-mvlhw" event={"ID":"5b6736ce-f9b4-4f30-823c-701eaf30331d","Type":"ContainerStarted","Data":"54cf8d314b1761a28b0e7cfc1cb53e2d970f0e7d99bff6d7cf208e8016fb84da"} Sep 17 07:33:17.523525 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:17.523497 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:33:20.478181 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:20.478138 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-757744b5df-mvlhw" event={"ID":"5b6736ce-f9b4-4f30-823c-701eaf30331d","Type":"ContainerStarted","Data":"4d69cbf52122022c20751cb5e955f04067101c42455db21a00069489410143ce"} Sep 17 07:33:20.478587 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:20.478262 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-757744b5df-mvlhw" Sep 17 07:33:20.503324 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:20.503271 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-757744b5df-mvlhw" podStartSLOduration=2.086219305 podStartE2EDuration="4.503258988s" totalImagesPullingTime="2.417039683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:33:20.503005842 +0000 UTC m=+322.061190668" watchObservedRunningTime="2026-09-17 07:33:20.503258988 +0000 UTC m=+322.061443835" Sep 17 07:33:21.723621 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:21.723589 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:33:21.824486 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:21.824449 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:33:23.720665 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:23.720627 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:33:23.783349 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:23.783315 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:33:31.484682 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:31.484648 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-757744b5df-mvlhw" Sep 17 07:33:55.656331 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:55.656294 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:33:55.755311 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:55.755280 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:33:56.329594 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.329562 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:33:56.430363 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.430329 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:33:56.525244 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.525219 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b"] Sep 17 07:33:56.530251 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.530222 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:33:56.538481 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.538457 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-p6677\"" Sep 17 07:33:56.538779 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.538756 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 17 07:33:56.539093 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.539052 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 17 07:33:56.539313 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.539292 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 17 07:33:56.539407 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.539364 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 17 07:33:56.539494 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.539478 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 17 07:33:56.539551 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.539520 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 17 07:33:56.541625 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.541607 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 17 07:33:56.549738 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.549717 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:33:56.550325 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.550306 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b"] Sep 17 07:33:56.677772 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.677684 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-ui-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-5bt2b\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:33:56.677772 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.677733 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-registry-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-5bt2b\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:33:56.678218 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.677784 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-online-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-5bt2b\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:33:56.678218 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.677831 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49dfl\" (UniqueName: \"kubernetes.io/projected/c368c010-42a2-4eb3-9183-4d76df8346f3-kube-api-access-49dfl\") pod \"feast-simple-feast-setup-66c88fcfbf-5bt2b\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:33:56.678218 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.677915 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c368c010-42a2-4eb3-9183-4d76df8346f3-feast-data\") pod \"feast-simple-feast-setup-66c88fcfbf-5bt2b\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:33:56.678218 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.677959 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-offline-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-5bt2b\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:33:56.736308 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.736274 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:33:56.778757 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.778726 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-ui-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-5bt2b\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:33:56.778865 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.778764 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-registry-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-5bt2b\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:33:56.778865 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.778788 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-online-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-5bt2b\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:33:56.778865 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.778834 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49dfl\" (UniqueName: \"kubernetes.io/projected/c368c010-42a2-4eb3-9183-4d76df8346f3-kube-api-access-49dfl\") pod \"feast-simple-feast-setup-66c88fcfbf-5bt2b\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:33:56.779053 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.778898 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c368c010-42a2-4eb3-9183-4d76df8346f3-feast-data\") pod \"feast-simple-feast-setup-66c88fcfbf-5bt2b\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:33:56.779053 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.778935 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-offline-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-5bt2b\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:33:56.779053 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:33:56.778963 2553 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 17 07:33:56.779053 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:33:56.779028 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-registry-tls podName:c368c010-42a2-4eb3-9183-4d76df8346f3 nodeName:}" failed. No retries permitted until 2026-09-17 07:33:57.279008649 +0000 UTC m=+358.837193450 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-registry-tls") pod "feast-simple-feast-setup-66c88fcfbf-5bt2b" (UID: "c368c010-42a2-4eb3-9183-4d76df8346f3") : secret "feast-simple-feast-setup-registry-tls" not found Sep 17 07:33:56.779434 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.779414 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c368c010-42a2-4eb3-9183-4d76df8346f3-feast-data\") pod \"feast-simple-feast-setup-66c88fcfbf-5bt2b\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:33:56.781255 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.781234 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-ui-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-5bt2b\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:33:56.781755 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.781736 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-offline-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-5bt2b\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:33:56.781816 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.781757 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-online-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-5bt2b\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:33:56.799927 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.799904 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49dfl\" (UniqueName: \"kubernetes.io/projected/c368c010-42a2-4eb3-9183-4d76df8346f3-kube-api-access-49dfl\") pod \"feast-simple-feast-setup-66c88fcfbf-5bt2b\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:33:56.848072 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.848047 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:33:56.941768 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:56.941712 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:33:57.283135 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:57.283109 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-registry-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-5bt2b\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:33:57.285275 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:57.285249 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-registry-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-5bt2b\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:33:57.451832 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:57.451799 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:33:57.731142 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:57.731116 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b"] Sep 17 07:33:58.631830 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:33:58.631769 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" event={"ID":"c368c010-42a2-4eb3-9183-4d76df8346f3","Type":"ContainerStarted","Data":"a7a892368da23a618c217dc0104921f39ca5bdb00a0392e28f1b8b5b1b67259a"} Sep 17 07:34:08.099359 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:08.099321 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:08.153324 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:08.153298 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:11.019515 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:11.019474 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:17.917725 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:17.917691 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b674557db-pfdg8"] Sep 17 07:34:17.981280 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:17.981246 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b674557db-pfdg8"] Sep 17 07:34:17.981444 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:17.981392 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:17.998241 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:17.998168 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:18.063432 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.063401 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:18.128940 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.128913 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:18.149406 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.149369 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea856b35-94ab-40f2-add2-c2007255505f-console-serving-cert\") pod \"console-5b674557db-pfdg8\" (UID: \"ea856b35-94ab-40f2-add2-c2007255505f\") " pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:18.149540 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.149441 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea856b35-94ab-40f2-add2-c2007255505f-oauth-serving-cert\") pod \"console-5b674557db-pfdg8\" (UID: \"ea856b35-94ab-40f2-add2-c2007255505f\") " pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:18.149607 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.149537 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea856b35-94ab-40f2-add2-c2007255505f-service-ca\") pod \"console-5b674557db-pfdg8\" (UID: \"ea856b35-94ab-40f2-add2-c2007255505f\") " pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:18.149607 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.149570 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea856b35-94ab-40f2-add2-c2007255505f-trusted-ca-bundle\") pod \"console-5b674557db-pfdg8\" (UID: \"ea856b35-94ab-40f2-add2-c2007255505f\") " pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:18.149607 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.149601 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjthl\" (UniqueName: \"kubernetes.io/projected/ea856b35-94ab-40f2-add2-c2007255505f-kube-api-access-fjthl\") pod \"console-5b674557db-pfdg8\" (UID: \"ea856b35-94ab-40f2-add2-c2007255505f\") " pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:18.149763 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.149706 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea856b35-94ab-40f2-add2-c2007255505f-console-config\") pod \"console-5b674557db-pfdg8\" (UID: \"ea856b35-94ab-40f2-add2-c2007255505f\") " pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:18.149867 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.149848 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea856b35-94ab-40f2-add2-c2007255505f-console-oauth-config\") pod \"console-5b674557db-pfdg8\" (UID: \"ea856b35-94ab-40f2-add2-c2007255505f\") " pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:18.250793 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.250760 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea856b35-94ab-40f2-add2-c2007255505f-console-config\") pod \"console-5b674557db-pfdg8\" (UID: \"ea856b35-94ab-40f2-add2-c2007255505f\") " pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:18.250988 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.250822 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea856b35-94ab-40f2-add2-c2007255505f-console-oauth-config\") pod \"console-5b674557db-pfdg8\" (UID: \"ea856b35-94ab-40f2-add2-c2007255505f\") " pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:18.250988 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.250866 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea856b35-94ab-40f2-add2-c2007255505f-console-serving-cert\") pod \"console-5b674557db-pfdg8\" (UID: \"ea856b35-94ab-40f2-add2-c2007255505f\") " pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:18.250988 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.250935 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea856b35-94ab-40f2-add2-c2007255505f-oauth-serving-cert\") pod \"console-5b674557db-pfdg8\" (UID: \"ea856b35-94ab-40f2-add2-c2007255505f\") " pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:18.251174 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.250996 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea856b35-94ab-40f2-add2-c2007255505f-service-ca\") pod \"console-5b674557db-pfdg8\" (UID: \"ea856b35-94ab-40f2-add2-c2007255505f\") " pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:18.251174 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.251019 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea856b35-94ab-40f2-add2-c2007255505f-trusted-ca-bundle\") pod \"console-5b674557db-pfdg8\" (UID: \"ea856b35-94ab-40f2-add2-c2007255505f\") " pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:18.251174 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.251044 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjthl\" (UniqueName: \"kubernetes.io/projected/ea856b35-94ab-40f2-add2-c2007255505f-kube-api-access-fjthl\") pod \"console-5b674557db-pfdg8\" (UID: \"ea856b35-94ab-40f2-add2-c2007255505f\") " pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:18.251649 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.251581 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea856b35-94ab-40f2-add2-c2007255505f-console-config\") pod \"console-5b674557db-pfdg8\" (UID: \"ea856b35-94ab-40f2-add2-c2007255505f\") " pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:18.251903 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.251868 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea856b35-94ab-40f2-add2-c2007255505f-oauth-serving-cert\") pod \"console-5b674557db-pfdg8\" (UID: \"ea856b35-94ab-40f2-add2-c2007255505f\") " pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:18.251903 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.251870 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea856b35-94ab-40f2-add2-c2007255505f-service-ca\") pod \"console-5b674557db-pfdg8\" (UID: \"ea856b35-94ab-40f2-add2-c2007255505f\") " pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:18.252100 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.251940 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea856b35-94ab-40f2-add2-c2007255505f-trusted-ca-bundle\") pod \"console-5b674557db-pfdg8\" (UID: \"ea856b35-94ab-40f2-add2-c2007255505f\") " pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:18.253619 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.253595 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea856b35-94ab-40f2-add2-c2007255505f-console-oauth-config\") pod \"console-5b674557db-pfdg8\" (UID: \"ea856b35-94ab-40f2-add2-c2007255505f\") " pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:18.253911 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.253867 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea856b35-94ab-40f2-add2-c2007255505f-console-serving-cert\") pod \"console-5b674557db-pfdg8\" (UID: \"ea856b35-94ab-40f2-add2-c2007255505f\") " pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:18.261534 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.261516 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjthl\" (UniqueName: \"kubernetes.io/projected/ea856b35-94ab-40f2-add2-c2007255505f-kube-api-access-fjthl\") pod \"console-5b674557db-pfdg8\" (UID: \"ea856b35-94ab-40f2-add2-c2007255505f\") " pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:18.295526 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.295501 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:18.708385 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.708339 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:18.793451 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:18.793418 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:24.973610 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:24.973581 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b674557db-pfdg8"] Sep 17 07:34:24.977346 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:34:24.977322 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea856b35_94ab_40f2_add2_c2007255505f.slice/crio-3052c29cc059e285a6e549496670ee8c34ec437b747399faf51625de4c6c9474 WatchSource:0}: Error finding container 3052c29cc059e285a6e549496670ee8c34ec437b747399faf51625de4c6c9474: Status 404 returned error can't find the container with id 3052c29cc059e285a6e549496670ee8c34ec437b747399faf51625de4c6c9474 Sep 17 07:34:25.751598 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:25.751566 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b674557db-pfdg8" event={"ID":"ea856b35-94ab-40f2-add2-c2007255505f","Type":"ContainerStarted","Data":"e7cee8d601b3cd688c0b19a6868ba271cae3869fbffab443f63aa070a40714e0"} Sep 17 07:34:25.751796 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:25.751610 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b674557db-pfdg8" event={"ID":"ea856b35-94ab-40f2-add2-c2007255505f","Type":"ContainerStarted","Data":"3052c29cc059e285a6e549496670ee8c34ec437b747399faf51625de4c6c9474"} Sep 17 07:34:25.753133 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:25.753100 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" event={"ID":"c368c010-42a2-4eb3-9183-4d76df8346f3","Type":"ContainerStarted","Data":"f3fab9b469eae1e8213a578dbe11e8e971a1d678eb3ee0c1d9bcdd88d2dffb5a"} Sep 17 07:34:25.783144 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:25.783097 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b674557db-pfdg8" podStartSLOduration=8.783082946 podStartE2EDuration="8.783082946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:34:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:34:25.781866578 +0000 UTC m=+387.340051389" watchObservedRunningTime="2026-09-17 07:34:25.783082946 +0000 UTC m=+387.341267788" Sep 17 07:34:28.296008 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:28.295973 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:28.296359 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:28.296043 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:28.300580 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:28.300559 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:28.768565 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:28.768537 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b674557db-pfdg8" Sep 17 07:34:28.824355 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:28.824321 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c655bd8c-xvl6f"] Sep 17 07:34:28.969791 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:28.969761 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:29.068525 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:29.068452 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:29.134657 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:29.134623 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:29.770817 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:29.770789 2553 generic.go:356] "Generic (PLEG): container finished" podID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerID="f3fab9b469eae1e8213a578dbe11e8e971a1d678eb3ee0c1d9bcdd88d2dffb5a" exitCode=0 Sep 17 07:34:29.771213 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:29.770869 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" event={"ID":"c368c010-42a2-4eb3-9183-4d76df8346f3","Type":"ContainerDied","Data":"f3fab9b469eae1e8213a578dbe11e8e971a1d678eb3ee0c1d9bcdd88d2dffb5a"} Sep 17 07:34:30.778057 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:30.778017 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" event={"ID":"c368c010-42a2-4eb3-9183-4d76df8346f3","Type":"ContainerStarted","Data":"e09e806a46cba4eebb7b90b2a3dba7339177aa40b82de762bf1389c4fb1b6331"} Sep 17 07:34:34.134074 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:34.134029 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:36.804445 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:36.804408 2553 generic.go:356] "Generic (PLEG): container finished" podID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerID="e09e806a46cba4eebb7b90b2a3dba7339177aa40b82de762bf1389c4fb1b6331" exitCode=0 Sep 17 07:34:36.804851 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:36.804475 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" event={"ID":"c368c010-42a2-4eb3-9183-4d76df8346f3","Type":"ContainerDied","Data":"e09e806a46cba4eebb7b90b2a3dba7339177aa40b82de762bf1389c4fb1b6331"} Sep 17 07:34:36.813264 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:36.813246 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 17 07:34:37.814873 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:37.814830 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" event={"ID":"c368c010-42a2-4eb3-9183-4d76df8346f3","Type":"ContainerStarted","Data":"9b98ded8bd5a250f3e1258d96dbe9946a1f930010bb153269a93c8151464de5b"} Sep 17 07:34:37.815363 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:37.814911 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" event={"ID":"c368c010-42a2-4eb3-9183-4d76df8346f3","Type":"ContainerStarted","Data":"3eaa53f338ef1b8fc76eefa496b4f156fcb9521609542f62b6991c9fd503640c"} Sep 17 07:34:37.815363 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:37.814934 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" event={"ID":"c368c010-42a2-4eb3-9183-4d76df8346f3","Type":"ContainerStarted","Data":"85a7a5694f7e42eadafe074a6a28ab941549da190f54e18aabfa7498b49df671"} Sep 17 07:34:37.815363 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:37.814951 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" event={"ID":"c368c010-42a2-4eb3-9183-4d76df8346f3","Type":"ContainerStarted","Data":"34964b8cdeb1cb29f483bf836bf1bd769d3df13266735d52fbb90be9cf0a511c"} Sep 17 07:34:37.845430 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:37.845334 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" podStartSLOduration=14.779122784 podStartE2EDuration="41.845308573s" totalImagesPullingTime="27.066185789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:33:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:34:37.844173385 +0000 UTC m=+399.402358223" watchObservedRunningTime="2026-09-17 07:34:37.845308573 +0000 UTC m=+399.403493399" Sep 17 07:34:39.452541 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:39.452467 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:34:39.452541 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:39.452533 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:34:39.453117 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:39.452555 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:34:39.453117 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:39.453034 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:34:39.454659 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:39.454625 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 17 07:34:39.454905 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:39.454638 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.26:6571: connect: connection refused" Sep 17 07:34:39.455085 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:39.454639 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused" Sep 17 07:34:39.455204 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:39.454692 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.26:8816: connect: connection refused" Sep 17 07:34:42.452428 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:42.452375 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.26:6571: connect: connection refused" Sep 17 07:34:42.452428 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:42.452418 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 17 07:34:42.453019 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:42.452381 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.26:8816: connect: connection refused" Sep 17 07:34:42.453019 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:42.452452 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused" Sep 17 07:34:45.453019 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:45.452990 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:34:45.453433 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:45.453085 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:34:45.453433 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:45.453385 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:34:45.454019 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:45.453996 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:34:45.454102 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:45.454037 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:34:45.454102 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:45.454055 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:34:45.454276 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:45.454241 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:34:45.454412 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:45.454394 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:34:45.455007 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:45.454992 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:34:45.458409 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:45.458392 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:34:45.858225 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:45.858155 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:34:45.861364 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:45.861346 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:34:46.366214 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:46.366161 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:34:53.844607 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:53.844551 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c655bd8c-xvl6f" podUID="137a96f3-3205-431b-9034-de0e2fd7b359" containerName="console" containerID="cri-o://95fd5e439284abcd84f1877bfa08a87474b1d7bacd1e72e0988471fdd0bd2060" gracePeriod=15 Sep 17 07:34:54.092969 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.092943 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:34:54.157409 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.157318 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/137a96f3-3205-431b-9034-de0e2fd7b359-oauth-serving-cert\") pod \"137a96f3-3205-431b-9034-de0e2fd7b359\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " Sep 17 07:34:54.157409 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.157364 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/137a96f3-3205-431b-9034-de0e2fd7b359-console-oauth-config\") pod \"137a96f3-3205-431b-9034-de0e2fd7b359\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " Sep 17 07:34:54.157626 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.157425 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/137a96f3-3205-431b-9034-de0e2fd7b359-service-ca\") pod \"137a96f3-3205-431b-9034-de0e2fd7b359\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " Sep 17 07:34:54.157626 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.157462 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2q4z5\" (UniqueName: \"kubernetes.io/projected/137a96f3-3205-431b-9034-de0e2fd7b359-kube-api-access-2q4z5\") pod \"137a96f3-3205-431b-9034-de0e2fd7b359\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " Sep 17 07:34:54.157626 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.157525 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/137a96f3-3205-431b-9034-de0e2fd7b359-console-config\") pod \"137a96f3-3205-431b-9034-de0e2fd7b359\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " Sep 17 07:34:54.157626 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.157550 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/137a96f3-3205-431b-9034-de0e2fd7b359-console-serving-cert\") pod \"137a96f3-3205-431b-9034-de0e2fd7b359\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " Sep 17 07:34:54.157626 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.157587 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/137a96f3-3205-431b-9034-de0e2fd7b359-trusted-ca-bundle\") pod \"137a96f3-3205-431b-9034-de0e2fd7b359\" (UID: \"137a96f3-3205-431b-9034-de0e2fd7b359\") " Sep 17 07:34:54.157856 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.157726 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/137a96f3-3205-431b-9034-de0e2fd7b359-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "137a96f3-3205-431b-9034-de0e2fd7b359" (UID: "137a96f3-3205-431b-9034-de0e2fd7b359"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:34:54.157942 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.157902 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/137a96f3-3205-431b-9034-de0e2fd7b359-oauth-serving-cert\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:34:54.158008 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.157961 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/137a96f3-3205-431b-9034-de0e2fd7b359-console-config" (OuterVolumeSpecName: "console-config") pod "137a96f3-3205-431b-9034-de0e2fd7b359" (UID: "137a96f3-3205-431b-9034-de0e2fd7b359"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:34:54.158059 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.158006 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/137a96f3-3205-431b-9034-de0e2fd7b359-service-ca" (OuterVolumeSpecName: "service-ca") pod "137a96f3-3205-431b-9034-de0e2fd7b359" (UID: "137a96f3-3205-431b-9034-de0e2fd7b359"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:34:54.158059 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.158015 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/137a96f3-3205-431b-9034-de0e2fd7b359-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "137a96f3-3205-431b-9034-de0e2fd7b359" (UID: "137a96f3-3205-431b-9034-de0e2fd7b359"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:34:54.159735 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.159706 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/137a96f3-3205-431b-9034-de0e2fd7b359-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "137a96f3-3205-431b-9034-de0e2fd7b359" (UID: "137a96f3-3205-431b-9034-de0e2fd7b359"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:34:54.160042 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.160023 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/137a96f3-3205-431b-9034-de0e2fd7b359-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "137a96f3-3205-431b-9034-de0e2fd7b359" (UID: "137a96f3-3205-431b-9034-de0e2fd7b359"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:34:54.160106 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.160089 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/137a96f3-3205-431b-9034-de0e2fd7b359-kube-api-access-2q4z5" (OuterVolumeSpecName: "kube-api-access-2q4z5") pod "137a96f3-3205-431b-9034-de0e2fd7b359" (UID: "137a96f3-3205-431b-9034-de0e2fd7b359"). InnerVolumeSpecName "kube-api-access-2q4z5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:34:54.258963 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.258928 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/137a96f3-3205-431b-9034-de0e2fd7b359-console-config\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:34:54.258963 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.258956 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/137a96f3-3205-431b-9034-de0e2fd7b359-console-serving-cert\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:34:54.258963 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.258966 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/137a96f3-3205-431b-9034-de0e2fd7b359-trusted-ca-bundle\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:34:54.259166 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.258974 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/137a96f3-3205-431b-9034-de0e2fd7b359-console-oauth-config\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:34:54.259166 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.258983 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/137a96f3-3205-431b-9034-de0e2fd7b359-service-ca\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:34:54.259166 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.258991 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2q4z5\" (UniqueName: \"kubernetes.io/projected/137a96f3-3205-431b-9034-de0e2fd7b359-kube-api-access-2q4z5\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:34:54.894958 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.894922 2553 generic.go:356] "Generic (PLEG): container finished" podID="137a96f3-3205-431b-9034-de0e2fd7b359" containerID="95fd5e439284abcd84f1877bfa08a87474b1d7bacd1e72e0988471fdd0bd2060" exitCode=0 Sep 17 07:34:54.895358 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.894965 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c655bd8c-xvl6f" Sep 17 07:34:54.895358 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.894970 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c655bd8c-xvl6f" event={"ID":"137a96f3-3205-431b-9034-de0e2fd7b359","Type":"ContainerDied","Data":"95fd5e439284abcd84f1877bfa08a87474b1d7bacd1e72e0988471fdd0bd2060"} Sep 17 07:34:54.895358 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.895092 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c655bd8c-xvl6f" event={"ID":"137a96f3-3205-431b-9034-de0e2fd7b359","Type":"ContainerDied","Data":"879aae759813c65625c6d091209b719f755122183dfe2f3b091e828b0a6a77d4"} Sep 17 07:34:54.895358 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.895111 2553 scope.go:117] "RemoveContainer" containerID="95fd5e439284abcd84f1877bfa08a87474b1d7bacd1e72e0988471fdd0bd2060" Sep 17 07:34:54.903761 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.903745 2553 scope.go:117] "RemoveContainer" containerID="95fd5e439284abcd84f1877bfa08a87474b1d7bacd1e72e0988471fdd0bd2060" Sep 17 07:34:54.904034 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:34:54.904014 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95fd5e439284abcd84f1877bfa08a87474b1d7bacd1e72e0988471fdd0bd2060\": container with ID starting with 95fd5e439284abcd84f1877bfa08a87474b1d7bacd1e72e0988471fdd0bd2060 not found: ID does not exist" containerID="95fd5e439284abcd84f1877bfa08a87474b1d7bacd1e72e0988471fdd0bd2060" Sep 17 07:34:54.904094 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.904041 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95fd5e439284abcd84f1877bfa08a87474b1d7bacd1e72e0988471fdd0bd2060"} err="failed to get container status \"95fd5e439284abcd84f1877bfa08a87474b1d7bacd1e72e0988471fdd0bd2060\": rpc error: code = NotFound desc = could not find container \"95fd5e439284abcd84f1877bfa08a87474b1d7bacd1e72e0988471fdd0bd2060\": container with ID starting with 95fd5e439284abcd84f1877bfa08a87474b1d7bacd1e72e0988471fdd0bd2060 not found: ID does not exist" Sep 17 07:34:54.917736 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.917713 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c655bd8c-xvl6f"] Sep 17 07:34:54.925699 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.925680 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c655bd8c-xvl6f"] Sep 17 07:34:54.933096 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:34:54.933076 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="137a96f3-3205-431b-9034-de0e2fd7b359" path="/var/lib/kubelet/pods/137a96f3-3205-431b-9034-de0e2fd7b359/volumes" Sep 17 07:35:48.828031 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:48.827997 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:35:48.868106 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:48.868072 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b"] Sep 17 07:35:48.868376 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:48.868325 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="registry" containerID="cri-o://34964b8cdeb1cb29f483bf836bf1bd769d3df13266735d52fbb90be9cf0a511c" gracePeriod=30 Sep 17 07:35:48.868509 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:48.868377 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="ui" containerID="cri-o://9b98ded8bd5a250f3e1258d96dbe9946a1f930010bb153269a93c8151464de5b" gracePeriod=30 Sep 17 07:35:48.868509 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:48.868397 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="offline" containerID="cri-o://3eaa53f338ef1b8fc76eefa496b4f156fcb9521609542f62b6991c9fd503640c" gracePeriod=30 Sep 17 07:35:48.868620 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:48.868411 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="online" containerID="cri-o://85a7a5694f7e42eadafe074a6a28ab941549da190f54e18aabfa7498b49df671" gracePeriod=30 Sep 17 07:35:48.908977 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:48.908946 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:35:49.167104 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.166979 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:35:49.265614 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.265584 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:35:49.351526 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.351485 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx"] Sep 17 07:35:49.351938 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.351920 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="137a96f3-3205-431b-9034-de0e2fd7b359" containerName="console" Sep 17 07:35:49.352003 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.351942 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="137a96f3-3205-431b-9034-de0e2fd7b359" containerName="console" Sep 17 07:35:49.352066 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.352054 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="137a96f3-3205-431b-9034-de0e2fd7b359" containerName="console" Sep 17 07:35:49.355195 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.355171 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:35:49.358912 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.358622 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-nk6np\"" Sep 17 07:35:49.373515 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.373488 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-online-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-pc9jx\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:35:49.373642 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.373534 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-ui-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-pc9jx\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:35:49.373642 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.373569 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3cf15699-c783-49c2-96b6-af9aa5939d29-feast-data\") pod \"feast-simple-feast-setup-66c88fcfbf-pc9jx\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:35:49.373642 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.373622 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-offline-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-pc9jx\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:35:49.373748 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.373694 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-registry-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-pc9jx\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:35:49.373748 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.373733 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmr94\" (UniqueName: \"kubernetes.io/projected/3cf15699-c783-49c2-96b6-af9aa5939d29-kube-api-access-lmr94\") pod \"feast-simple-feast-setup-66c88fcfbf-pc9jx\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:35:49.381385 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.381349 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx"] Sep 17 07:35:49.419980 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.419860 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:35:49.474491 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.474444 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-ui-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-pc9jx\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:35:49.474687 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.474502 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3cf15699-c783-49c2-96b6-af9aa5939d29-feast-data\") pod \"feast-simple-feast-setup-66c88fcfbf-pc9jx\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:35:49.474687 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.474542 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-offline-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-pc9jx\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:35:49.474687 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.474595 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-registry-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-pc9jx\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:35:49.474687 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.474628 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmr94\" (UniqueName: \"kubernetes.io/projected/3cf15699-c783-49c2-96b6-af9aa5939d29-kube-api-access-lmr94\") pod \"feast-simple-feast-setup-66c88fcfbf-pc9jx\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:35:49.474687 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.474671 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-online-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-pc9jx\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:35:49.474960 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:35:49.474784 2553 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Sep 17 07:35:49.474960 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:35:49.474849 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-online-tls podName:3cf15699-c783-49c2-96b6-af9aa5939d29 nodeName:}" failed. No retries permitted until 2026-09-17 07:35:49.974826522 +0000 UTC m=+471.533011310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-online-tls") pod "feast-simple-feast-setup-66c88fcfbf-pc9jx" (UID: "3cf15699-c783-49c2-96b6-af9aa5939d29") : secret "feast-simple-feast-setup-online-tls" not found Sep 17 07:35:49.475114 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:35:49.475061 2553 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 17 07:35:49.475170 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:35:49.475125 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-registry-tls podName:3cf15699-c783-49c2-96b6-af9aa5939d29 nodeName:}" failed. No retries permitted until 2026-09-17 07:35:49.975108006 +0000 UTC m=+471.533292793 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-registry-tls") pod "feast-simple-feast-setup-66c88fcfbf-pc9jx" (UID: "3cf15699-c783-49c2-96b6-af9aa5939d29") : secret "feast-simple-feast-setup-registry-tls" not found Sep 17 07:35:49.475349 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:35:49.475331 2553 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 17 07:35:49.475472 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.475448 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3cf15699-c783-49c2-96b6-af9aa5939d29-feast-data\") pod \"feast-simple-feast-setup-66c88fcfbf-pc9jx\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:35:49.475598 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:35:49.475584 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-offline-tls podName:3cf15699-c783-49c2-96b6-af9aa5939d29 nodeName:}" failed. No retries permitted until 2026-09-17 07:35:49.975566252 +0000 UTC m=+471.533751052 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-offline-tls") pod "feast-simple-feast-setup-66c88fcfbf-pc9jx" (UID: "3cf15699-c783-49c2-96b6-af9aa5939d29") : secret "feast-simple-feast-setup-offline-tls" not found Sep 17 07:35:49.478231 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.478202 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-ui-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-pc9jx\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:35:49.485920 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.485848 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmr94\" (UniqueName: \"kubernetes.io/projected/3cf15699-c783-49c2-96b6-af9aa5939d29-kube-api-access-lmr94\") pod \"feast-simple-feast-setup-66c88fcfbf-pc9jx\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:35:49.728764 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.728721 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:35:49.829452 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.829414 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:35:49.912177 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.912143 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:35:49.978950 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.978916 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-online-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-pc9jx\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:35:49.979119 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.978977 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-offline-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-pc9jx\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:35:49.979119 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.979040 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-registry-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-pc9jx\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:35:49.981378 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.981351 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-registry-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-pc9jx\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:35:49.981475 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.981406 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-online-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-pc9jx\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:35:49.981515 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:49.981479 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-offline-tls\") pod \"feast-simple-feast-setup-66c88fcfbf-pc9jx\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:35:50.108621 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:50.108588 2553 generic.go:356] "Generic (PLEG): container finished" podID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerID="9b98ded8bd5a250f3e1258d96dbe9946a1f930010bb153269a93c8151464de5b" exitCode=0 Sep 17 07:35:50.108621 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:50.108610 2553 generic.go:356] "Generic (PLEG): container finished" podID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerID="85a7a5694f7e42eadafe074a6a28ab941549da190f54e18aabfa7498b49df671" exitCode=0 Sep 17 07:35:50.108803 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:50.108660 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" event={"ID":"c368c010-42a2-4eb3-9183-4d76df8346f3","Type":"ContainerDied","Data":"9b98ded8bd5a250f3e1258d96dbe9946a1f930010bb153269a93c8151464de5b"} Sep 17 07:35:50.108803 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:50.108702 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" event={"ID":"c368c010-42a2-4eb3-9183-4d76df8346f3","Type":"ContainerDied","Data":"85a7a5694f7e42eadafe074a6a28ab941549da190f54e18aabfa7498b49df671"} Sep 17 07:35:50.266565 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:50.266494 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:35:50.549688 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:50.549615 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx"] Sep 17 07:35:50.552686 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:35:50.552658 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3cf15699_c783_49c2_96b6_af9aa5939d29.slice/crio-beb16b6ce91d673e21865694fab84303fe352ae3f17f02a5539b7024738445a1 WatchSource:0}: Error finding container beb16b6ce91d673e21865694fab84303fe352ae3f17f02a5539b7024738445a1: Status 404 returned error can't find the container with id beb16b6ce91d673e21865694fab84303fe352ae3f17f02a5539b7024738445a1 Sep 17 07:35:51.113700 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:51.113663 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" event={"ID":"3cf15699-c783-49c2-96b6-af9aa5939d29","Type":"ContainerStarted","Data":"d838c1af100afb86401ec7eb77bca37152e9afca63c04dfe00a13af5c3a7a5d5"} Sep 17 07:35:51.114106 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:51.113707 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" event={"ID":"3cf15699-c783-49c2-96b6-af9aa5939d29","Type":"ContainerStarted","Data":"beb16b6ce91d673e21865694fab84303fe352ae3f17f02a5539b7024738445a1"} Sep 17 07:35:55.454028 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:55.453969 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused" Sep 17 07:35:55.859203 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:55.859161 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 17 07:35:56.135478 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:56.135400 2553 generic.go:356] "Generic (PLEG): container finished" podID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerID="d838c1af100afb86401ec7eb77bca37152e9afca63c04dfe00a13af5c3a7a5d5" exitCode=0 Sep 17 07:35:56.135618 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:56.135475 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" event={"ID":"3cf15699-c783-49c2-96b6-af9aa5939d29","Type":"ContainerDied","Data":"d838c1af100afb86401ec7eb77bca37152e9afca63c04dfe00a13af5c3a7a5d5"} Sep 17 07:35:57.141476 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:35:57.141441 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" event={"ID":"3cf15699-c783-49c2-96b6-af9aa5939d29","Type":"ContainerStarted","Data":"cdd493e948111fe218f9d2e5e53ba97eb3e07dfb4613c2ddce9529593d45922e"} Sep 17 07:36:02.162527 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:02.162461 2553 generic.go:356] "Generic (PLEG): container finished" podID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerID="cdd493e948111fe218f9d2e5e53ba97eb3e07dfb4613c2ddce9529593d45922e" exitCode=0 Sep 17 07:36:02.162829 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:02.162535 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" event={"ID":"3cf15699-c783-49c2-96b6-af9aa5939d29","Type":"ContainerDied","Data":"cdd493e948111fe218f9d2e5e53ba97eb3e07dfb4613c2ddce9529593d45922e"} Sep 17 07:36:03.178810 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:03.178768 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" event={"ID":"3cf15699-c783-49c2-96b6-af9aa5939d29","Type":"ContainerStarted","Data":"2f957293428a06d806f4e15fe419cca0d0a187cffca02daa4c9cca0affdd75ec"} Sep 17 07:36:03.179303 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:03.178828 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" event={"ID":"3cf15699-c783-49c2-96b6-af9aa5939d29","Type":"ContainerStarted","Data":"7515a8f100456f68ef218fdcd879416200968a3df0308a8078957949e0c449f2"} Sep 17 07:36:03.179303 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:03.178850 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" event={"ID":"3cf15699-c783-49c2-96b6-af9aa5939d29","Type":"ContainerStarted","Data":"a3474f088ea79d32a6f70512c9af4de1144bfd155b19c678c1208d071b27f82a"} Sep 17 07:36:03.179303 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:03.178868 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" event={"ID":"3cf15699-c783-49c2-96b6-af9aa5939d29","Type":"ContainerStarted","Data":"2bc77b75e8b16f5e91e971db01072a7d9858447934329a57648da6c08f68585f"} Sep 17 07:36:03.212246 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:03.212150 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" podStartSLOduration=14.212128388 podStartE2EDuration="14.212128388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:35:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:36:03.210288139 +0000 UTC m=+484.768472946" watchObservedRunningTime="2026-09-17 07:36:03.212128388 +0000 UTC m=+484.770313197" Sep 17 07:36:05.267124 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:05.267035 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:36:05.267124 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:05.267101 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:36:05.267124 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:05.267122 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:36:05.267705 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:05.267139 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:36:05.269287 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:05.269154 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.28:8816: connect: connection refused" Sep 17 07:36:05.269287 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:05.269177 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.28:6571: connect: connection refused" Sep 17 07:36:05.269287 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:05.269182 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.28:8443: connect: connection refused" Sep 17 07:36:05.269287 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:05.269257 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused" Sep 17 07:36:05.455096 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:05.454849 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused" Sep 17 07:36:05.858571 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:05.858517 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 17 07:36:08.267415 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:08.267369 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.28:8816: connect: connection refused" Sep 17 07:36:08.267949 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:08.267424 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.28:6571: connect: connection refused" Sep 17 07:36:08.267949 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:08.267427 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused" Sep 17 07:36:08.267949 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:08.267370 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.28:8443: connect: connection refused" Sep 17 07:36:08.322125 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:08.322096 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:36:10.267134 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:10.267103 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:36:10.267549 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:10.267152 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:36:10.267549 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:10.267167 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:36:10.267549 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:10.267184 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:36:11.267640 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:11.267610 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:36:11.268111 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:11.268080 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:36:11.268250 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:11.268237 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:36:11.268818 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:11.268796 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:36:11.268932 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:11.268851 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:36:11.268993 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:11.268979 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:36:11.273095 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:11.273075 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:36:12.229312 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:12.229280 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:36:12.596456 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:12.596376 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:36:15.454401 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:15.454357 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.26:8443: connect: connection refused" Sep 17 07:36:15.454856 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:15.454516 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:36:15.858417 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:15.858340 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="online" probeResult="failure" output="Get \"https://10.133.0.26:6567/health\": dial tcp 10.133.0.26:6567: connect: connection refused" Sep 17 07:36:15.858552 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:15.858436 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:36:19.261144 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:19.261107 2553 generic.go:356] "Generic (PLEG): container finished" podID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerID="3eaa53f338ef1b8fc76eefa496b4f156fcb9521609542f62b6991c9fd503640c" exitCode=137 Sep 17 07:36:19.261144 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:19.261138 2553 generic.go:356] "Generic (PLEG): container finished" podID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerID="34964b8cdeb1cb29f483bf836bf1bd769d3df13266735d52fbb90be9cf0a511c" exitCode=137 Sep 17 07:36:19.261563 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:19.261188 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" event={"ID":"c368c010-42a2-4eb3-9183-4d76df8346f3","Type":"ContainerDied","Data":"3eaa53f338ef1b8fc76eefa496b4f156fcb9521609542f62b6991c9fd503640c"} Sep 17 07:36:19.261563 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:19.261247 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" event={"ID":"c368c010-42a2-4eb3-9183-4d76df8346f3","Type":"ContainerDied","Data":"34964b8cdeb1cb29f483bf836bf1bd769d3df13266735d52fbb90be9cf0a511c"} Sep 17 07:36:19.504612 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:19.504589 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:36:19.638227 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:19.638168 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49dfl\" (UniqueName: \"kubernetes.io/projected/c368c010-42a2-4eb3-9183-4d76df8346f3-kube-api-access-49dfl\") pod \"c368c010-42a2-4eb3-9183-4d76df8346f3\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " Sep 17 07:36:19.638227 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:19.638201 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-registry-tls\") pod \"c368c010-42a2-4eb3-9183-4d76df8346f3\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " Sep 17 07:36:19.638393 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:19.638249 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-offline-tls\") pod \"c368c010-42a2-4eb3-9183-4d76df8346f3\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " Sep 17 07:36:19.638393 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:19.638279 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c368c010-42a2-4eb3-9183-4d76df8346f3-feast-data\") pod \"c368c010-42a2-4eb3-9183-4d76df8346f3\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " Sep 17 07:36:19.638494 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:19.638465 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-ui-tls\") pod \"c368c010-42a2-4eb3-9183-4d76df8346f3\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " Sep 17 07:36:19.638548 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:19.638528 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-online-tls\") pod \"c368c010-42a2-4eb3-9183-4d76df8346f3\" (UID: \"c368c010-42a2-4eb3-9183-4d76df8346f3\") " Sep 17 07:36:19.638826 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:19.638800 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c368c010-42a2-4eb3-9183-4d76df8346f3-feast-data" (OuterVolumeSpecName: "feast-data") pod "c368c010-42a2-4eb3-9183-4d76df8346f3" (UID: "c368c010-42a2-4eb3-9183-4d76df8346f3"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:36:19.640740 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:19.640703 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-online-tls" (OuterVolumeSpecName: "online-tls") pod "c368c010-42a2-4eb3-9183-4d76df8346f3" (UID: "c368c010-42a2-4eb3-9183-4d76df8346f3"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:36:19.640920 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:19.640899 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c368c010-42a2-4eb3-9183-4d76df8346f3-kube-api-access-49dfl" (OuterVolumeSpecName: "kube-api-access-49dfl") pod "c368c010-42a2-4eb3-9183-4d76df8346f3" (UID: "c368c010-42a2-4eb3-9183-4d76df8346f3"). InnerVolumeSpecName "kube-api-access-49dfl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:36:19.640986 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:19.640923 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "c368c010-42a2-4eb3-9183-4d76df8346f3" (UID: "c368c010-42a2-4eb3-9183-4d76df8346f3"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:36:19.640986 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:19.640934 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c368c010-42a2-4eb3-9183-4d76df8346f3" (UID: "c368c010-42a2-4eb3-9183-4d76df8346f3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:36:19.640986 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:19.640958 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "c368c010-42a2-4eb3-9183-4d76df8346f3" (UID: "c368c010-42a2-4eb3-9183-4d76df8346f3"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:36:19.739988 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:19.739958 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49dfl\" (UniqueName: \"kubernetes.io/projected/c368c010-42a2-4eb3-9183-4d76df8346f3-kube-api-access-49dfl\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:36:19.739988 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:19.739987 2553 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-registry-tls\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:36:19.739988 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:19.739995 2553 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-offline-tls\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:36:19.740134 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:19.740003 2553 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c368c010-42a2-4eb3-9183-4d76df8346f3-feast-data\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:36:19.740134 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:19.740011 2553 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-ui-tls\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:36:19.740134 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:19.740018 2553 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c368c010-42a2-4eb3-9183-4d76df8346f3-online-tls\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:36:20.268894 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:20.268853 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" event={"ID":"c368c010-42a2-4eb3-9183-4d76df8346f3","Type":"ContainerDied","Data":"a7a892368da23a618c217dc0104921f39ca5bdb00a0392e28f1b8b5b1b67259a"} Sep 17 07:36:20.269230 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:20.268922 2553 scope.go:117] "RemoveContainer" containerID="9b98ded8bd5a250f3e1258d96dbe9946a1f930010bb153269a93c8151464de5b" Sep 17 07:36:20.269230 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:20.268922 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b" Sep 17 07:36:20.279095 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:20.279068 2553 scope.go:117] "RemoveContainer" containerID="3eaa53f338ef1b8fc76eefa496b4f156fcb9521609542f62b6991c9fd503640c" Sep 17 07:36:20.291765 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:20.291746 2553 scope.go:117] "RemoveContainer" containerID="85a7a5694f7e42eadafe074a6a28ab941549da190f54e18aabfa7498b49df671" Sep 17 07:36:20.298823 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:20.298791 2553 scope.go:117] "RemoveContainer" containerID="34964b8cdeb1cb29f483bf836bf1bd769d3df13266735d52fbb90be9cf0a511c" Sep 17 07:36:20.302247 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:20.302228 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b"] Sep 17 07:36:20.306699 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:20.306678 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-66c88fcfbf-5bt2b"] Sep 17 07:36:20.307785 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:20.307760 2553 scope.go:117] "RemoveContainer" containerID="e09e806a46cba4eebb7b90b2a3dba7339177aa40b82de762bf1389c4fb1b6331" Sep 17 07:36:20.314484 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:20.314465 2553 scope.go:117] "RemoveContainer" containerID="f3fab9b469eae1e8213a578dbe11e8e971a1d678eb3ee0c1d9bcdd88d2dffb5a" Sep 17 07:36:20.933891 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:36:20.933846 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" path="/var/lib/kubelet/pods/c368c010-42a2-4eb3-9183-4d76df8346f3/volumes" Sep 17 07:37:13.844964 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:13.844930 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:37:14.095045 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.094986 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:37:14.336161 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.336120 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:37:14.426841 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.426754 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:37:14.525057 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.525020 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:37:14.558214 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.558184 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj"] Sep 17 07:37:14.558480 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.558467 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="online" Sep 17 07:37:14.558539 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.558481 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="online" Sep 17 07:37:14.558539 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.558490 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="registry" Sep 17 07:37:14.558539 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.558496 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="registry" Sep 17 07:37:14.558539 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.558506 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="ui" Sep 17 07:37:14.558539 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.558511 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="ui" Sep 17 07:37:14.558539 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.558518 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="feast-apply" Sep 17 07:37:14.558539 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.558523 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="feast-apply" Sep 17 07:37:14.558539 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.558537 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="feast-init" Sep 17 07:37:14.558539 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.558542 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="feast-init" Sep 17 07:37:14.558976 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.558547 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="offline" Sep 17 07:37:14.558976 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.558552 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="offline" Sep 17 07:37:14.558976 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.558599 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="online" Sep 17 07:37:14.558976 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.558607 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="registry" Sep 17 07:37:14.558976 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.558614 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="offline" Sep 17 07:37:14.558976 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.558620 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="c368c010-42a2-4eb3-9183-4d76df8346f3" containerName="ui" Sep 17 07:37:14.561843 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.561827 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:14.565217 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.565191 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 17 07:37:14.565370 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.565354 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 17 07:37:14.565483 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.565468 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 17 07:37:14.565555 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.565493 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 17 07:37:14.565555 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.565504 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 17 07:37:14.565683 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.565562 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-ggvj9\"" Sep 17 07:37:14.565683 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.565649 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 17 07:37:14.566255 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.566239 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Sep 17 07:37:14.577183 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.577158 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj"] Sep 17 07:37:14.613076 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.613052 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9550c70e-cb68-40d4-b083-acdda258c483-registry-tls\") pod \"feast-simple-feast-remote-setup-86c6586dd4-2tlxj\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:14.613174 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.613090 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbdfv\" (UniqueName: \"kubernetes.io/projected/9550c70e-cb68-40d4-b083-acdda258c483-kube-api-access-fbdfv\") pod \"feast-simple-feast-remote-setup-86c6586dd4-2tlxj\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:14.613174 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.613131 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9550c70e-cb68-40d4-b083-acdda258c483-feast-data\") pod \"feast-simple-feast-remote-setup-86c6586dd4-2tlxj\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:14.613271 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.613184 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9550c70e-cb68-40d4-b083-acdda258c483-offline-tls\") pod \"feast-simple-feast-remote-setup-86c6586dd4-2tlxj\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:14.613271 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.613240 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9550c70e-cb68-40d4-b083-acdda258c483-online-tls\") pod \"feast-simple-feast-remote-setup-86c6586dd4-2tlxj\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:14.613344 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.613287 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9550c70e-cb68-40d4-b083-acdda258c483-ui-tls\") pod \"feast-simple-feast-remote-setup-86c6586dd4-2tlxj\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:14.629687 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.629667 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:37:14.714457 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.714436 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9550c70e-cb68-40d4-b083-acdda258c483-registry-tls\") pod \"feast-simple-feast-remote-setup-86c6586dd4-2tlxj\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:14.714570 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.714470 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbdfv\" (UniqueName: \"kubernetes.io/projected/9550c70e-cb68-40d4-b083-acdda258c483-kube-api-access-fbdfv\") pod \"feast-simple-feast-remote-setup-86c6586dd4-2tlxj\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:14.714570 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.714506 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9550c70e-cb68-40d4-b083-acdda258c483-feast-data\") pod \"feast-simple-feast-remote-setup-86c6586dd4-2tlxj\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:14.714684 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.714567 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9550c70e-cb68-40d4-b083-acdda258c483-offline-tls\") pod \"feast-simple-feast-remote-setup-86c6586dd4-2tlxj\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:14.714684 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.714611 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9550c70e-cb68-40d4-b083-acdda258c483-online-tls\") pod \"feast-simple-feast-remote-setup-86c6586dd4-2tlxj\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:14.714684 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.714649 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9550c70e-cb68-40d4-b083-acdda258c483-ui-tls\") pod \"feast-simple-feast-remote-setup-86c6586dd4-2tlxj\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:14.714829 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:37:14.714730 2553 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Sep 17 07:37:14.714829 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:37:14.714757 2553 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 17 07:37:14.714829 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:37:14.714792 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9550c70e-cb68-40d4-b083-acdda258c483-online-tls podName:9550c70e-cb68-40d4-b083-acdda258c483 nodeName:}" failed. No retries permitted until 2026-09-17 07:37:15.214772332 +0000 UTC m=+556.772957117 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/9550c70e-cb68-40d4-b083-acdda258c483-online-tls") pod "feast-simple-feast-remote-setup-86c6586dd4-2tlxj" (UID: "9550c70e-cb68-40d4-b083-acdda258c483") : secret "feast-simple-feast-remote-setup-online-tls" not found Sep 17 07:37:14.714829 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:37:14.714812 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9550c70e-cb68-40d4-b083-acdda258c483-ui-tls podName:9550c70e-cb68-40d4-b083-acdda258c483 nodeName:}" failed. No retries permitted until 2026-09-17 07:37:15.214800095 +0000 UTC m=+556.772984885 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/9550c70e-cb68-40d4-b083-acdda258c483-ui-tls") pod "feast-simple-feast-remote-setup-86c6586dd4-2tlxj" (UID: "9550c70e-cb68-40d4-b083-acdda258c483") : secret "feast-simple-feast-remote-setup-ui-tls" not found Sep 17 07:37:14.715066 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.715048 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9550c70e-cb68-40d4-b083-acdda258c483-feast-data\") pod \"feast-simple-feast-remote-setup-86c6586dd4-2tlxj\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:14.715223 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.715204 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9550c70e-cb68-40d4-b083-acdda258c483-registry-tls\") pod \"feast-simple-feast-remote-setup-86c6586dd4-2tlxj\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:14.717385 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.717355 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9550c70e-cb68-40d4-b083-acdda258c483-offline-tls\") pod \"feast-simple-feast-remote-setup-86c6586dd4-2tlxj\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:14.723420 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:14.723397 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbdfv\" (UniqueName: \"kubernetes.io/projected/9550c70e-cb68-40d4-b083-acdda258c483-kube-api-access-fbdfv\") pod \"feast-simple-feast-remote-setup-86c6586dd4-2tlxj\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:15.117078 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:15.117006 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:37:15.174439 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:15.174394 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:37:15.224768 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:15.224736 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9550c70e-cb68-40d4-b083-acdda258c483-online-tls\") pod \"feast-simple-feast-remote-setup-86c6586dd4-2tlxj\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:15.224858 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:15.224776 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9550c70e-cb68-40d4-b083-acdda258c483-ui-tls\") pod \"feast-simple-feast-remote-setup-86c6586dd4-2tlxj\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:15.226993 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:15.226976 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9550c70e-cb68-40d4-b083-acdda258c483-online-tls\") pod \"feast-simple-feast-remote-setup-86c6586dd4-2tlxj\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:15.227076 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:15.227060 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9550c70e-cb68-40d4-b083-acdda258c483-ui-tls\") pod \"feast-simple-feast-remote-setup-86c6586dd4-2tlxj\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:15.472887 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:15.472861 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:15.748091 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:15.748063 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj"] Sep 17 07:37:15.749528 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:37:15.749501 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9550c70e_cb68_40d4_b083_acdda258c483.slice/crio-e5a398ab068c07360704c1152f7fd5a305541c256ebc2b20877885e707886661 WatchSource:0}: Error finding container e5a398ab068c07360704c1152f7fd5a305541c256ebc2b20877885e707886661: Status 404 returned error can't find the container with id e5a398ab068c07360704c1152f7fd5a305541c256ebc2b20877885e707886661 Sep 17 07:37:16.483464 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:16.483430 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" event={"ID":"9550c70e-cb68-40d4-b083-acdda258c483","Type":"ContainerStarted","Data":"8fc551b189c290fd22c9f3eea9ac3f354e6c07f7c995faf02e7d6ebbdd9f21b9"} Sep 17 07:37:16.483940 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:16.483477 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" event={"ID":"9550c70e-cb68-40d4-b083-acdda258c483","Type":"ContainerStarted","Data":"e5a398ab068c07360704c1152f7fd5a305541c256ebc2b20877885e707886661"} Sep 17 07:37:20.502939 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:20.502912 2553 generic.go:356] "Generic (PLEG): container finished" podID="9550c70e-cb68-40d4-b083-acdda258c483" containerID="8fc551b189c290fd22c9f3eea9ac3f354e6c07f7c995faf02e7d6ebbdd9f21b9" exitCode=0 Sep 17 07:37:20.503253 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:20.502949 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" event={"ID":"9550c70e-cb68-40d4-b083-acdda258c483","Type":"ContainerDied","Data":"8fc551b189c290fd22c9f3eea9ac3f354e6c07f7c995faf02e7d6ebbdd9f21b9"} Sep 17 07:37:21.509456 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:21.509421 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" event={"ID":"9550c70e-cb68-40d4-b083-acdda258c483","Type":"ContainerStarted","Data":"81f06691c7eb787ccd5f712182b58cf24a3e80f7181f2dda0d2b08eb54824c50"} Sep 17 07:37:45.531727 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:45.531690 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:37:45.604058 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:45.604032 2553 generic.go:356] "Generic (PLEG): container finished" podID="9550c70e-cb68-40d4-b083-acdda258c483" containerID="81f06691c7eb787ccd5f712182b58cf24a3e80f7181f2dda0d2b08eb54824c50" exitCode=0 Sep 17 07:37:45.604171 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:45.604105 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" event={"ID":"9550c70e-cb68-40d4-b083-acdda258c483","Type":"ContainerDied","Data":"81f06691c7eb787ccd5f712182b58cf24a3e80f7181f2dda0d2b08eb54824c50"} Sep 17 07:37:46.612486 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:46.612448 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" event={"ID":"9550c70e-cb68-40d4-b083-acdda258c483","Type":"ContainerStarted","Data":"e15c2e143bba0d9fcbf5d10a922ae215ae24f06a0dcb9f6e001da01d6125c86c"} Sep 17 07:37:46.612847 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:46.612509 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" event={"ID":"9550c70e-cb68-40d4-b083-acdda258c483","Type":"ContainerStarted","Data":"7a8f96550bf4dbaa26a178a5137ad429e96b9427470a58cbf2c403507886336e"} Sep 17 07:37:46.612847 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:46.612530 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" event={"ID":"9550c70e-cb68-40d4-b083-acdda258c483","Type":"ContainerStarted","Data":"73e235699e4130875c9b2aff8963fc4573f7411d65ff88c0151936574d2a1754"} Sep 17 07:37:48.473223 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:48.473183 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:48.473688 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:48.473243 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:48.473688 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:48.473262 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:48.475113 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:48.475077 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.29:8816: connect: connection refused" Sep 17 07:37:48.475252 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:48.475103 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused" Sep 17 07:37:48.475252 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:48.475153 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Sep 17 07:37:51.473340 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:51.473294 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.29:8816: connect: connection refused" Sep 17 07:37:51.473340 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:51.473317 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused" Sep 17 07:37:51.474006 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:51.473323 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Sep 17 07:37:54.474202 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:54.474171 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:54.474560 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:54.474349 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:54.474718 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:54.474693 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:54.474860 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:54.474731 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:54.475031 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:54.475010 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:54.475282 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:54.475267 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:54.495933 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:54.495913 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:54.498261 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:54.498217 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" podStartSLOduration=40.498203019 podStartE2EDuration="40.498203019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:37:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:37:46.644017916 +0000 UTC m=+588.202202749" watchObservedRunningTime="2026-09-17 07:37:54.498203019 +0000 UTC m=+596.056387830" Sep 17 07:37:54.651585 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:54.651561 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:54.669318 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:54.669298 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:37:54.963687 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:54.963658 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:37:58.844747 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:58.844723 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-5kks2_2455e65f-98bb-463d-a204-5dbd5d69b5f3/console-operator/1.log" Sep 17 07:37:58.845643 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:58.845625 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-5kks2_2455e65f-98bb-463d-a204-5dbd5d69b5f3/console-operator/1.log" Sep 17 07:37:58.858074 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:58.858052 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/ovn-acl-logging/0.log" Sep 17 07:37:58.858608 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:37:58.858589 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/ovn-acl-logging/0.log" Sep 17 07:38:30.552208 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:38:30.552126 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:38:56.761001 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:38:56.760944 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:38:56.802814 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:38:56.802783 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj"] Sep 17 07:38:56.803065 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:38:56.803038 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="online" containerID="cri-o://73e235699e4130875c9b2aff8963fc4573f7411d65ff88c0151936574d2a1754" gracePeriod=30 Sep 17 07:38:56.803199 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:38:56.803070 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="ui" containerID="cri-o://e15c2e143bba0d9fcbf5d10a922ae215ae24f06a0dcb9f6e001da01d6125c86c" gracePeriod=30 Sep 17 07:38:56.803199 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:38:56.803081 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="offline" containerID="cri-o://7a8f96550bf4dbaa26a178a5137ad429e96b9427470a58cbf2c403507886336e" gracePeriod=30 Sep 17 07:38:56.870722 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:38:56.870695 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:38:56.951777 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:38:56.951745 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx"] Sep 17 07:38:56.952021 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:38:56.951991 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="registry" containerID="cri-o://2bc77b75e8b16f5e91e971db01072a7d9858447934329a57648da6c08f68585f" gracePeriod=30 Sep 17 07:38:56.952202 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:38:56.952022 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="offline" containerID="cri-o://7515a8f100456f68ef218fdcd879416200968a3df0308a8078957949e0c449f2" gracePeriod=30 Sep 17 07:38:56.952282 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:38:56.952060 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="ui" containerID="cri-o://2f957293428a06d806f4e15fe419cca0d0a187cffca02daa4c9cca0affdd75ec" gracePeriod=30 Sep 17 07:38:56.952343 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:38:56.952064 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="online" containerID="cri-o://a3474f088ea79d32a6f70512c9af4de1144bfd155b19c678c1208d071b27f82a" gracePeriod=30 Sep 17 07:38:56.956993 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:38:56.956968 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:38:57.020041 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:38:57.019968 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:38:57.921601 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:38:57.921530 2553 generic.go:356] "Generic (PLEG): container finished" podID="9550c70e-cb68-40d4-b083-acdda258c483" containerID="e15c2e143bba0d9fcbf5d10a922ae215ae24f06a0dcb9f6e001da01d6125c86c" exitCode=0 Sep 17 07:38:57.921601 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:38:57.921558 2553 generic.go:356] "Generic (PLEG): container finished" podID="9550c70e-cb68-40d4-b083-acdda258c483" containerID="73e235699e4130875c9b2aff8963fc4573f7411d65ff88c0151936574d2a1754" exitCode=0 Sep 17 07:38:57.921943 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:38:57.921601 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" event={"ID":"9550c70e-cb68-40d4-b083-acdda258c483","Type":"ContainerDied","Data":"e15c2e143bba0d9fcbf5d10a922ae215ae24f06a0dcb9f6e001da01d6125c86c"} Sep 17 07:38:57.921943 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:38:57.921645 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" event={"ID":"9550c70e-cb68-40d4-b083-acdda258c483","Type":"ContainerDied","Data":"73e235699e4130875c9b2aff8963fc4573f7411d65ff88c0151936574d2a1754"} Sep 17 07:38:57.924430 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:38:57.924412 2553 generic.go:356] "Generic (PLEG): container finished" podID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerID="2f957293428a06d806f4e15fe419cca0d0a187cffca02daa4c9cca0affdd75ec" exitCode=0 Sep 17 07:38:57.924430 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:38:57.924429 2553 generic.go:356] "Generic (PLEG): container finished" podID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerID="a3474f088ea79d32a6f70512c9af4de1144bfd155b19c678c1208d071b27f82a" exitCode=0 Sep 17 07:38:57.924530 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:38:57.924479 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" event={"ID":"3cf15699-c783-49c2-96b6-af9aa5939d29","Type":"ContainerDied","Data":"2f957293428a06d806f4e15fe419cca0d0a187cffca02daa4c9cca0affdd75ec"} Sep 17 07:38:57.924530 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:38:57.924511 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" event={"ID":"3cf15699-c783-49c2-96b6-af9aa5939d29","Type":"ContainerDied","Data":"a3474f088ea79d32a6f70512c9af4de1144bfd155b19c678c1208d071b27f82a"} Sep 17 07:39:01.269621 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:01.269579 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.28:8443: connect: connection refused" Sep 17 07:39:02.038814 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:02.038780 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:39:02.227210 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:02.227170 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused" Sep 17 07:39:04.474895 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:04.474850 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused" Sep 17 07:39:04.652398 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:04.652360 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Sep 17 07:39:11.269591 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:11.269542 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.28:8443: connect: connection refused" Sep 17 07:39:12.227201 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:12.227158 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused" Sep 17 07:39:13.537711 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:13.537682 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:39:13.621713 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:13.621681 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:39:14.474940 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:14.474903 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused" Sep 17 07:39:14.652013 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:14.651981 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Sep 17 07:39:21.269294 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:21.269257 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.28:8443: connect: connection refused" Sep 17 07:39:21.269730 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:21.269401 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:39:22.226992 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:22.226951 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused" Sep 17 07:39:22.227177 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:22.227072 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:39:24.475221 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:24.475183 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused" Sep 17 07:39:24.475653 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:24.475290 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:39:24.651952 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:24.651921 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Sep 17 07:39:24.652085 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:24.652002 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:39:27.044846 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.044811 2553 generic.go:356] "Generic (PLEG): container finished" podID="9550c70e-cb68-40d4-b083-acdda258c483" containerID="7a8f96550bf4dbaa26a178a5137ad429e96b9427470a58cbf2c403507886336e" exitCode=137 Sep 17 07:39:27.045162 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.044891 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" event={"ID":"9550c70e-cb68-40d4-b083-acdda258c483","Type":"ContainerDied","Data":"7a8f96550bf4dbaa26a178a5137ad429e96b9427470a58cbf2c403507886336e"} Sep 17 07:39:27.047781 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.047761 2553 generic.go:356] "Generic (PLEG): container finished" podID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerID="7515a8f100456f68ef218fdcd879416200968a3df0308a8078957949e0c449f2" exitCode=137 Sep 17 07:39:27.047781 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.047781 2553 generic.go:356] "Generic (PLEG): container finished" podID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerID="2bc77b75e8b16f5e91e971db01072a7d9858447934329a57648da6c08f68585f" exitCode=137 Sep 17 07:39:27.047932 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.047826 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" event={"ID":"3cf15699-c783-49c2-96b6-af9aa5939d29","Type":"ContainerDied","Data":"7515a8f100456f68ef218fdcd879416200968a3df0308a8078957949e0c449f2"} Sep 17 07:39:27.047932 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.047857 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" event={"ID":"3cf15699-c783-49c2-96b6-af9aa5939d29","Type":"ContainerDied","Data":"2bc77b75e8b16f5e91e971db01072a7d9858447934329a57648da6c08f68585f"} Sep 17 07:39:27.449817 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.449795 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:39:27.589378 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.589346 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbdfv\" (UniqueName: \"kubernetes.io/projected/9550c70e-cb68-40d4-b083-acdda258c483-kube-api-access-fbdfv\") pod \"9550c70e-cb68-40d4-b083-acdda258c483\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " Sep 17 07:39:27.589536 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.589418 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9550c70e-cb68-40d4-b083-acdda258c483-registry-tls\") pod \"9550c70e-cb68-40d4-b083-acdda258c483\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " Sep 17 07:39:27.589536 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.589468 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9550c70e-cb68-40d4-b083-acdda258c483-feast-data\") pod \"9550c70e-cb68-40d4-b083-acdda258c483\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " Sep 17 07:39:27.589536 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.589498 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9550c70e-cb68-40d4-b083-acdda258c483-online-tls\") pod \"9550c70e-cb68-40d4-b083-acdda258c483\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " Sep 17 07:39:27.589699 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.589565 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9550c70e-cb68-40d4-b083-acdda258c483-offline-tls\") pod \"9550c70e-cb68-40d4-b083-acdda258c483\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " Sep 17 07:39:27.589699 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.589591 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9550c70e-cb68-40d4-b083-acdda258c483-ui-tls\") pod \"9550c70e-cb68-40d4-b083-acdda258c483\" (UID: \"9550c70e-cb68-40d4-b083-acdda258c483\") " Sep 17 07:39:27.589803 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.589777 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9550c70e-cb68-40d4-b083-acdda258c483-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9550c70e-cb68-40d4-b083-acdda258c483" (UID: "9550c70e-cb68-40d4-b083-acdda258c483"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:39:27.590021 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.589981 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9550c70e-cb68-40d4-b083-acdda258c483-feast-data" (OuterVolumeSpecName: "feast-data") pod "9550c70e-cb68-40d4-b083-acdda258c483" (UID: "9550c70e-cb68-40d4-b083-acdda258c483"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:39:27.591639 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.591613 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9550c70e-cb68-40d4-b083-acdda258c483-kube-api-access-fbdfv" (OuterVolumeSpecName: "kube-api-access-fbdfv") pod "9550c70e-cb68-40d4-b083-acdda258c483" (UID: "9550c70e-cb68-40d4-b083-acdda258c483"). InnerVolumeSpecName "kube-api-access-fbdfv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:39:27.591751 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.591704 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9550c70e-cb68-40d4-b083-acdda258c483-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "9550c70e-cb68-40d4-b083-acdda258c483" (UID: "9550c70e-cb68-40d4-b083-acdda258c483"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:39:27.591751 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.591725 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9550c70e-cb68-40d4-b083-acdda258c483-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "9550c70e-cb68-40d4-b083-acdda258c483" (UID: "9550c70e-cb68-40d4-b083-acdda258c483"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:39:27.591847 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.591759 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9550c70e-cb68-40d4-b083-acdda258c483-online-tls" (OuterVolumeSpecName: "online-tls") pod "9550c70e-cb68-40d4-b083-acdda258c483" (UID: "9550c70e-cb68-40d4-b083-acdda258c483"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:39:27.598033 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.597999 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:39:27.690148 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.690117 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3cf15699-c783-49c2-96b6-af9aa5939d29-feast-data\") pod \"3cf15699-c783-49c2-96b6-af9aa5939d29\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " Sep 17 07:39:27.690276 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.690161 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmr94\" (UniqueName: \"kubernetes.io/projected/3cf15699-c783-49c2-96b6-af9aa5939d29-kube-api-access-lmr94\") pod \"3cf15699-c783-49c2-96b6-af9aa5939d29\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " Sep 17 07:39:27.690276 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.690181 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-online-tls\") pod \"3cf15699-c783-49c2-96b6-af9aa5939d29\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " Sep 17 07:39:27.690276 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.690198 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-registry-tls\") pod \"3cf15699-c783-49c2-96b6-af9aa5939d29\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " Sep 17 07:39:27.690276 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.690244 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-ui-tls\") pod \"3cf15699-c783-49c2-96b6-af9aa5939d29\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " Sep 17 07:39:27.690276 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.690265 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-offline-tls\") pod \"3cf15699-c783-49c2-96b6-af9aa5939d29\" (UID: \"3cf15699-c783-49c2-96b6-af9aa5939d29\") " Sep 17 07:39:27.690548 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.690466 2553 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/9550c70e-cb68-40d4-b083-acdda258c483-registry-tls\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:39:27.690548 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.690485 2553 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9550c70e-cb68-40d4-b083-acdda258c483-feast-data\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:39:27.690548 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.690521 2553 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9550c70e-cb68-40d4-b083-acdda258c483-online-tls\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:39:27.690548 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.690534 2553 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9550c70e-cb68-40d4-b083-acdda258c483-offline-tls\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:39:27.690548 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.690546 2553 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9550c70e-cb68-40d4-b083-acdda258c483-ui-tls\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:39:27.690831 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.690557 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbdfv\" (UniqueName: \"kubernetes.io/projected/9550c70e-cb68-40d4-b083-acdda258c483-kube-api-access-fbdfv\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:39:27.690831 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.690655 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cf15699-c783-49c2-96b6-af9aa5939d29-feast-data" (OuterVolumeSpecName: "feast-data") pod "3cf15699-c783-49c2-96b6-af9aa5939d29" (UID: "3cf15699-c783-49c2-96b6-af9aa5939d29"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:39:27.692091 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.692063 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cf15699-c783-49c2-96b6-af9aa5939d29-kube-api-access-lmr94" (OuterVolumeSpecName: "kube-api-access-lmr94") pod "3cf15699-c783-49c2-96b6-af9aa5939d29" (UID: "3cf15699-c783-49c2-96b6-af9aa5939d29"). InnerVolumeSpecName "kube-api-access-lmr94". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:39:27.692188 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.692095 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-online-tls" (OuterVolumeSpecName: "online-tls") pod "3cf15699-c783-49c2-96b6-af9aa5939d29" (UID: "3cf15699-c783-49c2-96b6-af9aa5939d29"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:39:27.692263 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.692246 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "3cf15699-c783-49c2-96b6-af9aa5939d29" (UID: "3cf15699-c783-49c2-96b6-af9aa5939d29"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:39:27.692566 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.692549 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "3cf15699-c783-49c2-96b6-af9aa5939d29" (UID: "3cf15699-c783-49c2-96b6-af9aa5939d29"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:39:27.692636 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.692621 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "3cf15699-c783-49c2-96b6-af9aa5939d29" (UID: "3cf15699-c783-49c2-96b6-af9aa5939d29"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:39:27.791714 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.791696 2553 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/3cf15699-c783-49c2-96b6-af9aa5939d29-feast-data\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:39:27.791714 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.791713 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmr94\" (UniqueName: \"kubernetes.io/projected/3cf15699-c783-49c2-96b6-af9aa5939d29-kube-api-access-lmr94\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:39:27.791824 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.791722 2553 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-online-tls\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:39:27.791824 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.791729 2553 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-registry-tls\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:39:27.791824 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.791737 2553 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-ui-tls\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:39:27.791824 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:27.791743 2553 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/3cf15699-c783-49c2-96b6-af9aa5939d29-offline-tls\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:39:28.053865 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:28.053837 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" event={"ID":"9550c70e-cb68-40d4-b083-acdda258c483","Type":"ContainerDied","Data":"e5a398ab068c07360704c1152f7fd5a305541c256ebc2b20877885e707886661"} Sep 17 07:39:28.054225 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:28.053896 2553 scope.go:117] "RemoveContainer" containerID="e15c2e143bba0d9fcbf5d10a922ae215ae24f06a0dcb9f6e001da01d6125c86c" Sep 17 07:39:28.054225 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:28.053848 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj" Sep 17 07:39:28.056995 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:28.056976 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" Sep 17 07:39:28.057081 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:28.057002 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx" event={"ID":"3cf15699-c783-49c2-96b6-af9aa5939d29","Type":"ContainerDied","Data":"beb16b6ce91d673e21865694fab84303fe352ae3f17f02a5539b7024738445a1"} Sep 17 07:39:28.062490 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:28.062468 2553 scope.go:117] "RemoveContainer" containerID="7a8f96550bf4dbaa26a178a5137ad429e96b9427470a58cbf2c403507886336e" Sep 17 07:39:28.069675 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:28.069659 2553 scope.go:117] "RemoveContainer" containerID="73e235699e4130875c9b2aff8963fc4573f7411d65ff88c0151936574d2a1754" Sep 17 07:39:28.076717 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:28.076702 2553 scope.go:117] "RemoveContainer" containerID="81f06691c7eb787ccd5f712182b58cf24a3e80f7181f2dda0d2b08eb54824c50" Sep 17 07:39:28.083155 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:28.083139 2553 scope.go:117] "RemoveContainer" containerID="8fc551b189c290fd22c9f3eea9ac3f354e6c07f7c995faf02e7d6ebbdd9f21b9" Sep 17 07:39:28.088715 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:28.088694 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx"] Sep 17 07:39:28.090186 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:28.090155 2553 scope.go:117] "RemoveContainer" containerID="2f957293428a06d806f4e15fe419cca0d0a187cffca02daa4c9cca0affdd75ec" Sep 17 07:39:28.096475 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:28.096457 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-66c88fcfbf-pc9jx"] Sep 17 07:39:28.097054 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:28.097038 2553 scope.go:117] "RemoveContainer" containerID="7515a8f100456f68ef218fdcd879416200968a3df0308a8078957949e0c449f2" Sep 17 07:39:28.103997 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:28.103982 2553 scope.go:117] "RemoveContainer" containerID="a3474f088ea79d32a6f70512c9af4de1144bfd155b19c678c1208d071b27f82a" Sep 17 07:39:28.110582 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:28.110564 2553 scope.go:117] "RemoveContainer" containerID="2bc77b75e8b16f5e91e971db01072a7d9858447934329a57648da6c08f68585f" Sep 17 07:39:28.116170 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:28.116148 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj"] Sep 17 07:39:28.118135 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:28.118120 2553 scope.go:117] "RemoveContainer" containerID="cdd493e948111fe218f9d2e5e53ba97eb3e07dfb4613c2ddce9529593d45922e" Sep 17 07:39:28.121448 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:28.121429 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-86c6586dd4-2tlxj"] Sep 17 07:39:28.125136 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:28.125118 2553 scope.go:117] "RemoveContainer" containerID="d838c1af100afb86401ec7eb77bca37152e9afca63c04dfe00a13af5c3a7a5d5" Sep 17 07:39:28.934550 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:28.934520 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" path="/var/lib/kubelet/pods/3cf15699-c783-49c2-96b6-af9aa5939d29/volumes" Sep 17 07:39:28.935100 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:28.935085 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9550c70e-cb68-40d4-b083-acdda258c483" path="/var/lib/kubelet/pods/9550c70e-cb68-40d4-b083-acdda258c483/volumes" Sep 17 07:39:41.041144 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.041109 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:39:41.312960 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.312873 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-s5lrs"] Sep 17 07:39:41.313262 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313245 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="online" Sep 17 07:39:41.313350 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313265 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="online" Sep 17 07:39:41.313350 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313277 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="offline" Sep 17 07:39:41.313350 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313285 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="offline" Sep 17 07:39:41.313350 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313296 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="ui" Sep 17 07:39:41.313350 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313305 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="ui" Sep 17 07:39:41.313350 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313341 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="feast-init" Sep 17 07:39:41.313350 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313350 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="feast-init" Sep 17 07:39:41.313703 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313367 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="offline" Sep 17 07:39:41.313703 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313377 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="offline" Sep 17 07:39:41.313703 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313393 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="registry" Sep 17 07:39:41.313703 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313401 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="registry" Sep 17 07:39:41.313703 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313411 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="ui" Sep 17 07:39:41.313703 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313418 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="ui" Sep 17 07:39:41.313703 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313430 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="feast-init" Sep 17 07:39:41.313703 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313437 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="feast-init" Sep 17 07:39:41.313703 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313447 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="online" Sep 17 07:39:41.313703 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313455 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="online" Sep 17 07:39:41.313703 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313468 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="feast-apply" Sep 17 07:39:41.313703 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313476 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="feast-apply" Sep 17 07:39:41.313703 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313490 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="feast-apply" Sep 17 07:39:41.313703 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313498 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="feast-apply" Sep 17 07:39:41.313703 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313563 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="online" Sep 17 07:39:41.313703 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313576 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="online" Sep 17 07:39:41.313703 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313587 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="offline" Sep 17 07:39:41.313703 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313597 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="registry" Sep 17 07:39:41.313703 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313606 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="offline" Sep 17 07:39:41.313703 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313616 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cf15699-c783-49c2-96b6-af9aa5939d29" containerName="ui" Sep 17 07:39:41.313703 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.313625 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="9550c70e-cb68-40d4-b083-acdda258c483" containerName="ui" Sep 17 07:39:41.318652 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.318634 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-s5lrs" Sep 17 07:39:41.331557 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.331541 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Sep 17 07:39:41.332583 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.332567 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-z4cfj\"" Sep 17 07:39:41.332654 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.332571 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 17 07:39:41.332654 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.332620 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 17 07:39:41.338307 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.338290 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-s5lrs"] Sep 17 07:39:41.397073 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.397052 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:39:41.424282 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.424259 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-69bbd798b6-ghkcf"] Sep 17 07:39:41.427254 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.427237 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-ghkcf" Sep 17 07:39:41.453677 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.453656 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-ghkcf"] Sep 17 07:39:41.485866 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.485841 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8-postgresdata\") pod \"postgres-864846d4f5-s5lrs\" (UID: \"86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8\") " pod="test-ns-feast/postgres-864846d4f5-s5lrs" Sep 17 07:39:41.485969 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.485952 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bm5v\" (UniqueName: \"kubernetes.io/projected/86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8-kube-api-access-8bm5v\") pod \"postgres-864846d4f5-s5lrs\" (UID: \"86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8\") " pod="test-ns-feast/postgres-864846d4f5-s5lrs" Sep 17 07:39:41.488917 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.488857 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:39:41.586808 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.586761 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bm5v\" (UniqueName: \"kubernetes.io/projected/86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8-kube-api-access-8bm5v\") pod \"postgres-864846d4f5-s5lrs\" (UID: \"86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8\") " pod="test-ns-feast/postgres-864846d4f5-s5lrs" Sep 17 07:39:41.586808 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.586799 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmdnz\" (UniqueName: \"kubernetes.io/projected/793abee0-3db5-4854-a589-989feaa3be59-kube-api-access-zmdnz\") pod \"redis-69bbd798b6-ghkcf\" (UID: \"793abee0-3db5-4854-a589-989feaa3be59\") " pod="test-ns-feast/redis-69bbd798b6-ghkcf" Sep 17 07:39:41.586949 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.586833 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8-postgresdata\") pod \"postgres-864846d4f5-s5lrs\" (UID: \"86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8\") " pod="test-ns-feast/postgres-864846d4f5-s5lrs" Sep 17 07:39:41.587172 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.587157 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8-postgresdata\") pod \"postgres-864846d4f5-s5lrs\" (UID: \"86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8\") " pod="test-ns-feast/postgres-864846d4f5-s5lrs" Sep 17 07:39:41.600283 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.600262 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bm5v\" (UniqueName: \"kubernetes.io/projected/86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8-kube-api-access-8bm5v\") pod \"postgres-864846d4f5-s5lrs\" (UID: \"86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8\") " pod="test-ns-feast/postgres-864846d4f5-s5lrs" Sep 17 07:39:41.638100 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.638077 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-s5lrs" Sep 17 07:39:41.687683 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.687284 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmdnz\" (UniqueName: \"kubernetes.io/projected/793abee0-3db5-4854-a589-989feaa3be59-kube-api-access-zmdnz\") pod \"redis-69bbd798b6-ghkcf\" (UID: \"793abee0-3db5-4854-a589-989feaa3be59\") " pod="test-ns-feast/redis-69bbd798b6-ghkcf" Sep 17 07:39:41.713389 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.713366 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmdnz\" (UniqueName: \"kubernetes.io/projected/793abee0-3db5-4854-a589-989feaa3be59-kube-api-access-zmdnz\") pod \"redis-69bbd798b6-ghkcf\" (UID: \"793abee0-3db5-4854-a589-989feaa3be59\") " pod="test-ns-feast/redis-69bbd798b6-ghkcf" Sep 17 07:39:41.735656 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.735631 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-ghkcf" Sep 17 07:39:41.922669 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.922647 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-s5lrs"] Sep 17 07:39:41.925147 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:39:41.925126 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86001f7b_1d5e_4a0f_86f5_ec3fa6bd61d8.slice/crio-f4a9f3b13c6524e3e4d266bac24b3bb9ec13eb326394b5d420b2ccec5469ca27 WatchSource:0}: Error finding container f4a9f3b13c6524e3e4d266bac24b3bb9ec13eb326394b5d420b2ccec5469ca27: Status 404 returned error can't find the container with id f4a9f3b13c6524e3e4d266bac24b3bb9ec13eb326394b5d420b2ccec5469ca27 Sep 17 07:39:41.926968 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:41.926953 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 07:39:42.018811 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:42.018787 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-ghkcf"] Sep 17 07:39:42.021252 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:39:42.021228 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod793abee0_3db5_4854_a589_989feaa3be59.slice/crio-7afc0992b7b916db978eb6d7152be610dae85b19825f410347dc8b672b995383 WatchSource:0}: Error finding container 7afc0992b7b916db978eb6d7152be610dae85b19825f410347dc8b672b995383: Status 404 returned error can't find the container with id 7afc0992b7b916db978eb6d7152be610dae85b19825f410347dc8b672b995383 Sep 17 07:39:42.108755 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:42.108718 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-ghkcf" event={"ID":"793abee0-3db5-4854-a589-989feaa3be59","Type":"ContainerStarted","Data":"7afc0992b7b916db978eb6d7152be610dae85b19825f410347dc8b672b995383"} Sep 17 07:39:42.109850 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:42.109829 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-s5lrs" event={"ID":"86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8","Type":"ContainerStarted","Data":"f4a9f3b13c6524e3e4d266bac24b3bb9ec13eb326394b5d420b2ccec5469ca27"} Sep 17 07:39:44.121365 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:44.121277 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-ghkcf" event={"ID":"793abee0-3db5-4854-a589-989feaa3be59","Type":"ContainerStarted","Data":"3546b83f64a6b0edcf0faf0dc924feb4af9d4d0fbe351c671ea52f0de1d0b1a6"} Sep 17 07:39:47.135796 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:47.135717 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-s5lrs" event={"ID":"86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8","Type":"ContainerStarted","Data":"fe7a63d176e6521d32dfc40c75e4325263137e1847f3261f1e252bdded999d0b"} Sep 17 07:39:47.169739 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:47.169687 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-s5lrs" podStartSLOduration=1.221937876 podStartE2EDuration="6.169667217s" totalImagesPullingTime="4.947729341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:39:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:39:47.169634321 +0000 UTC m=+708.727819129" watchObservedRunningTime="2026-09-17 07:39:47.169667217 +0000 UTC m=+708.727852040" Sep 17 07:39:47.169873 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:47.169783 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/redis-69bbd798b6-ghkcf" podStartSLOduration=4.418985822 podStartE2EDuration="6.169776541s" totalImagesPullingTime="1.750790719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:39:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:39:44.149059737 +0000 UTC m=+705.707244545" watchObservedRunningTime="2026-09-17 07:39:47.169776541 +0000 UTC m=+708.727961350" Sep 17 07:39:50.290297 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:50.290268 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:39:50.504076 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:50.504046 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:39:50.640279 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:50.640217 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:39:50.696581 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:50.696556 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5"] Sep 17 07:39:50.702862 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:50.702840 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:39:50.707111 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:50.707092 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-j22t5\"" Sep 17 07:39:50.707222 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:50.707093 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\"" Sep 17 07:39:50.707222 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:50.707093 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\"" Sep 17 07:39:50.707222 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:50.707149 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 17 07:39:50.708417 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:50.708400 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\"" Sep 17 07:39:50.723530 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:50.719982 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5"] Sep 17 07:39:50.751507 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:50.751485 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f7765c56-67a7-40e3-bc3f-a8906af2b27c-offline-tls\") pod \"feast-credit-scoring-78b5688d9c-prqr5\" (UID: \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\") " pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:39:50.751617 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:50.751515 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f7765c56-67a7-40e3-bc3f-a8906af2b27c-registry-tls\") pod \"feast-credit-scoring-78b5688d9c-prqr5\" (UID: \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\") " pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:39:50.751617 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:50.751531 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqz52\" (UniqueName: \"kubernetes.io/projected/f7765c56-67a7-40e3-bc3f-a8906af2b27c-kube-api-access-gqz52\") pod \"feast-credit-scoring-78b5688d9c-prqr5\" (UID: \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\") " pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:39:50.751617 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:50.751548 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f7765c56-67a7-40e3-bc3f-a8906af2b27c-feast-data\") pod \"feast-credit-scoring-78b5688d9c-prqr5\" (UID: \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\") " pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:39:50.751765 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:50.751654 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f7765c56-67a7-40e3-bc3f-a8906af2b27c-online-tls\") pod \"feast-credit-scoring-78b5688d9c-prqr5\" (UID: \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\") " pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:39:50.852545 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:50.852507 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f7765c56-67a7-40e3-bc3f-a8906af2b27c-online-tls\") pod \"feast-credit-scoring-78b5688d9c-prqr5\" (UID: \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\") " pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:39:50.852663 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:50.852560 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f7765c56-67a7-40e3-bc3f-a8906af2b27c-offline-tls\") pod \"feast-credit-scoring-78b5688d9c-prqr5\" (UID: \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\") " pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:39:50.852663 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:50.852591 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f7765c56-67a7-40e3-bc3f-a8906af2b27c-registry-tls\") pod \"feast-credit-scoring-78b5688d9c-prqr5\" (UID: \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\") " pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:39:50.852663 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:50.852615 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqz52\" (UniqueName: \"kubernetes.io/projected/f7765c56-67a7-40e3-bc3f-a8906af2b27c-kube-api-access-gqz52\") pod \"feast-credit-scoring-78b5688d9c-prqr5\" (UID: \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\") " pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:39:50.852663 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:39:50.852643 2553 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-online-tls: secret "feast-credit-scoring-online-tls" not found Sep 17 07:39:50.852928 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:39:50.852703 2553 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-registry-tls: secret "feast-credit-scoring-registry-tls" not found Sep 17 07:39:50.852928 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:50.852645 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f7765c56-67a7-40e3-bc3f-a8906af2b27c-feast-data\") pod \"feast-credit-scoring-78b5688d9c-prqr5\" (UID: \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\") " pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:39:50.852928 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:39:50.852705 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7765c56-67a7-40e3-bc3f-a8906af2b27c-online-tls podName:f7765c56-67a7-40e3-bc3f-a8906af2b27c nodeName:}" failed. No retries permitted until 2026-09-17 07:39:51.352686921 +0000 UTC m=+712.910871722 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/f7765c56-67a7-40e3-bc3f-a8906af2b27c-online-tls") pod "feast-credit-scoring-78b5688d9c-prqr5" (UID: "f7765c56-67a7-40e3-bc3f-a8906af2b27c") : secret "feast-credit-scoring-online-tls" not found Sep 17 07:39:50.852928 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:39:50.852767 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7765c56-67a7-40e3-bc3f-a8906af2b27c-registry-tls podName:f7765c56-67a7-40e3-bc3f-a8906af2b27c nodeName:}" failed. No retries permitted until 2026-09-17 07:39:51.352756044 +0000 UTC m=+712.910940841 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/f7765c56-67a7-40e3-bc3f-a8906af2b27c-registry-tls") pod "feast-credit-scoring-78b5688d9c-prqr5" (UID: "f7765c56-67a7-40e3-bc3f-a8906af2b27c") : secret "feast-credit-scoring-registry-tls" not found Sep 17 07:39:50.853154 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:50.853136 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f7765c56-67a7-40e3-bc3f-a8906af2b27c-feast-data\") pod \"feast-credit-scoring-78b5688d9c-prqr5\" (UID: \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\") " pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:39:50.854932 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:50.854908 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f7765c56-67a7-40e3-bc3f-a8906af2b27c-offline-tls\") pod \"feast-credit-scoring-78b5688d9c-prqr5\" (UID: \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\") " pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:39:50.874448 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:50.874426 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqz52\" (UniqueName: \"kubernetes.io/projected/f7765c56-67a7-40e3-bc3f-a8906af2b27c-kube-api-access-gqz52\") pod \"feast-credit-scoring-78b5688d9c-prqr5\" (UID: \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\") " pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:39:51.044294 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:51.044239 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:39:51.180984 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:51.180960 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:39:51.356958 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:51.356901 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f7765c56-67a7-40e3-bc3f-a8906af2b27c-online-tls\") pod \"feast-credit-scoring-78b5688d9c-prqr5\" (UID: \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\") " pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:39:51.356958 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:51.356946 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f7765c56-67a7-40e3-bc3f-a8906af2b27c-registry-tls\") pod \"feast-credit-scoring-78b5688d9c-prqr5\" (UID: \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\") " pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:39:51.359296 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:51.359275 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f7765c56-67a7-40e3-bc3f-a8906af2b27c-registry-tls\") pod \"feast-credit-scoring-78b5688d9c-prqr5\" (UID: \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\") " pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:39:51.359395 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:51.359275 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f7765c56-67a7-40e3-bc3f-a8906af2b27c-online-tls\") pod \"feast-credit-scoring-78b5688d9c-prqr5\" (UID: \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\") " pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:39:51.612303 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:51.612251 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:39:51.905136 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:51.905034 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5"] Sep 17 07:39:51.907595 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:39:51.907571 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7765c56_67a7_40e3_bc3f_a8906af2b27c.slice/crio-080c9f70cc85baec46c4aa238bfbcc56c20868b457932ed4b27c33f0b1e46f19 WatchSource:0}: Error finding container 080c9f70cc85baec46c4aa238bfbcc56c20868b457932ed4b27c33f0b1e46f19: Status 404 returned error can't find the container with id 080c9f70cc85baec46c4aa238bfbcc56c20868b457932ed4b27c33f0b1e46f19 Sep 17 07:39:52.154402 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:52.154368 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" event={"ID":"f7765c56-67a7-40e3-bc3f-a8906af2b27c","Type":"ContainerStarted","Data":"e242746708f0ab1431f22d3f5a74bb7514ef746d80068cd807d0221163f0da2e"} Sep 17 07:39:52.154513 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:52.154420 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" event={"ID":"f7765c56-67a7-40e3-bc3f-a8906af2b27c","Type":"ContainerStarted","Data":"080c9f70cc85baec46c4aa238bfbcc56c20868b457932ed4b27c33f0b1e46f19"} Sep 17 07:39:53.159190 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:53.159159 2553 generic.go:356] "Generic (PLEG): container finished" podID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerID="e242746708f0ab1431f22d3f5a74bb7514ef746d80068cd807d0221163f0da2e" exitCode=0 Sep 17 07:39:53.159521 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:53.159192 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" event={"ID":"f7765c56-67a7-40e3-bc3f-a8906af2b27c","Type":"ContainerDied","Data":"e242746708f0ab1431f22d3f5a74bb7514ef746d80068cd807d0221163f0da2e"} Sep 17 07:39:54.166272 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:39:54.166239 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" event={"ID":"f7765c56-67a7-40e3-bc3f-a8906af2b27c","Type":"ContainerStarted","Data":"fd639ecdfe8214fd7f597bd8c9a851c69fec33ae2d4eff5ca468a43f14a0df2c"} Sep 17 07:40:00.192488 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:00.192458 2553 generic.go:356] "Generic (PLEG): container finished" podID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerID="fd639ecdfe8214fd7f597bd8c9a851c69fec33ae2d4eff5ca468a43f14a0df2c" exitCode=0 Sep 17 07:40:00.192922 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:00.192530 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" event={"ID":"f7765c56-67a7-40e3-bc3f-a8906af2b27c","Type":"ContainerDied","Data":"fd639ecdfe8214fd7f597bd8c9a851c69fec33ae2d4eff5ca468a43f14a0df2c"} Sep 17 07:40:00.197933 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:00.197914 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 17 07:40:01.201614 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:01.201576 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" event={"ID":"f7765c56-67a7-40e3-bc3f-a8906af2b27c","Type":"ContainerStarted","Data":"37725065d75757925237d925122eba98940aad3e579229c377eeac77d720b369"} Sep 17 07:40:01.201988 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:01.201653 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" event={"ID":"f7765c56-67a7-40e3-bc3f-a8906af2b27c","Type":"ContainerStarted","Data":"eccdf96c65c624f14dadeb6b8570a50152020920710daa2d46ea69ce8bb9d37d"} Sep 17 07:40:01.201988 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:01.201682 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" event={"ID":"f7765c56-67a7-40e3-bc3f-a8906af2b27c","Type":"ContainerStarted","Data":"a5e821eb39209d88f185dafddedf6d40db1dfcdfbb5825fddf74d13d2d201945"} Sep 17 07:40:01.241026 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:01.240972 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" podStartSLOduration=11.240958261 podStartE2EDuration="11.240958261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:39:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:40:01.233420585 +0000 UTC m=+722.791605393" watchObservedRunningTime="2026-09-17 07:40:01.240958261 +0000 UTC m=+722.799143069" Sep 17 07:40:01.613147 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:01.613053 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:40:01.613297 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:01.613174 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:40:01.613297 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:01.613201 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:40:03.612944 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:03.612905 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:40:03.613433 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:03.612978 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:40:03.613433 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:03.613016 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:40:03.615223 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:03.615181 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.32:6571: connect: connection refused" Sep 17 07:40:03.615223 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:03.615202 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.32:8816: connect: connection refused" Sep 17 07:40:03.615410 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:03.615297 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused" Sep 17 07:40:06.613123 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:06.613075 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.32:8816: connect: connection refused" Sep 17 07:40:06.613610 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:06.613084 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.32:6571: connect: connection refused" Sep 17 07:40:06.613610 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:06.613097 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused" Sep 17 07:40:09.613036 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:09.612982 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.32:6571: connect: connection refused" Sep 17 07:40:09.614034 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:09.614008 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:40:09.614958 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:09.614936 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:40:09.619804 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:09.619783 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:40:10.257213 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:10.257186 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:40:12.612735 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:12.612693 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.32:6571: connect: connection refused" Sep 17 07:40:15.613865 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:15.613831 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:40:15.614707 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:15.614689 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:40:15.933244 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:40:15.933140 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:41:14.358265 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:14.358229 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:41:19.337201 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:19.337172 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5"] Sep 17 07:41:19.337630 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:19.337411 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="registry" containerID="cri-o://a5e821eb39209d88f185dafddedf6d40db1dfcdfbb5825fddf74d13d2d201945" gracePeriod=30 Sep 17 07:41:19.337630 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:19.337465 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="online" containerID="cri-o://eccdf96c65c624f14dadeb6b8570a50152020920710daa2d46ea69ce8bb9d37d" gracePeriod=30 Sep 17 07:41:19.337630 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:19.337563 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="offline" containerID="cri-o://37725065d75757925237d925122eba98940aad3e579229c377eeac77d720b369" gracePeriod=30 Sep 17 07:41:19.382276 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:19.382252 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-s5lrs"] Sep 17 07:41:19.382430 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:19.382401 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-s5lrs" podUID="86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8" containerName="postgres" containerID="cri-o://fe7a63d176e6521d32dfc40c75e4325263137e1847f3261f1e252bdded999d0b" gracePeriod=30 Sep 17 07:41:19.397546 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:19.397525 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-ghkcf"] Sep 17 07:41:19.397673 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:19.397657 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/redis-69bbd798b6-ghkcf" podUID="793abee0-3db5-4854-a589-989feaa3be59" containerName="redis" containerID="cri-o://3546b83f64a6b0edcf0faf0dc924feb4af9d4d0fbe351c671ea52f0de1d0b1a6" gracePeriod=30 Sep 17 07:41:19.474750 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:19.474725 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:41:19.539146 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:19.539115 2553 generic.go:356] "Generic (PLEG): container finished" podID="86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8" containerID="fe7a63d176e6521d32dfc40c75e4325263137e1847f3261f1e252bdded999d0b" exitCode=0 Sep 17 07:41:19.539315 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:19.539215 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-s5lrs" event={"ID":"86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8","Type":"ContainerDied","Data":"fe7a63d176e6521d32dfc40c75e4325263137e1847f3261f1e252bdded999d0b"} Sep 17 07:41:19.541110 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:19.541089 2553 generic.go:356] "Generic (PLEG): container finished" podID="793abee0-3db5-4854-a589-989feaa3be59" containerID="3546b83f64a6b0edcf0faf0dc924feb4af9d4d0fbe351c671ea52f0de1d0b1a6" exitCode=0 Sep 17 07:41:19.541218 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:19.541171 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-ghkcf" event={"ID":"793abee0-3db5-4854-a589-989feaa3be59","Type":"ContainerDied","Data":"3546b83f64a6b0edcf0faf0dc924feb4af9d4d0fbe351c671ea52f0de1d0b1a6"} Sep 17 07:41:19.655074 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:19.655053 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-s5lrs" Sep 17 07:41:19.657429 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:19.657411 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-ghkcf" Sep 17 07:41:19.795761 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:19.795722 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bm5v\" (UniqueName: \"kubernetes.io/projected/86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8-kube-api-access-8bm5v\") pod \"86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8\" (UID: \"86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8\") " Sep 17 07:41:19.795952 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:19.795806 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8-postgresdata\") pod \"86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8\" (UID: \"86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8\") " Sep 17 07:41:19.795952 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:19.795848 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmdnz\" (UniqueName: \"kubernetes.io/projected/793abee0-3db5-4854-a589-989feaa3be59-kube-api-access-zmdnz\") pod \"793abee0-3db5-4854-a589-989feaa3be59\" (UID: \"793abee0-3db5-4854-a589-989feaa3be59\") " Sep 17 07:41:19.797805 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:19.797770 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8-kube-api-access-8bm5v" (OuterVolumeSpecName: "kube-api-access-8bm5v") pod "86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8" (UID: "86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8"). InnerVolumeSpecName "kube-api-access-8bm5v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:41:19.798003 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:19.797976 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/793abee0-3db5-4854-a589-989feaa3be59-kube-api-access-zmdnz" (OuterVolumeSpecName: "kube-api-access-zmdnz") pod "793abee0-3db5-4854-a589-989feaa3be59" (UID: "793abee0-3db5-4854-a589-989feaa3be59"). InnerVolumeSpecName "kube-api-access-zmdnz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:41:19.824466 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:19.824435 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8" (UID: "86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:41:19.896999 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:19.896925 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmdnz\" (UniqueName: \"kubernetes.io/projected/793abee0-3db5-4854-a589-989feaa3be59-kube-api-access-zmdnz\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:41:19.896999 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:19.896952 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bm5v\" (UniqueName: \"kubernetes.io/projected/86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8-kube-api-access-8bm5v\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:41:19.896999 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:19.896963 2553 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8-postgresdata\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:41:20.252801 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:20.252768 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused" Sep 17 07:41:20.546066 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:20.546038 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-s5lrs" Sep 17 07:41:20.546415 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:20.546070 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-s5lrs" event={"ID":"86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8","Type":"ContainerDied","Data":"f4a9f3b13c6524e3e4d266bac24b3bb9ec13eb326394b5d420b2ccec5469ca27"} Sep 17 07:41:20.546415 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:20.546126 2553 scope.go:117] "RemoveContainer" containerID="fe7a63d176e6521d32dfc40c75e4325263137e1847f3261f1e252bdded999d0b" Sep 17 07:41:20.547557 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:20.547539 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-ghkcf" Sep 17 07:41:20.547557 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:20.547546 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-ghkcf" event={"ID":"793abee0-3db5-4854-a589-989feaa3be59","Type":"ContainerDied","Data":"7afc0992b7b916db978eb6d7152be610dae85b19825f410347dc8b672b995383"} Sep 17 07:41:20.550822 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:20.550799 2553 generic.go:356] "Generic (PLEG): container finished" podID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerID="eccdf96c65c624f14dadeb6b8570a50152020920710daa2d46ea69ce8bb9d37d" exitCode=0 Sep 17 07:41:20.550947 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:20.550855 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" event={"ID":"f7765c56-67a7-40e3-bc3f-a8906af2b27c","Type":"ContainerDied","Data":"eccdf96c65c624f14dadeb6b8570a50152020920710daa2d46ea69ce8bb9d37d"} Sep 17 07:41:20.555007 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:20.554990 2553 scope.go:117] "RemoveContainer" containerID="3546b83f64a6b0edcf0faf0dc924feb4af9d4d0fbe351c671ea52f0de1d0b1a6" Sep 17 07:41:20.585708 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:20.585691 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-s5lrs"] Sep 17 07:41:20.591430 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:20.591413 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-s5lrs"] Sep 17 07:41:20.604227 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:20.604208 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-ghkcf"] Sep 17 07:41:20.606666 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:20.606641 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-69bbd798b6-ghkcf"] Sep 17 07:41:20.934485 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:20.934461 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="793abee0-3db5-4854-a589-989feaa3be59" path="/var/lib/kubelet/pods/793abee0-3db5-4854-a589-989feaa3be59/volumes" Sep 17 07:41:20.934789 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:20.934776 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8" path="/var/lib/kubelet/pods/86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8/volumes" Sep 17 07:41:30.253503 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:30.253458 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused" Sep 17 07:41:40.253007 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:40.252964 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused" Sep 17 07:41:40.253476 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:40.253085 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:41:49.615008 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:49.614858 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.32:8816: connect: connection refused" Sep 17 07:41:49.674215 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:49.674183 2553 generic.go:356] "Generic (PLEG): container finished" podID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerID="37725065d75757925237d925122eba98940aad3e579229c377eeac77d720b369" exitCode=137 Sep 17 07:41:49.674215 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:49.674209 2553 generic.go:356] "Generic (PLEG): container finished" podID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerID="a5e821eb39209d88f185dafddedf6d40db1dfcdfbb5825fddf74d13d2d201945" exitCode=137 Sep 17 07:41:49.674441 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:49.674256 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" event={"ID":"f7765c56-67a7-40e3-bc3f-a8906af2b27c","Type":"ContainerDied","Data":"37725065d75757925237d925122eba98940aad3e579229c377eeac77d720b369"} Sep 17 07:41:49.674441 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:49.674319 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" event={"ID":"f7765c56-67a7-40e3-bc3f-a8906af2b27c","Type":"ContainerDied","Data":"a5e821eb39209d88f185dafddedf6d40db1dfcdfbb5825fddf74d13d2d201945"} Sep 17 07:41:49.985375 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:49.985346 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:41:50.029834 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.029798 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f7765c56-67a7-40e3-bc3f-a8906af2b27c-offline-tls\") pod \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\" (UID: \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\") " Sep 17 07:41:50.030045 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.029853 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f7765c56-67a7-40e3-bc3f-a8906af2b27c-feast-data\") pod \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\" (UID: \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\") " Sep 17 07:41:50.030045 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.029928 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f7765c56-67a7-40e3-bc3f-a8906af2b27c-online-tls\") pod \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\" (UID: \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\") " Sep 17 07:41:50.030045 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.029960 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f7765c56-67a7-40e3-bc3f-a8906af2b27c-registry-tls\") pod \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\" (UID: \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\") " Sep 17 07:41:50.030045 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.030019 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqz52\" (UniqueName: \"kubernetes.io/projected/f7765c56-67a7-40e3-bc3f-a8906af2b27c-kube-api-access-gqz52\") pod \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\" (UID: \"f7765c56-67a7-40e3-bc3f-a8906af2b27c\") " Sep 17 07:41:50.032092 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.032059 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7765c56-67a7-40e3-bc3f-a8906af2b27c-online-tls" (OuterVolumeSpecName: "online-tls") pod "f7765c56-67a7-40e3-bc3f-a8906af2b27c" (UID: "f7765c56-67a7-40e3-bc3f-a8906af2b27c"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:41:50.032514 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.032483 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7765c56-67a7-40e3-bc3f-a8906af2b27c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f7765c56-67a7-40e3-bc3f-a8906af2b27c" (UID: "f7765c56-67a7-40e3-bc3f-a8906af2b27c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:41:50.032598 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.032538 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7765c56-67a7-40e3-bc3f-a8906af2b27c-kube-api-access-gqz52" (OuterVolumeSpecName: "kube-api-access-gqz52") pod "f7765c56-67a7-40e3-bc3f-a8906af2b27c" (UID: "f7765c56-67a7-40e3-bc3f-a8906af2b27c"). InnerVolumeSpecName "kube-api-access-gqz52". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:41:50.032701 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.032686 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7765c56-67a7-40e3-bc3f-a8906af2b27c-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "f7765c56-67a7-40e3-bc3f-a8906af2b27c" (UID: "f7765c56-67a7-40e3-bc3f-a8906af2b27c"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:41:50.038057 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.038029 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7765c56-67a7-40e3-bc3f-a8906af2b27c-feast-data" (OuterVolumeSpecName: "feast-data") pod "f7765c56-67a7-40e3-bc3f-a8906af2b27c" (UID: "f7765c56-67a7-40e3-bc3f-a8906af2b27c"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:41:50.130791 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.130757 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqz52\" (UniqueName: \"kubernetes.io/projected/f7765c56-67a7-40e3-bc3f-a8906af2b27c-kube-api-access-gqz52\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:41:50.130791 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.130784 2553 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f7765c56-67a7-40e3-bc3f-a8906af2b27c-offline-tls\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:41:50.130791 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.130793 2553 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f7765c56-67a7-40e3-bc3f-a8906af2b27c-feast-data\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:41:50.130791 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.130802 2553 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f7765c56-67a7-40e3-bc3f-a8906af2b27c-online-tls\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:41:50.131081 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.130809 2553 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f7765c56-67a7-40e3-bc3f-a8906af2b27c-registry-tls\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:41:50.681821 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.681785 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" event={"ID":"f7765c56-67a7-40e3-bc3f-a8906af2b27c","Type":"ContainerDied","Data":"080c9f70cc85baec46c4aa238bfbcc56c20868b457932ed4b27c33f0b1e46f19"} Sep 17 07:41:50.681821 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.681824 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5" Sep 17 07:41:50.682312 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.681851 2553 scope.go:117] "RemoveContainer" containerID="37725065d75757925237d925122eba98940aad3e579229c377eeac77d720b369" Sep 17 07:41:50.690960 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.690942 2553 scope.go:117] "RemoveContainer" containerID="eccdf96c65c624f14dadeb6b8570a50152020920710daa2d46ea69ce8bb9d37d" Sep 17 07:41:50.698312 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.698290 2553 scope.go:117] "RemoveContainer" containerID="a5e821eb39209d88f185dafddedf6d40db1dfcdfbb5825fddf74d13d2d201945" Sep 17 07:41:50.705774 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.705750 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5"] Sep 17 07:41:50.706267 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.706243 2553 scope.go:117] "RemoveContainer" containerID="fd639ecdfe8214fd7f597bd8c9a851c69fec33ae2d4eff5ca468a43f14a0df2c" Sep 17 07:41:50.709791 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.709770 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-78b5688d9c-prqr5"] Sep 17 07:41:50.714030 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.714011 2553 scope.go:117] "RemoveContainer" containerID="e242746708f0ab1431f22d3f5a74bb7514ef746d80068cd807d0221163f0da2e" Sep 17 07:41:50.933401 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:50.933309 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" path="/var/lib/kubelet/pods/f7765c56-67a7-40e3-bc3f-a8906af2b27c/volumes" Sep 17 07:41:57.648171 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:57.648136 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:41:57.749378 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:57.749345 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:41:57.846436 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:57.846399 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:41:58.314900 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:41:58.314847 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:42:58.875469 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:42:58.875442 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-5kks2_2455e65f-98bb-463d-a204-5dbd5d69b5f3/console-operator/1.log" Sep 17 07:42:58.876213 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:42:58.876192 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-5kks2_2455e65f-98bb-463d-a204-5dbd5d69b5f3/console-operator/1.log" Sep 17 07:42:58.882636 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:42:58.882611 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/ovn-acl-logging/0.log" Sep 17 07:42:58.883028 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:42:58.883013 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/ovn-acl-logging/0.log" Sep 17 07:43:11.053945 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:43:11.053914 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:44:30.439993 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.439921 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:44:30.601369 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.601341 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-5f69j"] Sep 17 07:44:30.601636 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.601626 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="registry" Sep 17 07:44:30.601691 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.601638 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="registry" Sep 17 07:44:30.601691 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.601647 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="793abee0-3db5-4854-a589-989feaa3be59" containerName="redis" Sep 17 07:44:30.601691 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.601654 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="793abee0-3db5-4854-a589-989feaa3be59" containerName="redis" Sep 17 07:44:30.601691 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.601660 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="feast-init" Sep 17 07:44:30.601691 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.601666 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="feast-init" Sep 17 07:44:30.601691 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.601676 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="offline" Sep 17 07:44:30.601691 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.601681 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="offline" Sep 17 07:44:30.601691 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.601690 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8" containerName="postgres" Sep 17 07:44:30.601691 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.601695 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8" containerName="postgres" Sep 17 07:44:30.602029 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.601702 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="feast-apply" Sep 17 07:44:30.602029 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.601707 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="feast-apply" Sep 17 07:44:30.602029 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.601712 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="online" Sep 17 07:44:30.602029 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.601717 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="online" Sep 17 07:44:30.602029 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.601771 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="registry" Sep 17 07:44:30.602029 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.601778 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="offline" Sep 17 07:44:30.602029 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.601784 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="86001f7b-1d5e-4a0f-86f5-ec3fa6bd61d8" containerName="postgres" Sep 17 07:44:30.602029 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.601789 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7765c56-67a7-40e3-bc3f-a8906af2b27c" containerName="online" Sep 17 07:44:30.602029 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.601795 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="793abee0-3db5-4854-a589-989feaa3be59" containerName="redis" Sep 17 07:44:30.604541 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.604525 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-5f69j" Sep 17 07:44:30.611262 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.611240 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Sep 17 07:44:30.611262 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.611256 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Sep 17 07:44:30.612152 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.612132 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-c55gk\"" Sep 17 07:44:30.674981 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.674949 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-5f69j"] Sep 17 07:44:30.711533 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.711507 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f89l4\" (UniqueName: \"kubernetes.io/projected/0e93f70f-0817-42c7-89be-b7cff2abff09-kube-api-access-f89l4\") pod \"redis-69bbd798b6-5f69j\" (UID: \"0e93f70f-0817-42c7-89be-b7cff2abff09\") " pod="test-ns-feast-rest/redis-69bbd798b6-5f69j" Sep 17 07:44:30.812786 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.812750 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f89l4\" (UniqueName: \"kubernetes.io/projected/0e93f70f-0817-42c7-89be-b7cff2abff09-kube-api-access-f89l4\") pod \"redis-69bbd798b6-5f69j\" (UID: \"0e93f70f-0817-42c7-89be-b7cff2abff09\") " pod="test-ns-feast-rest/redis-69bbd798b6-5f69j" Sep 17 07:44:30.823089 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.823061 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f89l4\" (UniqueName: \"kubernetes.io/projected/0e93f70f-0817-42c7-89be-b7cff2abff09-kube-api-access-f89l4\") pod \"redis-69bbd798b6-5f69j\" (UID: \"0e93f70f-0817-42c7-89be-b7cff2abff09\") " pod="test-ns-feast-rest/redis-69bbd798b6-5f69j" Sep 17 07:44:30.932091 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:30.932057 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-5f69j" Sep 17 07:44:31.212388 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:31.212361 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-5f69j"] Sep 17 07:44:31.213738 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:44:31.213714 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e93f70f_0817_42c7_89be_b7cff2abff09.slice/crio-4c09c3d13b34ff05838dc374e26aa2f8748e8081e8f94e5a54f5fb7a21850de2 WatchSource:0}: Error finding container 4c09c3d13b34ff05838dc374e26aa2f8748e8081e8f94e5a54f5fb7a21850de2: Status 404 returned error can't find the container with id 4c09c3d13b34ff05838dc374e26aa2f8748e8081e8f94e5a54f5fb7a21850de2 Sep 17 07:44:31.283710 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:31.283683 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-5f69j" event={"ID":"0e93f70f-0817-42c7-89be-b7cff2abff09","Type":"ContainerStarted","Data":"df37daa0a19a2ed2de0c05d0c6ce6da5911349a3bf7d60f993893e09517ce029"} Sep 17 07:44:31.283832 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:31.283721 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-5f69j" event={"ID":"0e93f70f-0817-42c7-89be-b7cff2abff09","Type":"ContainerStarted","Data":"4c09c3d13b34ff05838dc374e26aa2f8748e8081e8f94e5a54f5fb7a21850de2"} Sep 17 07:44:32.097045 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:32.097011 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:44:36.129148 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:36.129090 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/redis-69bbd798b6-5f69j" podStartSLOduration=6.129069706 podStartE2EDuration="6.129069706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:44:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:44:31.306092399 +0000 UTC m=+992.864277189" watchObservedRunningTime="2026-09-17 07:44:36.129069706 +0000 UTC m=+997.687254513" Sep 17 07:44:36.129993 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:36.129975 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-csz68"] Sep 17 07:44:36.133192 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:36.133177 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-csz68" Sep 17 07:44:36.136837 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:36.136814 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Sep 17 07:44:36.160896 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:36.160852 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-csz68"] Sep 17 07:44:36.181412 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:36.181361 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:44:36.257535 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:36.257504 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx5rj\" (UniqueName: \"kubernetes.io/projected/68606f55-e7f8-446f-bbf5-2ca1607659b5-kube-api-access-xx5rj\") pod \"postgres-864846d4f5-csz68\" (UID: \"68606f55-e7f8-446f-bbf5-2ca1607659b5\") " pod="test-ns-feast-rest/postgres-864846d4f5-csz68" Sep 17 07:44:36.257535 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:36.257541 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/68606f55-e7f8-446f-bbf5-2ca1607659b5-postgresdata\") pod \"postgres-864846d4f5-csz68\" (UID: \"68606f55-e7f8-446f-bbf5-2ca1607659b5\") " pod="test-ns-feast-rest/postgres-864846d4f5-csz68" Sep 17 07:44:36.358426 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:36.358384 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx5rj\" (UniqueName: \"kubernetes.io/projected/68606f55-e7f8-446f-bbf5-2ca1607659b5-kube-api-access-xx5rj\") pod \"postgres-864846d4f5-csz68\" (UID: \"68606f55-e7f8-446f-bbf5-2ca1607659b5\") " pod="test-ns-feast-rest/postgres-864846d4f5-csz68" Sep 17 07:44:36.358426 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:36.358432 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/68606f55-e7f8-446f-bbf5-2ca1607659b5-postgresdata\") pod \"postgres-864846d4f5-csz68\" (UID: \"68606f55-e7f8-446f-bbf5-2ca1607659b5\") " pod="test-ns-feast-rest/postgres-864846d4f5-csz68" Sep 17 07:44:36.358822 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:36.358801 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/68606f55-e7f8-446f-bbf5-2ca1607659b5-postgresdata\") pod \"postgres-864846d4f5-csz68\" (UID: \"68606f55-e7f8-446f-bbf5-2ca1607659b5\") " pod="test-ns-feast-rest/postgres-864846d4f5-csz68" Sep 17 07:44:36.371401 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:36.371372 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx5rj\" (UniqueName: \"kubernetes.io/projected/68606f55-e7f8-446f-bbf5-2ca1607659b5-kube-api-access-xx5rj\") pod \"postgres-864846d4f5-csz68\" (UID: \"68606f55-e7f8-446f-bbf5-2ca1607659b5\") " pod="test-ns-feast-rest/postgres-864846d4f5-csz68" Sep 17 07:44:36.441603 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:36.441510 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-csz68" Sep 17 07:44:36.726579 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:36.726553 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-csz68"] Sep 17 07:44:36.729321 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:44:36.729276 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod68606f55_e7f8_446f_bbf5_2ca1607659b5.slice/crio-13423aa817a98254fe06fa125d1f334194f72c3382c5ddc5d9a04bc9bae91284 WatchSource:0}: Error finding container 13423aa817a98254fe06fa125d1f334194f72c3382c5ddc5d9a04bc9bae91284: Status 404 returned error can't find the container with id 13423aa817a98254fe06fa125d1f334194f72c3382c5ddc5d9a04bc9bae91284 Sep 17 07:44:37.309445 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:37.309408 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-csz68" event={"ID":"68606f55-e7f8-446f-bbf5-2ca1607659b5","Type":"ContainerStarted","Data":"bdc746c9a2e0c3987dcb9ccde772bd29198acb3b84d84e31368c28bbee2ba178"} Sep 17 07:44:37.309898 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:37.309449 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-csz68" event={"ID":"68606f55-e7f8-446f-bbf5-2ca1607659b5","Type":"ContainerStarted","Data":"13423aa817a98254fe06fa125d1f334194f72c3382c5ddc5d9a04bc9bae91284"} Sep 17 07:44:37.329643 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:37.329590 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-csz68" podStartSLOduration=1.329576956 podStartE2EDuration="1.329576956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:44:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:44:37.327377836 +0000 UTC m=+998.885562645" watchObservedRunningTime="2026-09-17 07:44:37.329576956 +0000 UTC m=+998.887761763" Sep 17 07:44:41.671711 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:41.671676 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:44:41.768642 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:41.768602 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:44:41.824801 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:41.824760 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x"] Sep 17 07:44:41.828477 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:41.828454 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:44:41.835850 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:41.833435 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Sep 17 07:44:41.835850 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:41.833690 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\"" Sep 17 07:44:41.835850 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:41.833921 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\"" Sep 17 07:44:41.835850 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:41.834198 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-t8qw5\"" Sep 17 07:44:41.902928 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:41.902895 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:44:41.911027 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:41.910997 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x"] Sep 17 07:44:42.004627 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:42.004592 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/25a504f9-ac01-4933-b934-cb9c30167f2b-feast-data\") pod \"feast-credit-scoring-7d499b5545-vmk6x\" (UID: \"25a504f9-ac01-4933-b934-cb9c30167f2b\") " pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:44:42.004809 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:42.004651 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch2xr\" (UniqueName: \"kubernetes.io/projected/25a504f9-ac01-4933-b934-cb9c30167f2b-kube-api-access-ch2xr\") pod \"feast-credit-scoring-7d499b5545-vmk6x\" (UID: \"25a504f9-ac01-4933-b934-cb9c30167f2b\") " pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:44:42.004809 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:42.004728 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/25a504f9-ac01-4933-b934-cb9c30167f2b-registry-tls\") pod \"feast-credit-scoring-7d499b5545-vmk6x\" (UID: \"25a504f9-ac01-4933-b934-cb9c30167f2b\") " pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:44:42.004809 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:42.004761 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/25a504f9-ac01-4933-b934-cb9c30167f2b-online-tls\") pod \"feast-credit-scoring-7d499b5545-vmk6x\" (UID: \"25a504f9-ac01-4933-b934-cb9c30167f2b\") " pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:44:42.007918 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:42.007894 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:44:42.105664 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:42.105628 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch2xr\" (UniqueName: \"kubernetes.io/projected/25a504f9-ac01-4933-b934-cb9c30167f2b-kube-api-access-ch2xr\") pod \"feast-credit-scoring-7d499b5545-vmk6x\" (UID: \"25a504f9-ac01-4933-b934-cb9c30167f2b\") " pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:44:42.105856 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:42.105678 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/25a504f9-ac01-4933-b934-cb9c30167f2b-registry-tls\") pod \"feast-credit-scoring-7d499b5545-vmk6x\" (UID: \"25a504f9-ac01-4933-b934-cb9c30167f2b\") " pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:44:42.105856 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:42.105776 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/25a504f9-ac01-4933-b934-cb9c30167f2b-online-tls\") pod \"feast-credit-scoring-7d499b5545-vmk6x\" (UID: \"25a504f9-ac01-4933-b934-cb9c30167f2b\") " pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:44:42.105856 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:42.105849 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/25a504f9-ac01-4933-b934-cb9c30167f2b-feast-data\") pod \"feast-credit-scoring-7d499b5545-vmk6x\" (UID: \"25a504f9-ac01-4933-b934-cb9c30167f2b\") " pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:44:42.106241 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:42.106220 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/25a504f9-ac01-4933-b934-cb9c30167f2b-feast-data\") pod \"feast-credit-scoring-7d499b5545-vmk6x\" (UID: \"25a504f9-ac01-4933-b934-cb9c30167f2b\") " pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:44:42.108080 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:42.108059 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/25a504f9-ac01-4933-b934-cb9c30167f2b-registry-tls\") pod \"feast-credit-scoring-7d499b5545-vmk6x\" (UID: \"25a504f9-ac01-4933-b934-cb9c30167f2b\") " pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:44:42.108244 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:42.108198 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/25a504f9-ac01-4933-b934-cb9c30167f2b-online-tls\") pod \"feast-credit-scoring-7d499b5545-vmk6x\" (UID: \"25a504f9-ac01-4933-b934-cb9c30167f2b\") " pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:44:42.119199 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:42.119179 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch2xr\" (UniqueName: \"kubernetes.io/projected/25a504f9-ac01-4933-b934-cb9c30167f2b-kube-api-access-ch2xr\") pod \"feast-credit-scoring-7d499b5545-vmk6x\" (UID: \"25a504f9-ac01-4933-b934-cb9c30167f2b\") " pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:44:42.143118 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:42.143091 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:44:42.432846 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:42.432818 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x"] Sep 17 07:44:42.433289 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:44:42.433266 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25a504f9_ac01_4933_b934_cb9c30167f2b.slice/crio-fe82759a85c65c2a6c17e66e85011d21ae11a76cc6bf25f8415227e049d784b3 WatchSource:0}: Error finding container fe82759a85c65c2a6c17e66e85011d21ae11a76cc6bf25f8415227e049d784b3: Status 404 returned error can't find the container with id fe82759a85c65c2a6c17e66e85011d21ae11a76cc6bf25f8415227e049d784b3 Sep 17 07:44:43.333653 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:43.333618 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" event={"ID":"25a504f9-ac01-4933-b934-cb9c30167f2b","Type":"ContainerStarted","Data":"8c4afa3f9ff23aecf2f33cb241011ddbe567e8ca21b716ed8a4f68852f2daf2a"} Sep 17 07:44:43.334127 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:43.333674 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" event={"ID":"25a504f9-ac01-4933-b934-cb9c30167f2b","Type":"ContainerStarted","Data":"fe82759a85c65c2a6c17e66e85011d21ae11a76cc6bf25f8415227e049d784b3"} Sep 17 07:44:44.339714 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:44.339681 2553 generic.go:356] "Generic (PLEG): container finished" podID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerID="8c4afa3f9ff23aecf2f33cb241011ddbe567e8ca21b716ed8a4f68852f2daf2a" exitCode=0 Sep 17 07:44:44.340124 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:44.339737 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" event={"ID":"25a504f9-ac01-4933-b934-cb9c30167f2b","Type":"ContainerDied","Data":"8c4afa3f9ff23aecf2f33cb241011ddbe567e8ca21b716ed8a4f68852f2daf2a"} Sep 17 07:44:45.346132 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:45.346097 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" event={"ID":"25a504f9-ac01-4933-b934-cb9c30167f2b","Type":"ContainerStarted","Data":"6f4efbe03b075eb133bff84697ae28760982876f744019283c8f3da6611492f7"} Sep 17 07:44:51.372764 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:51.372733 2553 generic.go:356] "Generic (PLEG): container finished" podID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerID="6f4efbe03b075eb133bff84697ae28760982876f744019283c8f3da6611492f7" exitCode=0 Sep 17 07:44:51.373098 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:51.372806 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" event={"ID":"25a504f9-ac01-4933-b934-cb9c30167f2b","Type":"ContainerDied","Data":"6f4efbe03b075eb133bff84697ae28760982876f744019283c8f3da6611492f7"} Sep 17 07:44:52.380057 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:52.380014 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" event={"ID":"25a504f9-ac01-4933-b934-cb9c30167f2b","Type":"ContainerStarted","Data":"76e3db87df313021138dd3aa5569bb461e428dbe95e52141b8d4f7be43c236c1"} Sep 17 07:44:52.380427 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:52.380141 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" event={"ID":"25a504f9-ac01-4933-b934-cb9c30167f2b","Type":"ContainerStarted","Data":"085a542f70d8ab99e5c035d82cc13249ec95f55553aeed5bf5bccfb4a93ee3be"} Sep 17 07:44:52.426671 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:52.426605 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" podStartSLOduration=11.426584106 podStartE2EDuration="11.426584106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:44:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:44:52.424562699 +0000 UTC m=+1013.982747507" watchObservedRunningTime="2026-09-17 07:44:52.426584106 +0000 UTC m=+1013.984768933" Sep 17 07:44:54.143505 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:54.143465 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:44:54.143908 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:54.143537 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:44:54.145193 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:54.145163 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" podUID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.35:6571: connect: connection refused" Sep 17 07:44:54.145326 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:54.145205 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" podUID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerName="online" probeResult="failure" output="Get \"https://10.133.0.35:6567/health\": dial tcp 10.133.0.35:6567: connect: connection refused" Sep 17 07:44:55.925504 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:55.925467 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:44:57.144327 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:57.144280 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" podUID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.35:6571: connect: connection refused" Sep 17 07:44:57.144814 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:44:57.144366 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" podUID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerName="online" probeResult="failure" output="Get \"https://10.133.0.35:6567/health\": dial tcp 10.133.0.35:6567: connect: connection refused" Sep 17 07:45:00.144502 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:00.144458 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" podUID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.35:6571: connect: connection refused" Sep 17 07:45:00.150775 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:00.150741 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:45:00.151174 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:00.151149 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:45:00.154057 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:00.154038 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:45:02.144125 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:02.144065 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:45:03.144243 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:03.144196 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" podUID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.35:6571: connect: connection refused" Sep 17 07:45:06.144324 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:06.144291 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:45:06.145033 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:06.145013 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:45:06.577469 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:06.577435 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:45:07.496412 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:07.496364 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:45:07.599339 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:07.599301 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:45:07.631475 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:07.631445 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5"] Sep 17 07:45:07.636661 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:07.636642 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:45:07.644581 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:07.644565 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Sep 17 07:45:07.644849 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:07.644834 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-7hffv\"" Sep 17 07:45:07.644963 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:07.644847 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\"" Sep 17 07:45:07.646091 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:07.646072 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\"" Sep 17 07:45:07.653173 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:07.653149 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5"] Sep 17 07:45:07.687908 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:07.687862 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:45:07.720610 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:07.720584 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-online-tls\") pod \"feast-driver-ranking-85f79c89d-cb5x5\" (UID: \"63e7a5cd-3238-4d6a-848f-3fd1fce9f149\") " pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:45:07.720730 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:07.720621 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4qtx\" (UniqueName: \"kubernetes.io/projected/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-kube-api-access-c4qtx\") pod \"feast-driver-ranking-85f79c89d-cb5x5\" (UID: \"63e7a5cd-3238-4d6a-848f-3fd1fce9f149\") " pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:45:07.720730 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:07.720689 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-feast-data\") pod \"feast-driver-ranking-85f79c89d-cb5x5\" (UID: \"63e7a5cd-3238-4d6a-848f-3fd1fce9f149\") " pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:45:07.720850 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:07.720734 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-registry-tls\") pod \"feast-driver-ranking-85f79c89d-cb5x5\" (UID: \"63e7a5cd-3238-4d6a-848f-3fd1fce9f149\") " pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:45:07.821811 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:07.821738 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4qtx\" (UniqueName: \"kubernetes.io/projected/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-kube-api-access-c4qtx\") pod \"feast-driver-ranking-85f79c89d-cb5x5\" (UID: \"63e7a5cd-3238-4d6a-848f-3fd1fce9f149\") " pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:45:07.821811 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:07.821781 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-feast-data\") pod \"feast-driver-ranking-85f79c89d-cb5x5\" (UID: \"63e7a5cd-3238-4d6a-848f-3fd1fce9f149\") " pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:45:07.821811 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:07.821803 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-registry-tls\") pod \"feast-driver-ranking-85f79c89d-cb5x5\" (UID: \"63e7a5cd-3238-4d6a-848f-3fd1fce9f149\") " pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:45:07.822097 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:07.821909 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-online-tls\") pod \"feast-driver-ranking-85f79c89d-cb5x5\" (UID: \"63e7a5cd-3238-4d6a-848f-3fd1fce9f149\") " pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:45:07.822097 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:45:07.821985 2553 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-online-tls: secret "feast-driver-ranking-online-tls" not found Sep 17 07:45:07.822097 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:45:07.822057 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-online-tls podName:63e7a5cd-3238-4d6a-848f-3fd1fce9f149 nodeName:}" failed. No retries permitted until 2026-09-17 07:45:08.322038026 +0000 UTC m=+1029.880222812 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-online-tls") pod "feast-driver-ranking-85f79c89d-cb5x5" (UID: "63e7a5cd-3238-4d6a-848f-3fd1fce9f149") : secret "feast-driver-ranking-online-tls" not found Sep 17 07:45:07.822261 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:07.822208 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-feast-data\") pod \"feast-driver-ranking-85f79c89d-cb5x5\" (UID: \"63e7a5cd-3238-4d6a-848f-3fd1fce9f149\") " pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:45:07.824296 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:07.824279 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-registry-tls\") pod \"feast-driver-ranking-85f79c89d-cb5x5\" (UID: \"63e7a5cd-3238-4d6a-848f-3fd1fce9f149\") " pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:45:07.835415 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:07.835383 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4qtx\" (UniqueName: \"kubernetes.io/projected/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-kube-api-access-c4qtx\") pod \"feast-driver-ranking-85f79c89d-cb5x5\" (UID: \"63e7a5cd-3238-4d6a-848f-3fd1fce9f149\") " pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:45:07.967500 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:07.967470 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:45:08.326252 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:08.326221 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-online-tls\") pod \"feast-driver-ranking-85f79c89d-cb5x5\" (UID: \"63e7a5cd-3238-4d6a-848f-3fd1fce9f149\") " pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:45:08.328514 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:08.328483 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-online-tls\") pod \"feast-driver-ranking-85f79c89d-cb5x5\" (UID: \"63e7a5cd-3238-4d6a-848f-3fd1fce9f149\") " pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:45:08.546244 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:08.546204 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:45:08.823554 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:08.823529 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5"] Sep 17 07:45:08.825664 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:45:08.825636 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63e7a5cd_3238_4d6a_848f_3fd1fce9f149.slice/crio-8e48e85bde3d376c8071881f819ba74d7c22f9e86f340820469d61e065c50f52 WatchSource:0}: Error finding container 8e48e85bde3d376c8071881f819ba74d7c22f9e86f340820469d61e065c50f52: Status 404 returned error can't find the container with id 8e48e85bde3d376c8071881f819ba74d7c22f9e86f340820469d61e065c50f52 Sep 17 07:45:09.461931 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:09.461863 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" event={"ID":"63e7a5cd-3238-4d6a-848f-3fd1fce9f149","Type":"ContainerStarted","Data":"41776bc0d9c647c6929a2c307565d7fdb2fa2ec11477082f923f0109fd32aa58"} Sep 17 07:45:09.461931 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:09.461936 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" event={"ID":"63e7a5cd-3238-4d6a-848f-3fd1fce9f149","Type":"ContainerStarted","Data":"8e48e85bde3d376c8071881f819ba74d7c22f9e86f340820469d61e065c50f52"} Sep 17 07:45:14.487049 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:14.487013 2553 generic.go:356] "Generic (PLEG): container finished" podID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerID="41776bc0d9c647c6929a2c307565d7fdb2fa2ec11477082f923f0109fd32aa58" exitCode=0 Sep 17 07:45:14.487426 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:14.487086 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" event={"ID":"63e7a5cd-3238-4d6a-848f-3fd1fce9f149","Type":"ContainerDied","Data":"41776bc0d9c647c6929a2c307565d7fdb2fa2ec11477082f923f0109fd32aa58"} Sep 17 07:45:15.493628 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:15.493589 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" event={"ID":"63e7a5cd-3238-4d6a-848f-3fd1fce9f149","Type":"ContainerStarted","Data":"b918ca7415def1426710683d75c32a39e8124ec68463f370eb0586789071efdd"} Sep 17 07:45:21.527544 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:21.527504 2553 generic.go:356] "Generic (PLEG): container finished" podID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerID="b918ca7415def1426710683d75c32a39e8124ec68463f370eb0586789071efdd" exitCode=0 Sep 17 07:45:21.527926 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:21.527550 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" event={"ID":"63e7a5cd-3238-4d6a-848f-3fd1fce9f149","Type":"ContainerDied","Data":"b918ca7415def1426710683d75c32a39e8124ec68463f370eb0586789071efdd"} Sep 17 07:45:22.535621 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:22.535589 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" event={"ID":"63e7a5cd-3238-4d6a-848f-3fd1fce9f149","Type":"ContainerStarted","Data":"c8f2b022545aa248ca16459ee4f4b60941c9a17c80fee8c90ea0ae3850b76141"} Sep 17 07:45:22.536167 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:22.535640 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" event={"ID":"63e7a5cd-3238-4d6a-848f-3fd1fce9f149","Type":"ContainerStarted","Data":"ec97b235d5610625e5295c9eb7443649a18f30a28400573817f45414c642dfe4"} Sep 17 07:45:22.580632 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:22.580567 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" podStartSLOduration=15.580542484 podStartE2EDuration="15.580542484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:45:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:45:22.579070379 +0000 UTC m=+1044.137255199" watchObservedRunningTime="2026-09-17 07:45:22.580542484 +0000 UTC m=+1044.138727295" Sep 17 07:45:23.546642 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:23.546607 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:45:23.547073 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:23.546673 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:45:23.548379 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:23.548353 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" podUID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.36:6571: connect: connection refused" Sep 17 07:45:23.548464 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:23.548393 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" podUID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerName="online" probeResult="failure" output="Get \"https://10.133.0.36:6567/health\": dial tcp 10.133.0.36:6567: connect: connection refused" Sep 17 07:45:26.547234 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:26.547191 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" podUID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.36:6571: connect: connection refused" Sep 17 07:45:26.547790 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:26.547197 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" podUID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerName="online" probeResult="failure" output="Get \"https://10.133.0.36:6567/health\": dial tcp 10.133.0.36:6567: connect: connection refused" Sep 17 07:45:28.547106 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:28.547066 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:45:28.547480 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:28.547130 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:45:29.547545 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:29.547500 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" podUID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.36:6571: connect: connection refused" Sep 17 07:45:29.556016 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:29.555990 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:45:29.560681 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:29.560657 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:45:32.547263 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:32.547218 2553 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" podUID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.36:6571: connect: connection refused" Sep 17 07:45:35.547146 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:35.547112 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:45:35.547911 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:35.547867 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:45:48.295509 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:45:48.295473 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:46:53.745914 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:53.745857 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:46:55.847020 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:55.846983 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:46:56.245057 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:56.245024 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-757744b5df-mvlhw"] Sep 17 07:46:56.245254 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:56.245214 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-757744b5df-mvlhw" podUID="5b6736ce-f9b4-4f30-823c-701eaf30331d" containerName="manager" containerID="cri-o://4d69cbf52122022c20751cb5e955f04067101c42455db21a00069489410143ce" gracePeriod=10 Sep 17 07:46:56.249370 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:56.249338 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:46:56.482083 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:56.482053 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-757744b5df-mvlhw" Sep 17 07:46:56.555543 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:56.555456 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jswdg\" (UniqueName: \"kubernetes.io/projected/5b6736ce-f9b4-4f30-823c-701eaf30331d-kube-api-access-jswdg\") pod \"5b6736ce-f9b4-4f30-823c-701eaf30331d\" (UID: \"5b6736ce-f9b4-4f30-823c-701eaf30331d\") " Sep 17 07:46:56.557601 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:56.557574 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b6736ce-f9b4-4f30-823c-701eaf30331d-kube-api-access-jswdg" (OuterVolumeSpecName: "kube-api-access-jswdg") pod "5b6736ce-f9b4-4f30-823c-701eaf30331d" (UID: "5b6736ce-f9b4-4f30-823c-701eaf30331d"). InnerVolumeSpecName "kube-api-access-jswdg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:46:56.656108 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:56.656080 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jswdg\" (UniqueName: \"kubernetes.io/projected/5b6736ce-f9b4-4f30-823c-701eaf30331d-kube-api-access-jswdg\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:46:56.851779 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:56.851683 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:46:56.953224 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:56.953195 2553 generic.go:356] "Generic (PLEG): container finished" podID="5b6736ce-f9b4-4f30-823c-701eaf30331d" containerID="4d69cbf52122022c20751cb5e955f04067101c42455db21a00069489410143ce" exitCode=0 Sep 17 07:46:56.953358 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:56.953239 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-757744b5df-mvlhw" Sep 17 07:46:56.953358 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:56.953267 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-757744b5df-mvlhw" event={"ID":"5b6736ce-f9b4-4f30-823c-701eaf30331d","Type":"ContainerDied","Data":"4d69cbf52122022c20751cb5e955f04067101c42455db21a00069489410143ce"} Sep 17 07:46:56.953358 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:56.953316 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-757744b5df-mvlhw" event={"ID":"5b6736ce-f9b4-4f30-823c-701eaf30331d","Type":"ContainerDied","Data":"54cf8d314b1761a28b0e7cfc1cb53e2d970f0e7d99bff6d7cf208e8016fb84da"} Sep 17 07:46:56.953358 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:56.953339 2553 scope.go:117] "RemoveContainer" containerID="4d69cbf52122022c20751cb5e955f04067101c42455db21a00069489410143ce" Sep 17 07:46:56.961401 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:56.961371 2553 scope.go:117] "RemoveContainer" containerID="4d69cbf52122022c20751cb5e955f04067101c42455db21a00069489410143ce" Sep 17 07:46:56.961718 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:46:56.961698 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d69cbf52122022c20751cb5e955f04067101c42455db21a00069489410143ce\": container with ID starting with 4d69cbf52122022c20751cb5e955f04067101c42455db21a00069489410143ce not found: ID does not exist" containerID="4d69cbf52122022c20751cb5e955f04067101c42455db21a00069489410143ce" Sep 17 07:46:56.961769 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:56.961732 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d69cbf52122022c20751cb5e955f04067101c42455db21a00069489410143ce"} err="failed to get container status \"4d69cbf52122022c20751cb5e955f04067101c42455db21a00069489410143ce\": rpc error: code = NotFound desc = could not find container \"4d69cbf52122022c20751cb5e955f04067101c42455db21a00069489410143ce\": container with ID starting with 4d69cbf52122022c20751cb5e955f04067101c42455db21a00069489410143ce not found: ID does not exist" Sep 17 07:46:56.971820 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:56.971803 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:46:56.984020 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:56.983992 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-757744b5df-mvlhw"] Sep 17 07:46:56.991091 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:56.991065 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-757744b5df-mvlhw"] Sep 17 07:46:57.008696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:57.008668 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x"] Sep 17 07:46:57.008925 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:57.008893 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" podUID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerName="registry" containerID="cri-o://085a542f70d8ab99e5c035d82cc13249ec95f55553aeed5bf5bccfb4a93ee3be" gracePeriod=30 Sep 17 07:46:57.009028 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:57.008950 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" podUID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerName="online" containerID="cri-o://76e3db87df313021138dd3aa5569bb461e428dbe95e52141b8d4f7be43c236c1" gracePeriod=30 Sep 17 07:46:57.017740 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:57.017718 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5"] Sep 17 07:46:57.017944 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:57.017918 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" podUID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerName="registry" containerID="cri-o://ec97b235d5610625e5295c9eb7443649a18f30a28400573817f45414c642dfe4" gracePeriod=30 Sep 17 07:46:57.017999 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:57.017949 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" podUID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerName="online" containerID="cri-o://c8f2b022545aa248ca16459ee4f4b60941c9a17c80fee8c90ea0ae3850b76141" gracePeriod=30 Sep 17 07:46:57.960322 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:57.960292 2553 generic.go:356] "Generic (PLEG): container finished" podID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerID="c8f2b022545aa248ca16459ee4f4b60941c9a17c80fee8c90ea0ae3850b76141" exitCode=0 Sep 17 07:46:57.960719 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:57.960370 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" event={"ID":"63e7a5cd-3238-4d6a-848f-3fd1fce9f149","Type":"ContainerDied","Data":"c8f2b022545aa248ca16459ee4f4b60941c9a17c80fee8c90ea0ae3850b76141"} Sep 17 07:46:57.962644 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:57.962623 2553 generic.go:356] "Generic (PLEG): container finished" podID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerID="76e3db87df313021138dd3aa5569bb461e428dbe95e52141b8d4f7be43c236c1" exitCode=0 Sep 17 07:46:57.962748 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:57.962665 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" event={"ID":"25a504f9-ac01-4933-b934-cb9c30167f2b","Type":"ContainerDied","Data":"76e3db87df313021138dd3aa5569bb461e428dbe95e52141b8d4f7be43c236c1"} Sep 17 07:46:58.733716 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:58.733682 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-csz68"] Sep 17 07:46:58.733914 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:58.733849 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-csz68" podUID="68606f55-e7f8-446f-bbf5-2ca1607659b5" containerName="postgres" containerID="cri-o://bdc746c9a2e0c3987dcb9ccde772bd29198acb3b84d84e31368c28bbee2ba178" gracePeriod=30 Sep 17 07:46:58.743297 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:58.743267 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-5f69j"] Sep 17 07:46:58.743488 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:58.743464 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/redis-69bbd798b6-5f69j" podUID="0e93f70f-0817-42c7-89be-b7cff2abff09" containerName="redis" containerID="cri-o://df37daa0a19a2ed2de0c05d0c6ce6da5911349a3bf7d60f993893e09517ce029" gracePeriod=30 Sep 17 07:46:58.812864 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:58.812836 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:46:58.944957 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:58.944931 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b6736ce-f9b4-4f30-823c-701eaf30331d" path="/var/lib/kubelet/pods/5b6736ce-f9b4-4f30-823c-701eaf30331d/volumes" Sep 17 07:46:58.968376 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:58.968342 2553 generic.go:356] "Generic (PLEG): container finished" podID="68606f55-e7f8-446f-bbf5-2ca1607659b5" containerID="bdc746c9a2e0c3987dcb9ccde772bd29198acb3b84d84e31368c28bbee2ba178" exitCode=0 Sep 17 07:46:58.968749 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:58.968386 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-csz68" event={"ID":"68606f55-e7f8-446f-bbf5-2ca1607659b5","Type":"ContainerDied","Data":"bdc746c9a2e0c3987dcb9ccde772bd29198acb3b84d84e31368c28bbee2ba178"} Sep 17 07:46:58.970076 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:58.970052 2553 generic.go:356] "Generic (PLEG): container finished" podID="0e93f70f-0817-42c7-89be-b7cff2abff09" containerID="df37daa0a19a2ed2de0c05d0c6ce6da5911349a3bf7d60f993893e09517ce029" exitCode=0 Sep 17 07:46:58.970188 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:58.970114 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-5f69j" event={"ID":"0e93f70f-0817-42c7-89be-b7cff2abff09","Type":"ContainerDied","Data":"df37daa0a19a2ed2de0c05d0c6ce6da5911349a3bf7d60f993893e09517ce029"} Sep 17 07:46:59.009534 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:59.009507 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-5f69j" Sep 17 07:46:59.011576 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:59.011560 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-csz68" Sep 17 07:46:59.076627 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:59.076584 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f89l4\" (UniqueName: \"kubernetes.io/projected/0e93f70f-0817-42c7-89be-b7cff2abff09-kube-api-access-f89l4\") pod \"0e93f70f-0817-42c7-89be-b7cff2abff09\" (UID: \"0e93f70f-0817-42c7-89be-b7cff2abff09\") " Sep 17 07:46:59.076824 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:59.076642 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/68606f55-e7f8-446f-bbf5-2ca1607659b5-postgresdata\") pod \"68606f55-e7f8-446f-bbf5-2ca1607659b5\" (UID: \"68606f55-e7f8-446f-bbf5-2ca1607659b5\") " Sep 17 07:46:59.076824 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:59.076693 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xx5rj\" (UniqueName: \"kubernetes.io/projected/68606f55-e7f8-446f-bbf5-2ca1607659b5-kube-api-access-xx5rj\") pod \"68606f55-e7f8-446f-bbf5-2ca1607659b5\" (UID: \"68606f55-e7f8-446f-bbf5-2ca1607659b5\") " Sep 17 07:46:59.079412 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:59.079381 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e93f70f-0817-42c7-89be-b7cff2abff09-kube-api-access-f89l4" (OuterVolumeSpecName: "kube-api-access-f89l4") pod "0e93f70f-0817-42c7-89be-b7cff2abff09" (UID: "0e93f70f-0817-42c7-89be-b7cff2abff09"). InnerVolumeSpecName "kube-api-access-f89l4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:46:59.079412 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:59.079381 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68606f55-e7f8-446f-bbf5-2ca1607659b5-kube-api-access-xx5rj" (OuterVolumeSpecName: "kube-api-access-xx5rj") pod "68606f55-e7f8-446f-bbf5-2ca1607659b5" (UID: "68606f55-e7f8-446f-bbf5-2ca1607659b5"). InnerVolumeSpecName "kube-api-access-xx5rj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:46:59.111132 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:59.111098 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68606f55-e7f8-446f-bbf5-2ca1607659b5-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "68606f55-e7f8-446f-bbf5-2ca1607659b5" (UID: "68606f55-e7f8-446f-bbf5-2ca1607659b5"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:46:59.178230 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:59.178201 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f89l4\" (UniqueName: \"kubernetes.io/projected/0e93f70f-0817-42c7-89be-b7cff2abff09-kube-api-access-f89l4\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:46:59.178230 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:59.178224 2553 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/68606f55-e7f8-446f-bbf5-2ca1607659b5-postgresdata\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:46:59.178230 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:59.178233 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xx5rj\" (UniqueName: \"kubernetes.io/projected/68606f55-e7f8-446f-bbf5-2ca1607659b5-kube-api-access-xx5rj\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:46:59.557291 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:59.557252 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" podUID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerName="online" probeResult="failure" output="Get \"https://10.133.0.36:6567/health\": dial tcp 10.133.0.36:6567: connect: connection refused" Sep 17 07:46:59.975115 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:59.975082 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-5f69j" Sep 17 07:46:59.975115 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:59.975102 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-5f69j" event={"ID":"0e93f70f-0817-42c7-89be-b7cff2abff09","Type":"ContainerDied","Data":"4c09c3d13b34ff05838dc374e26aa2f8748e8081e8f94e5a54f5fb7a21850de2"} Sep 17 07:46:59.975619 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:59.975153 2553 scope.go:117] "RemoveContainer" containerID="df37daa0a19a2ed2de0c05d0c6ce6da5911349a3bf7d60f993893e09517ce029" Sep 17 07:46:59.976508 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:59.976494 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-csz68" Sep 17 07:46:59.976602 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:59.976492 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-csz68" event={"ID":"68606f55-e7f8-446f-bbf5-2ca1607659b5","Type":"ContainerDied","Data":"13423aa817a98254fe06fa125d1f334194f72c3382c5ddc5d9a04bc9bae91284"} Sep 17 07:46:59.984085 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:46:59.984063 2553 scope.go:117] "RemoveContainer" containerID="bdc746c9a2e0c3987dcb9ccde772bd29198acb3b84d84e31368c28bbee2ba178" Sep 17 07:47:00.008183 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:00.008161 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-csz68"] Sep 17 07:47:00.014705 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:00.014687 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-csz68"] Sep 17 07:47:00.035327 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:00.035307 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-5f69j"] Sep 17 07:47:00.039547 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:00.039526 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-5f69j"] Sep 17 07:47:00.152115 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:00.152083 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" podUID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerName="online" probeResult="failure" output="Get \"https://10.133.0.35:6567/health\": dial tcp 10.133.0.35:6567: connect: connection refused" Sep 17 07:47:00.933182 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:00.933142 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e93f70f-0817-42c7-89be-b7cff2abff09" path="/var/lib/kubelet/pods/0e93f70f-0817-42c7-89be-b7cff2abff09/volumes" Sep 17 07:47:00.933571 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:00.933551 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68606f55-e7f8-446f-bbf5-2ca1607659b5" path="/var/lib/kubelet/pods/68606f55-e7f8-446f-bbf5-2ca1607659b5/volumes" Sep 17 07:47:02.075768 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:02.075738 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:47:02.147993 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:02.147960 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:47:08.169221 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:08.169184 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:47:08.273408 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:08.273376 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:47:08.805977 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:08.805944 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:47:09.557279 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:09.557237 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" podUID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerName="online" probeResult="failure" output="Get \"https://10.133.0.36:6567/health\": dial tcp 10.133.0.36:6567: connect: connection refused" Sep 17 07:47:10.151974 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:10.151928 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" podUID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerName="online" probeResult="failure" output="Get \"https://10.133.0.35:6567/health\": dial tcp 10.133.0.35:6567: connect: connection refused" Sep 17 07:47:19.557292 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:19.557247 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" podUID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerName="online" probeResult="failure" output="Get \"https://10.133.0.36:6567/health\": dial tcp 10.133.0.36:6567: connect: connection refused" Sep 17 07:47:19.557696 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:19.557359 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:47:20.152168 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:20.152122 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" podUID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerName="online" probeResult="failure" output="Get \"https://10.133.0.35:6567/health\": dial tcp 10.133.0.35:6567: connect: connection refused" Sep 17 07:47:20.152347 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:20.152241 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:47:25.048094 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:25.047990 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:47:25.136502 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:25.136449 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:47:25.689854 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:25.689821 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:47:25.789582 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:25.789527 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:47:25.865088 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:25.865062 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:47:25.934187 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:25.934137 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:47:25.994390 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:25.994366 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:47:27.092674 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.092647 2553 generic.go:356] "Generic (PLEG): container finished" podID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerID="085a542f70d8ab99e5c035d82cc13249ec95f55553aeed5bf5bccfb4a93ee3be" exitCode=137 Sep 17 07:47:27.093075 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.092725 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" event={"ID":"25a504f9-ac01-4933-b934-cb9c30167f2b","Type":"ContainerDied","Data":"085a542f70d8ab99e5c035d82cc13249ec95f55553aeed5bf5bccfb4a93ee3be"} Sep 17 07:47:27.095237 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.095216 2553 generic.go:356] "Generic (PLEG): container finished" podID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerID="ec97b235d5610625e5295c9eb7443649a18f30a28400573817f45414c642dfe4" exitCode=137 Sep 17 07:47:27.095331 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.095266 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" event={"ID":"63e7a5cd-3238-4d6a-848f-3fd1fce9f149","Type":"ContainerDied","Data":"ec97b235d5610625e5295c9eb7443649a18f30a28400573817f45414c642dfe4"} Sep 17 07:47:27.674541 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.674519 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:47:27.677147 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.677130 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:47:27.698028 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.698010 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/25a504f9-ac01-4933-b934-cb9c30167f2b-registry-tls\") pod \"25a504f9-ac01-4933-b934-cb9c30167f2b\" (UID: \"25a504f9-ac01-4933-b934-cb9c30167f2b\") " Sep 17 07:47:27.698132 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.698044 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/25a504f9-ac01-4933-b934-cb9c30167f2b-online-tls\") pod \"25a504f9-ac01-4933-b934-cb9c30167f2b\" (UID: \"25a504f9-ac01-4933-b934-cb9c30167f2b\") " Sep 17 07:47:27.698132 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.698091 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ch2xr\" (UniqueName: \"kubernetes.io/projected/25a504f9-ac01-4933-b934-cb9c30167f2b-kube-api-access-ch2xr\") pod \"25a504f9-ac01-4933-b934-cb9c30167f2b\" (UID: \"25a504f9-ac01-4933-b934-cb9c30167f2b\") " Sep 17 07:47:27.698244 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.698143 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-registry-tls\") pod \"63e7a5cd-3238-4d6a-848f-3fd1fce9f149\" (UID: \"63e7a5cd-3238-4d6a-848f-3fd1fce9f149\") " Sep 17 07:47:27.698244 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.698183 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-online-tls\") pod \"63e7a5cd-3238-4d6a-848f-3fd1fce9f149\" (UID: \"63e7a5cd-3238-4d6a-848f-3fd1fce9f149\") " Sep 17 07:47:27.698353 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.698249 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/25a504f9-ac01-4933-b934-cb9c30167f2b-feast-data\") pod \"25a504f9-ac01-4933-b934-cb9c30167f2b\" (UID: \"25a504f9-ac01-4933-b934-cb9c30167f2b\") " Sep 17 07:47:27.698353 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.698290 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4qtx\" (UniqueName: \"kubernetes.io/projected/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-kube-api-access-c4qtx\") pod \"63e7a5cd-3238-4d6a-848f-3fd1fce9f149\" (UID: \"63e7a5cd-3238-4d6a-848f-3fd1fce9f149\") " Sep 17 07:47:27.698353 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.698316 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-feast-data\") pod \"63e7a5cd-3238-4d6a-848f-3fd1fce9f149\" (UID: \"63e7a5cd-3238-4d6a-848f-3fd1fce9f149\") " Sep 17 07:47:27.698921 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.698900 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-feast-data" (OuterVolumeSpecName: "feast-data") pod "63e7a5cd-3238-4d6a-848f-3fd1fce9f149" (UID: "63e7a5cd-3238-4d6a-848f-3fd1fce9f149"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:47:27.700531 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.700490 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-online-tls" (OuterVolumeSpecName: "online-tls") pod "63e7a5cd-3238-4d6a-848f-3fd1fce9f149" (UID: "63e7a5cd-3238-4d6a-848f-3fd1fce9f149"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:47:27.700619 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.700582 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25a504f9-ac01-4933-b934-cb9c30167f2b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "25a504f9-ac01-4933-b934-cb9c30167f2b" (UID: "25a504f9-ac01-4933-b934-cb9c30167f2b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:47:27.700619 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.700590 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25a504f9-ac01-4933-b934-cb9c30167f2b-online-tls" (OuterVolumeSpecName: "online-tls") pod "25a504f9-ac01-4933-b934-cb9c30167f2b" (UID: "25a504f9-ac01-4933-b934-cb9c30167f2b"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:47:27.700701 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.700662 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25a504f9-ac01-4933-b934-cb9c30167f2b-kube-api-access-ch2xr" (OuterVolumeSpecName: "kube-api-access-ch2xr") pod "25a504f9-ac01-4933-b934-cb9c30167f2b" (UID: "25a504f9-ac01-4933-b934-cb9c30167f2b"). InnerVolumeSpecName "kube-api-access-ch2xr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:47:27.700803 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.700768 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "63e7a5cd-3238-4d6a-848f-3fd1fce9f149" (UID: "63e7a5cd-3238-4d6a-848f-3fd1fce9f149"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:47:27.701125 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.701106 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-kube-api-access-c4qtx" (OuterVolumeSpecName: "kube-api-access-c4qtx") pod "63e7a5cd-3238-4d6a-848f-3fd1fce9f149" (UID: "63e7a5cd-3238-4d6a-848f-3fd1fce9f149"). InnerVolumeSpecName "kube-api-access-c4qtx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:47:27.708440 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.708416 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25a504f9-ac01-4933-b934-cb9c30167f2b-feast-data" (OuterVolumeSpecName: "feast-data") pod "25a504f9-ac01-4933-b934-cb9c30167f2b" (UID: "25a504f9-ac01-4933-b934-cb9c30167f2b"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:47:27.799064 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.799001 2553 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/25a504f9-ac01-4933-b934-cb9c30167f2b-feast-data\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:47:27.799064 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.799026 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4qtx\" (UniqueName: \"kubernetes.io/projected/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-kube-api-access-c4qtx\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:47:27.799064 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.799040 2553 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-feast-data\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:47:27.799064 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.799052 2553 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/25a504f9-ac01-4933-b934-cb9c30167f2b-registry-tls\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:47:27.799064 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.799063 2553 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/25a504f9-ac01-4933-b934-cb9c30167f2b-online-tls\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:47:27.799288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.799075 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ch2xr\" (UniqueName: \"kubernetes.io/projected/25a504f9-ac01-4933-b934-cb9c30167f2b-kube-api-access-ch2xr\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:47:27.799288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.799087 2553 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-registry-tls\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:47:27.799288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:27.799097 2553 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/63e7a5cd-3238-4d6a-848f-3fd1fce9f149-online-tls\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:47:28.101713 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:28.101648 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" Sep 17 07:47:28.101713 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:28.101674 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x" event={"ID":"25a504f9-ac01-4933-b934-cb9c30167f2b","Type":"ContainerDied","Data":"fe82759a85c65c2a6c17e66e85011d21ae11a76cc6bf25f8415227e049d784b3"} Sep 17 07:47:28.102188 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:28.101734 2553 scope.go:117] "RemoveContainer" containerID="76e3db87df313021138dd3aa5569bb461e428dbe95e52141b8d4f7be43c236c1" Sep 17 07:47:28.104249 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:28.104229 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" event={"ID":"63e7a5cd-3238-4d6a-848f-3fd1fce9f149","Type":"ContainerDied","Data":"8e48e85bde3d376c8071881f819ba74d7c22f9e86f340820469d61e065c50f52"} Sep 17 07:47:28.104345 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:28.104282 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5" Sep 17 07:47:28.111952 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:28.111933 2553 scope.go:117] "RemoveContainer" containerID="085a542f70d8ab99e5c035d82cc13249ec95f55553aeed5bf5bccfb4a93ee3be" Sep 17 07:47:28.119684 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:28.119665 2553 scope.go:117] "RemoveContainer" containerID="6f4efbe03b075eb133bff84697ae28760982876f744019283c8f3da6611492f7" Sep 17 07:47:28.126697 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:28.126675 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x"] Sep 17 07:47:28.127292 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:28.127277 2553 scope.go:117] "RemoveContainer" containerID="8c4afa3f9ff23aecf2f33cb241011ddbe567e8ca21b716ed8a4f68852f2daf2a" Sep 17 07:47:28.135778 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:28.134044 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-7d499b5545-vmk6x"] Sep 17 07:47:28.136088 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:28.136068 2553 scope.go:117] "RemoveContainer" containerID="c8f2b022545aa248ca16459ee4f4b60941c9a17c80fee8c90ea0ae3850b76141" Sep 17 07:47:28.143444 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:28.143429 2553 scope.go:117] "RemoveContainer" containerID="ec97b235d5610625e5295c9eb7443649a18f30a28400573817f45414c642dfe4" Sep 17 07:47:28.146805 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:28.146783 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5"] Sep 17 07:47:28.150910 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:28.150868 2553 scope.go:117] "RemoveContainer" containerID="b918ca7415def1426710683d75c32a39e8124ec68463f370eb0586789071efdd" Sep 17 07:47:28.151650 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:28.151633 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-85f79c89d-cb5x5"] Sep 17 07:47:28.157506 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:28.157490 2553 scope.go:117] "RemoveContainer" containerID="41776bc0d9c647c6929a2c307565d7fdb2fa2ec11477082f923f0109fd32aa58" Sep 17 07:47:28.938291 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:28.938259 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25a504f9-ac01-4933-b934-cb9c30167f2b" path="/var/lib/kubelet/pods/25a504f9-ac01-4933-b934-cb9c30167f2b/volumes" Sep 17 07:47:28.938739 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:28.938726 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" path="/var/lib/kubelet/pods/63e7a5cd-3238-4d6a-848f-3fd1fce9f149/volumes" Sep 17 07:47:36.680278 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:36.680240 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:47:36.765452 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:36.765424 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:47:37.130469 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:37.130433 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:47:41.624033 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:41.623995 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:47:58.899828 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:58.899798 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-5kks2_2455e65f-98bb-463d-a204-5dbd5d69b5f3/console-operator/1.log" Sep 17 07:47:58.901118 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:58.901099 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-5kks2_2455e65f-98bb-463d-a204-5dbd5d69b5f3/console-operator/1.log" Sep 17 07:47:58.906990 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:58.906973 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/ovn-acl-logging/0.log" Sep 17 07:47:58.907954 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:47:58.907939 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/ovn-acl-logging/0.log" Sep 17 07:48:21.918478 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:48:21.918441 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:49:04.135454 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:49:04.135413 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:49:04.235554 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:49:04.235524 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:49:04.415762 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:49:04.415686 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:49:04.516097 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:49:04.516065 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:49:04.576894 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:49:04.576841 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:49:04.783370 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:49:04.783336 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:49:04.883999 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:49:04.883966 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:49:05.060007 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:49:05.059937 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:50:31.138571 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:50:31.138492 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:50:32.863425 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:50:32.863397 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:51:30.823468 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:51:30.823431 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:52:24.429125 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:52:24.429092 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:52:24.487405 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:52:24.487375 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:52:28.114102 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:52:28.114071 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:52:28.185645 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:52:28.185608 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:52:28.306179 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:52:28.306146 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:52:28.401575 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:52:28.401494 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:52:28.457453 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:52:28.457407 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:52:33.476547 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:52:33.476510 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:52:58.923288 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:52:58.923258 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-5kks2_2455e65f-98bb-463d-a204-5dbd5d69b5f3/console-operator/1.log" Sep 17 07:52:58.925475 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:52:58.925457 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-5kks2_2455e65f-98bb-463d-a204-5dbd5d69b5f3/console-operator/1.log" Sep 17 07:52:58.930330 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:52:58.930310 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/ovn-acl-logging/0.log" Sep 17 07:52:58.933318 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:52:58.933300 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/ovn-acl-logging/0.log" Sep 17 07:53:12.242926 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:53:12.242893 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:53:12.305006 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:53:12.304971 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:53:17.454676 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:53:17.454643 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:53:17.509663 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:53:17.509638 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:53:38.132926 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:53:38.132817 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:53:38.232630 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:53:38.232600 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:53:40.364384 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:53:40.364353 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:53:40.465519 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:53:40.465476 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:53:40.565189 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:53:40.565156 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:53:40.676314 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:53:40.676229 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:53:40.877399 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:53:40.877361 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:53:40.995254 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:53:40.995219 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:55:05.149211 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:55:05.149137 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:55:05.239051 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:55:05.239021 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:55:05.569854 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:55:05.569824 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:55:05.667091 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:55:05.667039 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:55:05.740047 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:55:05.740020 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:55:05.824416 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:55:05.824326 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:55:05.926957 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:55:05.926934 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:55:06.026134 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:55:06.026108 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:56:31.771684 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:56:31.771609 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:56:32.957232 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:56:32.957198 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:57:58.946733 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:57:58.946706 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-5kks2_2455e65f-98bb-463d-a204-5dbd5d69b5f3/console-operator/1.log" Sep 17 07:57:58.951255 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:57:58.951236 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-5kks2_2455e65f-98bb-463d-a204-5dbd5d69b5f3/console-operator/1.log" Sep 17 07:57:58.954178 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:57:58.954158 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/ovn-acl-logging/0.log" Sep 17 07:57:58.958257 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:57:58.958241 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/ovn-acl-logging/0.log" Sep 17 07:58:32.412818 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:58:32.412772 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:58:32.467835 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:58:32.467807 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:58:32.579756 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:58:32.579725 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:58:32.678240 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:58:32.678158 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:58:32.732450 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:58:32.732419 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:58:37.771719 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:58:37.771676 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:58:55.609682 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:58:55.609639 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:58:55.670839 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:58:55.670803 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:59:12.366659 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:12.366612 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:59:12.480029 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:12.479997 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:59:18.180924 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.180866 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:59:18.281063 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.281030 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:59:18.631735 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.631708 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-k9pbm/must-gather-sg9c2"] Sep 17 07:59:18.632024 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632010 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerName="online" Sep 17 07:59:18.632075 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632026 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerName="online" Sep 17 07:59:18.632075 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632038 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerName="feast-init" Sep 17 07:59:18.632075 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632044 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerName="feast-init" Sep 17 07:59:18.632075 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632058 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerName="feast-apply" Sep 17 07:59:18.632075 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632064 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerName="feast-apply" Sep 17 07:59:18.632075 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632070 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e93f70f-0817-42c7-89be-b7cff2abff09" containerName="redis" Sep 17 07:59:18.632075 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632076 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e93f70f-0817-42c7-89be-b7cff2abff09" containerName="redis" Sep 17 07:59:18.632289 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632082 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerName="registry" Sep 17 07:59:18.632289 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632087 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerName="registry" Sep 17 07:59:18.632289 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632096 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerName="feast-init" Sep 17 07:59:18.632289 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632101 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerName="feast-init" Sep 17 07:59:18.632289 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632109 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68606f55-e7f8-446f-bbf5-2ca1607659b5" containerName="postgres" Sep 17 07:59:18.632289 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632115 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="68606f55-e7f8-446f-bbf5-2ca1607659b5" containerName="postgres" Sep 17 07:59:18.632289 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632120 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerName="feast-apply" Sep 17 07:59:18.632289 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632128 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerName="feast-apply" Sep 17 07:59:18.632289 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632138 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerName="online" Sep 17 07:59:18.632289 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632143 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerName="online" Sep 17 07:59:18.632289 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632149 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b6736ce-f9b4-4f30-823c-701eaf30331d" containerName="manager" Sep 17 07:59:18.632289 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632154 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b6736ce-f9b4-4f30-823c-701eaf30331d" containerName="manager" Sep 17 07:59:18.632289 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632160 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerName="registry" Sep 17 07:59:18.632289 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632166 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerName="registry" Sep 17 07:59:18.632289 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632213 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e93f70f-0817-42c7-89be-b7cff2abff09" containerName="redis" Sep 17 07:59:18.632289 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632221 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b6736ce-f9b4-4f30-823c-701eaf30331d" containerName="manager" Sep 17 07:59:18.632289 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632227 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerName="registry" Sep 17 07:59:18.632289 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632233 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="68606f55-e7f8-446f-bbf5-2ca1607659b5" containerName="postgres" Sep 17 07:59:18.632289 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632239 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="63e7a5cd-3238-4d6a-848f-3fd1fce9f149" containerName="online" Sep 17 07:59:18.632289 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632244 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerName="online" Sep 17 07:59:18.632289 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.632250 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="25a504f9-ac01-4933-b934-cb9c30167f2b" containerName="registry" Sep 17 07:59:18.634383 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.634360 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:59:18.635170 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.635155 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k9pbm/must-gather-sg9c2" Sep 17 07:59:18.637734 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.637715 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k9pbm\"/\"openshift-service-ca.crt\"" Sep 17 07:59:18.638704 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.638687 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k9pbm\"/\"kube-root-ca.crt\"" Sep 17 07:59:18.638801 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.638748 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-k9pbm\"/\"default-dockercfg-mpdkb\"" Sep 17 07:59:18.643665 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.643639 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k9pbm/must-gather-sg9c2"] Sep 17 07:59:18.648476 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.648453 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2vnt\" (UniqueName: \"kubernetes.io/projected/79bf3f73-80e2-45e7-8f24-0652def7e9f9-kube-api-access-b2vnt\") pod \"must-gather-sg9c2\" (UID: \"79bf3f73-80e2-45e7-8f24-0652def7e9f9\") " pod="openshift-must-gather-k9pbm/must-gather-sg9c2" Sep 17 07:59:18.648559 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.648509 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/79bf3f73-80e2-45e7-8f24-0652def7e9f9-must-gather-output\") pod \"must-gather-sg9c2\" (UID: \"79bf3f73-80e2-45e7-8f24-0652def7e9f9\") " pod="openshift-must-gather-k9pbm/must-gather-sg9c2" Sep 17 07:59:18.695684 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.695663 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:59:18.749188 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.749166 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2vnt\" (UniqueName: \"kubernetes.io/projected/79bf3f73-80e2-45e7-8f24-0652def7e9f9-kube-api-access-b2vnt\") pod \"must-gather-sg9c2\" (UID: \"79bf3f73-80e2-45e7-8f24-0652def7e9f9\") " pod="openshift-must-gather-k9pbm/must-gather-sg9c2" Sep 17 07:59:18.749275 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.749194 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/79bf3f73-80e2-45e7-8f24-0652def7e9f9-must-gather-output\") pod \"must-gather-sg9c2\" (UID: \"79bf3f73-80e2-45e7-8f24-0652def7e9f9\") " pod="openshift-must-gather-k9pbm/must-gather-sg9c2" Sep 17 07:59:18.749449 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.749436 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/79bf3f73-80e2-45e7-8f24-0652def7e9f9-must-gather-output\") pod \"must-gather-sg9c2\" (UID: \"79bf3f73-80e2-45e7-8f24-0652def7e9f9\") " pod="openshift-must-gather-k9pbm/must-gather-sg9c2" Sep 17 07:59:18.757465 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.757441 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2vnt\" (UniqueName: \"kubernetes.io/projected/79bf3f73-80e2-45e7-8f24-0652def7e9f9-kube-api-access-b2vnt\") pod \"must-gather-sg9c2\" (UID: \"79bf3f73-80e2-45e7-8f24-0652def7e9f9\") " pod="openshift-must-gather-k9pbm/must-gather-sg9c2" Sep 17 07:59:18.954960 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:18.954944 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k9pbm/must-gather-sg9c2" Sep 17 07:59:19.227533 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:19.227508 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k9pbm/must-gather-sg9c2"] Sep 17 07:59:19.229825 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:59:19.229800 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79bf3f73_80e2_45e7_8f24_0652def7e9f9.slice/crio-3dba8441b95b0833009a94ea8b6a236b48020d53252c804d0b4e94f608d87b44 WatchSource:0}: Error finding container 3dba8441b95b0833009a94ea8b6a236b48020d53252c804d0b4e94f608d87b44: Status 404 returned error can't find the container with id 3dba8441b95b0833009a94ea8b6a236b48020d53252c804d0b4e94f608d87b44 Sep 17 07:59:19.231537 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:19.231519 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 07:59:19.816321 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:19.816279 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k9pbm/must-gather-sg9c2" event={"ID":"79bf3f73-80e2-45e7-8f24-0652def7e9f9","Type":"ContainerStarted","Data":"3dba8441b95b0833009a94ea8b6a236b48020d53252c804d0b4e94f608d87b44"} Sep 17 07:59:25.844408 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:25.844370 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k9pbm/must-gather-sg9c2" event={"ID":"79bf3f73-80e2-45e7-8f24-0652def7e9f9","Type":"ContainerStarted","Data":"5d1e16034ca145a5aa8d944d656b782612fe765cb916ed14b0137c0bbd470c25"} Sep 17 07:59:25.844795 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:25.844423 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k9pbm/must-gather-sg9c2" event={"ID":"79bf3f73-80e2-45e7-8f24-0652def7e9f9","Type":"ContainerStarted","Data":"f928deb3365c80c74c75b315d2cb26e9f395a61f6ba53741513be9e3ddfdfe8a"} Sep 17 07:59:25.862352 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:25.862277 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-k9pbm/must-gather-sg9c2" podStartSLOduration=1.912351583 podStartE2EDuration="7.862259907s" totalImagesPullingTime="5.949908324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:59:25.861910953 +0000 UTC m=+1887.420095758" watchObservedRunningTime="2026-09-17 07:59:25.862259907 +0000 UTC m=+1887.420444733" Sep 17 07:59:29.328202 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:29.328165 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:59:34.883095 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:34.883007 2553 generic.go:356] "Generic (PLEG): container finished" podID="79bf3f73-80e2-45e7-8f24-0652def7e9f9" containerID="f928deb3365c80c74c75b315d2cb26e9f395a61f6ba53741513be9e3ddfdfe8a" exitCode=0 Sep 17 07:59:34.883095 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:34.883083 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k9pbm/must-gather-sg9c2" event={"ID":"79bf3f73-80e2-45e7-8f24-0652def7e9f9","Type":"ContainerDied","Data":"f928deb3365c80c74c75b315d2cb26e9f395a61f6ba53741513be9e3ddfdfe8a"} Sep 17 07:59:34.883506 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:34.883396 2553 scope.go:117] "RemoveContainer" containerID="f928deb3365c80c74c75b315d2cb26e9f395a61f6ba53741513be9e3ddfdfe8a" Sep 17 07:59:35.629991 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:35.629964 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-k9pbm_must-gather-sg9c2_79bf3f73-80e2-45e7-8f24-0652def7e9f9/gather/0.log" Sep 17 07:59:35.977657 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:35.977624 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:59:36.167095 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:36.167066 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:59:36.240639 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:36.240576 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:59:38.851195 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:38.851166 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gh9zp_f68f4c1a-7946-43c1-80ca-b472129b2bfe/global-pull-secret-syncer/0.log" Sep 17 07:59:39.034646 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:39.034623 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" Sep 17 07:59:39.177328 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:39.177265 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-54.ec2.internal_ea85235c507e0a22eb3b6e21763019df/haproxy/0.log" Sep 17 07:59:40.961875 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:40.961845 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-k9pbm/must-gather-sg9c2"] Sep 17 07:59:40.962284 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:40.962005 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-k9pbm/must-gather-sg9c2" podUID="79bf3f73-80e2-45e7-8f24-0652def7e9f9" containerName="copy" containerID="cri-o://5d1e16034ca145a5aa8d944d656b782612fe765cb916ed14b0137c0bbd470c25" gracePeriod=2 Sep 17 07:59:40.964490 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:59:40.964461 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-sg9c2\" is forbidden: User \"system:node:ip-10-0-141-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-k9pbm\": no relationship found between node 'ip-10-0-141-54.ec2.internal' and this object" podUID="79bf3f73-80e2-45e7-8f24-0652def7e9f9" pod="openshift-must-gather-k9pbm/must-gather-sg9c2" Sep 17 07:59:40.965459 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:40.965435 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-k9pbm/must-gather-sg9c2"] Sep 17 07:59:41.181694 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:41.181676 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-k9pbm_must-gather-sg9c2_79bf3f73-80e2-45e7-8f24-0652def7e9f9/copy/0.log" Sep 17 07:59:41.183179 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:41.183163 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k9pbm/must-gather-sg9c2" Sep 17 07:59:41.185428 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:59:41.185405 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-sg9c2\" is forbidden: User \"system:node:ip-10-0-141-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-k9pbm\": no relationship found between node 'ip-10-0-141-54.ec2.internal' and this object" podUID="79bf3f73-80e2-45e7-8f24-0652def7e9f9" pod="openshift-must-gather-k9pbm/must-gather-sg9c2" Sep 17 07:59:41.220743 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:41.220703 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2vnt\" (UniqueName: \"kubernetes.io/projected/79bf3f73-80e2-45e7-8f24-0652def7e9f9-kube-api-access-b2vnt\") pod \"79bf3f73-80e2-45e7-8f24-0652def7e9f9\" (UID: \"79bf3f73-80e2-45e7-8f24-0652def7e9f9\") " Sep 17 07:59:41.220824 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:41.220764 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/79bf3f73-80e2-45e7-8f24-0652def7e9f9-must-gather-output\") pod \"79bf3f73-80e2-45e7-8f24-0652def7e9f9\" (UID: \"79bf3f73-80e2-45e7-8f24-0652def7e9f9\") " Sep 17 07:59:41.221147 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:41.221127 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79bf3f73-80e2-45e7-8f24-0652def7e9f9-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "79bf3f73-80e2-45e7-8f24-0652def7e9f9" (UID: "79bf3f73-80e2-45e7-8f24-0652def7e9f9"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:59:41.222770 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:41.222742 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79bf3f73-80e2-45e7-8f24-0652def7e9f9-kube-api-access-b2vnt" (OuterVolumeSpecName: "kube-api-access-b2vnt") pod "79bf3f73-80e2-45e7-8f24-0652def7e9f9" (UID: "79bf3f73-80e2-45e7-8f24-0652def7e9f9"). InnerVolumeSpecName "kube-api-access-b2vnt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:59:41.321670 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:41.321648 2553 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/79bf3f73-80e2-45e7-8f24-0652def7e9f9-must-gather-output\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:59:41.321670 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:41.321667 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2vnt\" (UniqueName: \"kubernetes.io/projected/79bf3f73-80e2-45e7-8f24-0652def7e9f9-kube-api-access-b2vnt\") on node \"ip-10-0-141-54.ec2.internal\" DevicePath \"\"" Sep 17 07:59:41.833919 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:41.833869 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:59:41.893920 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:41.893896 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-crmdq_5bea09db-e130-47db-83df-6467274459a1/konnectivity-agent/0.log" limit=1024 Sep 17 07:59:41.908136 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:41.908122 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-k9pbm_must-gather-sg9c2_79bf3f73-80e2-45e7-8f24-0652def7e9f9/copy/0.log" Sep 17 07:59:41.908475 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:41.908458 2553 generic.go:356] "Generic (PLEG): container finished" podID="79bf3f73-80e2-45e7-8f24-0652def7e9f9" containerID="5d1e16034ca145a5aa8d944d656b782612fe765cb916ed14b0137c0bbd470c25" exitCode=143 Sep 17 07:59:41.908548 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:41.908497 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k9pbm/must-gather-sg9c2" Sep 17 07:59:41.908587 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:41.908550 2553 scope.go:117] "RemoveContainer" containerID="5d1e16034ca145a5aa8d944d656b782612fe765cb916ed14b0137c0bbd470c25" Sep 17 07:59:41.911035 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:59:41.911014 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-sg9c2\" is forbidden: User \"system:node:ip-10-0-141-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-k9pbm\": no relationship found between node 'ip-10-0-141-54.ec2.internal' and this object" podUID="79bf3f73-80e2-45e7-8f24-0652def7e9f9" pod="openshift-must-gather-k9pbm/must-gather-sg9c2" Sep 17 07:59:41.916042 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:41.916025 2553 scope.go:117] "RemoveContainer" containerID="f928deb3365c80c74c75b315d2cb26e9f395a61f6ba53741513be9e3ddfdfe8a" Sep 17 07:59:41.918610 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:59:41.918582 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-sg9c2\" is forbidden: User \"system:node:ip-10-0-141-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-k9pbm\": no relationship found between node 'ip-10-0-141-54.ec2.internal' and this object" podUID="79bf3f73-80e2-45e7-8f24-0652def7e9f9" pod="openshift-must-gather-k9pbm/must-gather-sg9c2" Sep 17 07:59:41.926599 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:41.926583 2553 scope.go:117] "RemoveContainer" containerID="5d1e16034ca145a5aa8d944d656b782612fe765cb916ed14b0137c0bbd470c25" Sep 17 07:59:41.926836 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:59:41.926820 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d1e16034ca145a5aa8d944d656b782612fe765cb916ed14b0137c0bbd470c25\": container with ID starting with 5d1e16034ca145a5aa8d944d656b782612fe765cb916ed14b0137c0bbd470c25 not found: ID does not exist" containerID="5d1e16034ca145a5aa8d944d656b782612fe765cb916ed14b0137c0bbd470c25" Sep 17 07:59:41.926893 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:41.926843 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d1e16034ca145a5aa8d944d656b782612fe765cb916ed14b0137c0bbd470c25"} err="failed to get container status \"5d1e16034ca145a5aa8d944d656b782612fe765cb916ed14b0137c0bbd470c25\": rpc error: code = NotFound desc = could not find container \"5d1e16034ca145a5aa8d944d656b782612fe765cb916ed14b0137c0bbd470c25\": container with ID starting with 5d1e16034ca145a5aa8d944d656b782612fe765cb916ed14b0137c0bbd470c25 not found: ID does not exist" Sep 17 07:59:41.926893 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:41.926859 2553 scope.go:117] "RemoveContainer" containerID="f928deb3365c80c74c75b315d2cb26e9f395a61f6ba53741513be9e3ddfdfe8a" Sep 17 07:59:41.927088 ip-10-0-141-54 kubenswrapper[2553]: E0917 07:59:41.927072 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f928deb3365c80c74c75b315d2cb26e9f395a61f6ba53741513be9e3ddfdfe8a\": container with ID starting with f928deb3365c80c74c75b315d2cb26e9f395a61f6ba53741513be9e3ddfdfe8a not found: ID does not exist" containerID="f928deb3365c80c74c75b315d2cb26e9f395a61f6ba53741513be9e3ddfdfe8a" Sep 17 07:59:41.927128 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:41.927094 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f928deb3365c80c74c75b315d2cb26e9f395a61f6ba53741513be9e3ddfdfe8a"} err="failed to get container status \"f928deb3365c80c74c75b315d2cb26e9f395a61f6ba53741513be9e3ddfdfe8a\": rpc error: code = NotFound desc = could not find container \"f928deb3365c80c74c75b315d2cb26e9f395a61f6ba53741513be9e3ddfdfe8a\": container with ID starting with f928deb3365c80c74c75b315d2cb26e9f395a61f6ba53741513be9e3ddfdfe8a not found: ID does not exist" Sep 17 07:59:42.933849 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:42.933820 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79bf3f73-80e2-45e7-8f24-0652def7e9f9" path="/var/lib/kubelet/pods/79bf3f73-80e2-45e7-8f24-0652def7e9f9/volumes" Sep 17 07:59:42.986488 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:42.986468 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f749cdd2-2dc4-475f-be93-a7dc4d12bfc0/alertmanager/0.log" Sep 17 07:59:43.012670 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:43.012654 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f749cdd2-2dc4-475f-be93-a7dc4d12bfc0/config-reloader/0.log" Sep 17 07:59:43.049438 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:43.049416 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f749cdd2-2dc4-475f-be93-a7dc4d12bfc0/kube-rbac-proxy-web/0.log" Sep 17 07:59:43.075077 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:43.075058 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f749cdd2-2dc4-475f-be93-a7dc4d12bfc0/kube-rbac-proxy/0.log" Sep 17 07:59:43.101801 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:43.101785 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f749cdd2-2dc4-475f-be93-a7dc4d12bfc0/kube-rbac-proxy-metric/0.log" Sep 17 07:59:43.124129 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:43.124114 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f749cdd2-2dc4-475f-be93-a7dc4d12bfc0/prom-label-proxy/0.log" Sep 17 07:59:43.152538 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:43.152517 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f749cdd2-2dc4-475f-be93-a7dc4d12bfc0/init-config-reloader/0.log" Sep 17 07:59:43.192384 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:43.192328 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-6tzzh_7c06b4ad-93ab-43cc-a24a-9c758f3c957d/cluster-monitoring-operator/0.log" Sep 17 07:59:43.450660 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:43.450645 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wkvwb_ee598350-452c-4087-9df4-729094cfd671/node-exporter/0.log" Sep 17 07:59:43.474081 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:43.474065 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wkvwb_ee598350-452c-4087-9df4-729094cfd671/kube-rbac-proxy/0.log" Sep 17 07:59:43.496587 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:43.496571 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wkvwb_ee598350-452c-4087-9df4-729094cfd671/init-textfile/0.log" Sep 17 07:59:43.966152 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:43.966134 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-xngr7_ff6d3938-9c3b-4629-82ed-83fe088836d8/prometheus-operator/0.log" Sep 17 07:59:43.990704 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:43.990688 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-xngr7_ff6d3938-9c3b-4629-82ed-83fe088836d8/kube-rbac-proxy/0.log" Sep 17 07:59:44.020515 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:44.020491 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-6rfjp_c71c7342-234e-4b40-ac3c-f7579ee3ffc5/prometheus-operator-admission-webhook/0.log" Sep 17 07:59:44.066627 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:44.066612 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b49b5b569-4nw84_aec98c2a-2693-44a2-a6a4-6b53e2bca7ed/telemeter-client/0.log" Sep 17 07:59:44.101581 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:44.101553 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b49b5b569-4nw84_aec98c2a-2693-44a2-a6a4-6b53e2bca7ed/reload/0.log" Sep 17 07:59:44.124676 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:44.124659 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b49b5b569-4nw84_aec98c2a-2693-44a2-a6a4-6b53e2bca7ed/kube-rbac-proxy/0.log" Sep 17 07:59:45.657445 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.657417 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9"] Sep 17 07:59:45.657844 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.657724 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79bf3f73-80e2-45e7-8f24-0652def7e9f9" containerName="gather" Sep 17 07:59:45.657844 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.657735 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="79bf3f73-80e2-45e7-8f24-0652def7e9f9" containerName="gather" Sep 17 07:59:45.657844 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.657743 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79bf3f73-80e2-45e7-8f24-0652def7e9f9" containerName="copy" Sep 17 07:59:45.657844 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.657748 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="79bf3f73-80e2-45e7-8f24-0652def7e9f9" containerName="copy" Sep 17 07:59:45.657844 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.657803 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="79bf3f73-80e2-45e7-8f24-0652def7e9f9" containerName="copy" Sep 17 07:59:45.657844 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.657812 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="79bf3f73-80e2-45e7-8f24-0652def7e9f9" containerName="gather" Sep 17 07:59:45.659725 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.659710 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9" Sep 17 07:59:45.662295 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.662254 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tskbl\"/\"kube-root-ca.crt\"" Sep 17 07:59:45.662404 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.662308 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tskbl\"/\"openshift-service-ca.crt\"" Sep 17 07:59:45.663285 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.663269 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tskbl\"/\"default-dockercfg-vpx4f\"" Sep 17 07:59:45.672965 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.672946 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9"] Sep 17 07:59:45.751493 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.751469 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5fr4\" (UniqueName: \"kubernetes.io/projected/9b216267-94b8-4398-9fe6-2b4110a96df9-kube-api-access-g5fr4\") pod \"perf-node-gather-daemonset-z9qj9\" (UID: \"9b216267-94b8-4398-9fe6-2b4110a96df9\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9" Sep 17 07:59:45.751598 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.751503 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9b216267-94b8-4398-9fe6-2b4110a96df9-proc\") pod \"perf-node-gather-daemonset-z9qj9\" (UID: \"9b216267-94b8-4398-9fe6-2b4110a96df9\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9" Sep 17 07:59:45.751598 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.751524 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b216267-94b8-4398-9fe6-2b4110a96df9-sys\") pod \"perf-node-gather-daemonset-z9qj9\" (UID: \"9b216267-94b8-4398-9fe6-2b4110a96df9\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9" Sep 17 07:59:45.751720 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.751619 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9b216267-94b8-4398-9fe6-2b4110a96df9-podres\") pod \"perf-node-gather-daemonset-z9qj9\" (UID: \"9b216267-94b8-4398-9fe6-2b4110a96df9\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9" Sep 17 07:59:45.751720 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.751662 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9b216267-94b8-4398-9fe6-2b4110a96df9-lib-modules\") pod \"perf-node-gather-daemonset-z9qj9\" (UID: \"9b216267-94b8-4398-9fe6-2b4110a96df9\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9" Sep 17 07:59:45.807968 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.807952 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-5kks2_2455e65f-98bb-463d-a204-5dbd5d69b5f3/console-operator/1.log" Sep 17 07:59:45.812525 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.812503 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-5kks2_2455e65f-98bb-463d-a204-5dbd5d69b5f3/console-operator/2.log" Sep 17 07:59:45.852151 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.852128 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9b216267-94b8-4398-9fe6-2b4110a96df9-lib-modules\") pod \"perf-node-gather-daemonset-z9qj9\" (UID: \"9b216267-94b8-4398-9fe6-2b4110a96df9\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9" Sep 17 07:59:45.852253 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.852179 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5fr4\" (UniqueName: \"kubernetes.io/projected/9b216267-94b8-4398-9fe6-2b4110a96df9-kube-api-access-g5fr4\") pod \"perf-node-gather-daemonset-z9qj9\" (UID: \"9b216267-94b8-4398-9fe6-2b4110a96df9\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9" Sep 17 07:59:45.852253 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.852216 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9b216267-94b8-4398-9fe6-2b4110a96df9-proc\") pod \"perf-node-gather-daemonset-z9qj9\" (UID: \"9b216267-94b8-4398-9fe6-2b4110a96df9\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9" Sep 17 07:59:45.852253 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.852246 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b216267-94b8-4398-9fe6-2b4110a96df9-sys\") pod \"perf-node-gather-daemonset-z9qj9\" (UID: \"9b216267-94b8-4398-9fe6-2b4110a96df9\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9" Sep 17 07:59:45.852404 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.852285 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9b216267-94b8-4398-9fe6-2b4110a96df9-lib-modules\") pod \"perf-node-gather-daemonset-z9qj9\" (UID: \"9b216267-94b8-4398-9fe6-2b4110a96df9\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9" Sep 17 07:59:45.852404 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.852287 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9b216267-94b8-4398-9fe6-2b4110a96df9-podres\") pod \"perf-node-gather-daemonset-z9qj9\" (UID: \"9b216267-94b8-4398-9fe6-2b4110a96df9\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9" Sep 17 07:59:45.852404 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.852318 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9b216267-94b8-4398-9fe6-2b4110a96df9-proc\") pod \"perf-node-gather-daemonset-z9qj9\" (UID: \"9b216267-94b8-4398-9fe6-2b4110a96df9\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9" Sep 17 07:59:45.852404 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.852330 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9b216267-94b8-4398-9fe6-2b4110a96df9-sys\") pod \"perf-node-gather-daemonset-z9qj9\" (UID: \"9b216267-94b8-4398-9fe6-2b4110a96df9\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9" Sep 17 07:59:45.852404 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.852372 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9b216267-94b8-4398-9fe6-2b4110a96df9-podres\") pod \"perf-node-gather-daemonset-z9qj9\" (UID: \"9b216267-94b8-4398-9fe6-2b4110a96df9\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9" Sep 17 07:59:45.861400 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.861383 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5fr4\" (UniqueName: \"kubernetes.io/projected/9b216267-94b8-4398-9fe6-2b4110a96df9-kube-api-access-g5fr4\") pod \"perf-node-gather-daemonset-z9qj9\" (UID: \"9b216267-94b8-4398-9fe6-2b4110a96df9\") " pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9" Sep 17 07:59:45.969530 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:45.969512 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9" Sep 17 07:59:46.241439 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:46.241360 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9"] Sep 17 07:59:46.245353 ip-10-0-141-54 kubenswrapper[2553]: W0917 07:59:46.245325 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9b216267_94b8_4398_9fe6_2b4110a96df9.slice/crio-1b54071de2312d1af154dc539d797fca2ef840eab738e00241de4a849a845758 WatchSource:0}: Error finding container 1b54071de2312d1af154dc539d797fca2ef840eab738e00241de4a849a845758: Status 404 returned error can't find the container with id 1b54071de2312d1af154dc539d797fca2ef840eab738e00241de4a849a845758 Sep 17 07:59:46.246938 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:46.246916 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5b674557db-pfdg8_ea856b35-94ab-40f2-add2-c2007255505f/console/0.log" Sep 17 07:59:46.291637 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:46.291611 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-jbx97_5541ff6e-1169-489f-adf9-fe21aac2cd9a/download-server/0.log" Sep 17 07:59:46.799113 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:46.799091 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-cn25t_bb0ab5e4-50f2-430c-ac0a-5c451b8715b6/volume-data-source-validator/0.log" Sep 17 07:59:46.935894 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:46.935859 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9" event={"ID":"9b216267-94b8-4398-9fe6-2b4110a96df9","Type":"ContainerStarted","Data":"4db3e1c46e2966196ff2138c039994771fc2ca5f27e5264efa08515084de6451"} Sep 17 07:59:46.936012 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:46.935924 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9" event={"ID":"9b216267-94b8-4398-9fe6-2b4110a96df9","Type":"ContainerStarted","Data":"1b54071de2312d1af154dc539d797fca2ef840eab738e00241de4a849a845758"} Sep 17 07:59:46.936012 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:46.935952 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9" Sep 17 07:59:46.950848 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:46.950811 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9" podStartSLOduration=1.950799322 podStartE2EDuration="1.950799322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:59:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:59:46.950457296 +0000 UTC m=+1908.508642126" watchObservedRunningTime="2026-09-17 07:59:46.950799322 +0000 UTC m=+1908.508984175" Sep 17 07:59:47.781929 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:47.781871 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ctfdd_688b5736-013f-4a52-8145-0fb44f6eb438/dns/0.log" Sep 17 07:59:47.821016 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:47.820995 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ctfdd_688b5736-013f-4a52-8145-0fb44f6eb438/kube-rbac-proxy/0.log" Sep 17 07:59:48.019852 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:48.019826 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ddqsw_6d54b36a-cd25-4e0e-8a1d-a0b1ce6de23e/dns-node-resolver/0.log" Sep 17 07:59:48.604176 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:48.604150 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7l8hw_b388be8d-ade7-4e8d-b182-c27d6b5c5b1a/node-ca/0.log" Sep 17 07:59:49.910934 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:49.910908 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qsklm_fdc52c65-c729-4ef4-9dde-14b700bc0a46/serve-healthcheck-canary/0.log" Sep 17 07:59:50.559674 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:50.559644 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nwctn_2669c49a-7327-437d-8ed2-fcba617cc62e/kube-rbac-proxy/0.log" Sep 17 07:59:50.582302 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:50.582277 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nwctn_2669c49a-7327-437d-8ed2-fcba617cc62e/exporter/0.log" Sep 17 07:59:50.605256 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:50.605239 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nwctn_2669c49a-7327-437d-8ed2-fcba617cc62e/extractor/0.log" Sep 17 07:59:52.941435 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:52.941411 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tskbl/perf-node-gather-daemonset-z9qj9" Sep 17 07:59:56.963312 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:56.963288 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-spnx4_8af26520-8854-41c6-a082-58fd1e819408/kube-multus-additional-cni-plugins/0.log" Sep 17 07:59:56.984363 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:56.984341 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-spnx4_8af26520-8854-41c6-a082-58fd1e819408/egress-router-binary-copy/0.log" Sep 17 07:59:57.011358 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:57.011334 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-spnx4_8af26520-8854-41c6-a082-58fd1e819408/cni-plugins/0.log" Sep 17 07:59:57.034590 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:57.034567 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-spnx4_8af26520-8854-41c6-a082-58fd1e819408/bond-cni-plugin/0.log" Sep 17 07:59:57.057862 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:57.057842 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-spnx4_8af26520-8854-41c6-a082-58fd1e819408/routeoverride-cni/0.log" Sep 17 07:59:57.081306 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:57.081289 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-spnx4_8af26520-8854-41c6-a082-58fd1e819408/whereabouts-cni-bincopy/0.log" Sep 17 07:59:57.107209 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:57.107193 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-spnx4_8af26520-8854-41c6-a082-58fd1e819408/whereabouts-cni/0.log" Sep 17 07:59:57.142289 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:57.142272 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-rb9tt_9ab08789-ddd8-47f2-af25-0e3ab923c5ff/kube-multus/0.log" Sep 17 07:59:57.246799 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:57.246754 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xh2n2_c35e7d45-1111-40a8-acf5-666b633e8180/network-metrics-daemon/0.log" Sep 17 07:59:57.268630 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:57.268610 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xh2n2_c35e7d45-1111-40a8-acf5-666b633e8180/kube-rbac-proxy/0.log" Sep 17 07:59:58.572866 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:58.572839 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/ovn-controller/0.log" Sep 17 07:59:58.600435 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:58.600414 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/ovn-acl-logging/1.log" Sep 17 07:59:58.600627 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:58.600617 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/ovn-acl-logging/0.log" Sep 17 07:59:58.626674 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:58.626658 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/kube-rbac-proxy-node/0.log" Sep 17 07:59:58.660888 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:58.660864 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/kube-rbac-proxy-ovn-metrics/0.log" Sep 17 07:59:58.699967 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:58.699950 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/northd/0.log" Sep 17 07:59:58.730229 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:58.730214 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/nbdb/0.log" Sep 17 07:59:58.759928 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:58.759910 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/sbdb/0.log" Sep 17 07:59:58.810679 ip-10-0-141-54 kubenswrapper[2553]: I0917 07:59:58.810656 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zhw5d_77d69f00-3692-47b0-845d-f07095a74966/ovnkube-controller/0.log" Sep 17 08:00:00.068944 ip-10-0-141-54 kubenswrapper[2553]: I0917 08:00:00.068914 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-tsc9z_6a96dd04-5687-4059-bf3f-ee1864ed8e40/network-check-target-container/0.log" Sep 17 08:00:01.178591 ip-10-0-141-54 kubenswrapper[2553]: I0917 08:00:01.178565 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-h5j5d_251d0b36-1511-412f-8a1c-d599e9c134b6/iptables-alerter/0.log" Sep 17 08:00:02.047323 ip-10-0-141-54 kubenswrapper[2553]: I0917 08:00:02.047293 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-bb6f2_76674b13-2b8e-49bf-b508-773f53a76a95/tuned/0.log"