Sep 17 07:18:03.346812 ip-10-0-141-215 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 17 07:18:03.346927 ip-10-0-141-215 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 17 07:18:03.346943 ip-10-0-141-215 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 17 07:18:03.347157 ip-10-0-141-215 systemd[1]: Failed to start Kubernetes Kubelet. Sep 17 07:18:13.431926 ip-10-0-141-215 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 17 07:18:13.431945 ip-10-0-141-215 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot fa059cd981214f71b5e3028940c022b0 -- Sep 17 07:19:51.216806 ip-10-0-141-215 systemd[1]: Starting Kubernetes Kubelet... Sep 17 07:19:51.634318 ip-10-0-141-215 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 07:19:51.634814 ip-10-0-141-215 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 17 07:19:51.634814 ip-10-0-141-215 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 07:19:51.634814 ip-10-0-141-215 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 17 07:19:51.634814 ip-10-0-141-215 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 07:19:51.636257 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.636175 2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 17 07:19:51.642107 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642090 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 07:19:51.642107 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642107 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 07:19:51.642171 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642111 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 07:19:51.642171 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642115 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 07:19:51.642171 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642118 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 07:19:51.642171 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642121 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 07:19:51.642171 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642125 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 07:19:51.642171 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642128 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 07:19:51.642171 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642131 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 07:19:51.642171 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642134 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 07:19:51.642171 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642136 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 07:19:51.642171 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642139 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 07:19:51.642171 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642142 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 07:19:51.642171 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642144 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 07:19:51.642171 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642147 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 07:19:51.642171 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642149 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 07:19:51.642171 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642152 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 07:19:51.642171 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642154 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 07:19:51.642171 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642161 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 07:19:51.642171 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642163 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 07:19:51.642171 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642166 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 07:19:51.642171 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642168 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 07:19:51.642665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642171 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 07:19:51.642665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642173 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 07:19:51.642665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642176 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 07:19:51.642665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642179 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 07:19:51.642665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642182 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 07:19:51.642665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642186 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 07:19:51.642665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642188 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 07:19:51.642665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642191 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 07:19:51.642665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642193 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 07:19:51.642665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642196 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 07:19:51.642665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642199 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 07:19:51.642665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642201 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 07:19:51.642665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642204 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 07:19:51.642665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642206 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 07:19:51.642665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642209 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 07:19:51.642665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642212 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 07:19:51.642665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642214 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 07:19:51.642665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642217 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 07:19:51.642665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642219 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 07:19:51.643164 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642223 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 07:19:51.643164 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642228 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 07:19:51.643164 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642231 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 07:19:51.643164 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642233 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 07:19:51.643164 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642236 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 07:19:51.643164 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642239 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 07:19:51.643164 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642242 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 07:19:51.643164 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642244 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 07:19:51.643164 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642247 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 07:19:51.643164 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642249 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 07:19:51.643164 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642252 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 07:19:51.643164 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642255 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 07:19:51.643164 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642257 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 07:19:51.643164 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642259 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 07:19:51.643164 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642263 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 07:19:51.643164 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642266 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 07:19:51.643164 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642268 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 07:19:51.643164 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642271 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 07:19:51.643164 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642273 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 07:19:51.643164 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642276 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 07:19:51.643665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642278 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 07:19:51.643665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642281 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 07:19:51.643665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642283 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 07:19:51.643665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642286 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 07:19:51.643665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642288 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 07:19:51.643665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642291 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 07:19:51.643665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642294 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 07:19:51.643665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642297 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 07:19:51.643665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642299 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 07:19:51.643665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642302 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 07:19:51.643665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642304 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 07:19:51.643665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642307 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 07:19:51.643665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642309 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 07:19:51.643665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642312 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 07:19:51.643665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642314 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 17 07:19:51.643665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642318 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 07:19:51.643665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642322 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 07:19:51.643665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642326 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 07:19:51.643665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642329 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 07:19:51.643665 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642332 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 07:19:51.644440 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642334 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 07:19:51.644440 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642337 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 07:19:51.644440 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642340 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 07:19:51.644440 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642344 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 07:19:51.644440 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642346 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 07:19:51.644440 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642349 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 07:19:51.644440 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642353 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 07:19:51.644440 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642356 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 07:19:51.644440 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642360 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 07:19:51.644440 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642376 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 07:19:51.644440 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642379 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 07:19:51.644440 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642382 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 07:19:51.644440 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642384 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 07:19:51.644440 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642387 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 07:19:51.644440 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642389 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 07:19:51.644440 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642392 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 07:19:51.644440 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642395 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 07:19:51.644440 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.642397 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 07:19:51.644440 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643118 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 07:19:51.644440 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643125 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 07:19:51.644961 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643128 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 07:19:51.644961 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643131 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 07:19:51.644961 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643134 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 07:19:51.644961 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643137 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 07:19:51.644961 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643140 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 07:19:51.644961 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643142 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 07:19:51.644961 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643146 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 07:19:51.644961 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643148 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 07:19:51.644961 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643151 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 07:19:51.644961 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643153 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 07:19:51.644961 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643156 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 07:19:51.644961 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643159 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 07:19:51.644961 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643161 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 07:19:51.644961 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643163 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 07:19:51.644961 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643166 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 07:19:51.644961 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643169 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 07:19:51.644961 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643172 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 07:19:51.644961 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643175 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 07:19:51.644961 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643179 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 07:19:51.644961 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643181 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 07:19:51.645476 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643184 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 07:19:51.645476 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643186 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 07:19:51.645476 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643189 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 07:19:51.645476 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643191 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 07:19:51.645476 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643194 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 07:19:51.645476 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643197 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 07:19:51.645476 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643199 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 07:19:51.645476 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643202 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 07:19:51.645476 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643204 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 07:19:51.645476 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643206 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 07:19:51.645476 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643209 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 07:19:51.645476 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643212 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 07:19:51.645476 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643214 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 07:19:51.645476 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643217 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 07:19:51.645476 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643219 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 07:19:51.645476 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643222 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 07:19:51.645476 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643224 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 07:19:51.645476 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643227 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 07:19:51.645476 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643229 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 07:19:51.646024 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643231 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 07:19:51.646024 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643234 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 07:19:51.646024 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643236 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 07:19:51.646024 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643239 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 07:19:51.646024 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643241 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 07:19:51.646024 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643243 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 07:19:51.646024 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643246 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 07:19:51.646024 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643248 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 07:19:51.646024 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643250 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 07:19:51.646024 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643253 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 07:19:51.646024 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643256 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 07:19:51.646024 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643261 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 07:19:51.646024 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643265 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 07:19:51.646024 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643269 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 07:19:51.646024 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643271 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 07:19:51.646024 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643274 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 07:19:51.646024 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643278 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 07:19:51.646024 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643281 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 07:19:51.646024 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643284 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 07:19:51.646517 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643288 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 07:19:51.646517 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643291 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 07:19:51.646517 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643293 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 07:19:51.646517 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643296 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 17 07:19:51.646517 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643298 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 07:19:51.646517 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643300 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 07:19:51.646517 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643303 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 07:19:51.646517 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643305 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 07:19:51.646517 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643308 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 07:19:51.646517 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643312 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 07:19:51.646517 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643314 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 07:19:51.646517 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643317 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 07:19:51.646517 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643319 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 07:19:51.646517 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643322 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 07:19:51.646517 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643324 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 07:19:51.646517 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643327 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 07:19:51.646517 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643329 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 07:19:51.646517 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643332 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 07:19:51.646517 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643334 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 07:19:51.646517 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643338 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 07:19:51.647001 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643341 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 07:19:51.647001 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643344 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 07:19:51.647001 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643346 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 07:19:51.647001 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643349 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 07:19:51.647001 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643355 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 07:19:51.647001 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643359 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 07:19:51.647001 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643362 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 07:19:51.647001 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643380 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 07:19:51.647001 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643383 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 07:19:51.647001 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643387 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 07:19:51.647001 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643390 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 07:19:51.647001 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643393 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 07:19:51.647001 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643396 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 07:19:51.647001 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643398 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 07:19:51.647001 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643401 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 07:19:51.647001 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643404 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 07:19:51.647001 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643407 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 07:19:51.647001 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643410 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 07:19:51.647001 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.643412 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 07:19:51.647493 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645234 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 07:19:51.647493 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645248 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 07:19:51.647493 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645254 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 07:19:51.647493 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645258 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 07:19:51.647493 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645261 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 07:19:51.647493 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645264 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 07:19:51.647493 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645266 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 07:19:51.647493 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645269 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 07:19:51.647493 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645272 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 07:19:51.647493 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645274 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 07:19:51.647493 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645277 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 07:19:51.647493 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645279 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 07:19:51.647493 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645282 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 07:19:51.647493 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645285 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 07:19:51.647493 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645287 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 07:19:51.647493 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645290 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 07:19:51.647493 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645292 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 07:19:51.647493 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645295 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 07:19:51.647493 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645298 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 07:19:51.647953 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645301 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 07:19:51.647953 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645304 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 07:19:51.647953 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645306 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 07:19:51.647953 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645309 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 07:19:51.647953 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645311 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 07:19:51.647953 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645314 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 07:19:51.647953 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645317 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 07:19:51.647953 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645319 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 07:19:51.647953 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645322 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 07:19:51.647953 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645324 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 07:19:51.647953 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645327 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 07:19:51.647953 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645330 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 17 07:19:51.647953 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645332 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 07:19:51.647953 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645335 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 07:19:51.647953 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645338 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 07:19:51.647953 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645341 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 07:19:51.647953 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645343 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 07:19:51.647953 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645346 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 07:19:51.647953 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645349 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 07:19:51.647953 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645351 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 07:19:51.648495 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645354 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 07:19:51.648495 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645356 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 07:19:51.648495 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645359 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 07:19:51.648495 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645361 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 07:19:51.648495 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645381 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 07:19:51.648495 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645384 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 07:19:51.648495 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645387 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 07:19:51.648495 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645390 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 07:19:51.648495 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645392 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 07:19:51.648495 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645394 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 07:19:51.648495 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645398 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 07:19:51.648495 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645402 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 07:19:51.648495 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645404 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 07:19:51.648495 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645407 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 07:19:51.648495 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645410 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 07:19:51.648495 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645412 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 07:19:51.648495 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645415 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 07:19:51.648495 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645417 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 07:19:51.648495 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645420 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 07:19:51.648495 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645423 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 07:19:51.648995 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645425 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 07:19:51.648995 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645428 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 07:19:51.648995 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645431 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 07:19:51.648995 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645433 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 07:19:51.648995 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645436 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 07:19:51.648995 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645439 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 07:19:51.648995 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645442 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 07:19:51.648995 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645445 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 07:19:51.648995 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645447 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 07:19:51.648995 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645450 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 07:19:51.648995 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645453 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 07:19:51.648995 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645456 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 07:19:51.648995 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645459 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 07:19:51.648995 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645461 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 07:19:51.648995 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645464 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 07:19:51.648995 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645467 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 07:19:51.648995 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645471 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 07:19:51.648995 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645474 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 07:19:51.648995 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645477 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 07:19:51.648995 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645480 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 07:19:51.649497 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645483 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 07:19:51.649497 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645486 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 07:19:51.649497 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645488 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 07:19:51.649497 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645491 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 07:19:51.649497 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645494 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 07:19:51.649497 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645497 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 07:19:51.649497 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645500 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 07:19:51.649497 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645502 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 07:19:51.649497 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645505 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 07:19:51.649497 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645507 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 07:19:51.649497 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645509 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 07:19:51.649497 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645513 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 07:19:51.649497 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645516 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 07:19:51.649497 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645520 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 07:19:51.649497 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645524 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 07:19:51.649497 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645527 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 07:19:51.649497 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645530 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 07:19:51.649497 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645533 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 07:19:51.649497 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645537 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 07:19:51.649953 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.645540 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 07:19:51.649953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645618 2563 flags.go:64] FLAG: --address="0.0.0.0" Sep 17 07:19:51.649953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645653 2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 17 07:19:51.649953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645660 2563 flags.go:64] FLAG: --anonymous-auth="true" Sep 17 07:19:51.649953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645665 2563 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 17 07:19:51.649953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645670 2563 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 17 07:19:51.649953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645673 2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 17 07:19:51.649953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645677 2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 17 07:19:51.649953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645682 2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 17 07:19:51.649953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645685 2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 17 07:19:51.649953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645688 2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 17 07:19:51.649953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645691 2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 17 07:19:51.649953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645694 2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 17 07:19:51.649953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645697 2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 17 07:19:51.649953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645700 2563 flags.go:64] FLAG: --cgroup-root="" Sep 17 07:19:51.649953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645703 2563 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 17 07:19:51.649953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645706 2563 flags.go:64] FLAG: --client-ca-file="" Sep 17 07:19:51.649953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645708 2563 flags.go:64] FLAG: --cloud-provider="external" Sep 17 07:19:51.649953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645713 2563 flags.go:64] FLAG: --cluster-dns="[]" Sep 17 07:19:51.649953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645717 2563 flags.go:64] FLAG: --cluster-domain="" Sep 17 07:19:51.649953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645720 2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 17 07:19:51.649953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645723 2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 17 07:19:51.649953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645726 2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 17 07:19:51.649953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645729 2563 flags.go:64] FLAG: --container-log-max-files="5" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645733 2563 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645736 2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645739 2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645742 2563 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645745 2563 flags.go:64] FLAG: --contention-profiling="false" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645747 2563 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645751 2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645754 2563 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645757 2563 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645762 2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645764 2563 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645767 2563 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645770 2563 flags.go:64] FLAG: --enable-load-reader="false" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645773 2563 flags.go:64] FLAG: --enable-server="true" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645776 2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645780 2563 flags.go:64] FLAG: --event-burst="100" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645783 2563 flags.go:64] FLAG: --event-qps="50" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645786 2563 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645789 2563 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645792 2563 flags.go:64] FLAG: --eviction-hard="" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645795 2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645799 2563 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645802 2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645805 2563 flags.go:64] FLAG: --eviction-soft="" Sep 17 07:19:51.650541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645808 2563 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 17 07:19:51.651140 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645811 2563 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 17 07:19:51.651140 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645813 2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 17 07:19:51.651140 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645817 2563 flags.go:64] FLAG: --experimental-mounter-path="" Sep 17 07:19:51.651140 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645820 2563 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 17 07:19:51.651140 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645823 2563 flags.go:64] FLAG: --fail-swap-on="true" Sep 17 07:19:51.651140 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645828 2563 flags.go:64] FLAG: --feature-gates="" Sep 17 07:19:51.651140 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645832 2563 flags.go:64] FLAG: --file-check-frequency="20s" Sep 17 07:19:51.651140 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645835 2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 17 07:19:51.651140 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645838 2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 17 07:19:51.651140 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645841 2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 17 07:19:51.651140 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645844 2563 flags.go:64] FLAG: --healthz-port="10248" Sep 17 07:19:51.651140 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645847 2563 flags.go:64] FLAG: --help="false" Sep 17 07:19:51.651140 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645850 2563 flags.go:64] FLAG: --hostname-override="ip-10-0-141-215.ec2.internal" Sep 17 07:19:51.651140 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645853 2563 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 17 07:19:51.651140 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645856 2563 flags.go:64] FLAG: --http-check-frequency="20s" Sep 17 07:19:51.651140 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645859 2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 17 07:19:51.651140 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645862 2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 17 07:19:51.651140 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645865 2563 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 17 07:19:51.651140 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645868 2563 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 17 07:19:51.651140 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645870 2563 flags.go:64] FLAG: --image-service-endpoint="" Sep 17 07:19:51.651140 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645873 2563 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 17 07:19:51.651140 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645876 2563 flags.go:64] FLAG: --kube-api-burst="100" Sep 17 07:19:51.651140 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645878 2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645881 2563 flags.go:64] FLAG: --kube-api-qps="50" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645885 2563 flags.go:64] FLAG: --kube-reserved="" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645887 2563 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645890 2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645893 2563 flags.go:64] FLAG: --kubelet-cgroups="" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645896 2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645898 2563 flags.go:64] FLAG: --lock-file="" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645901 2563 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645904 2563 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645907 2563 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645912 2563 flags.go:64] FLAG: --log-json-split-stream="false" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645914 2563 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645918 2563 flags.go:64] FLAG: --log-text-split-stream="false" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645920 2563 flags.go:64] FLAG: --logging-format="text" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645924 2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645928 2563 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645930 2563 flags.go:64] FLAG: --manifest-url="" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645933 2563 flags.go:64] FLAG: --manifest-url-header="" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645938 2563 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645941 2563 flags.go:64] FLAG: --max-open-files="1000000" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645944 2563 flags.go:64] FLAG: --max-pods="110" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645947 2563 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645950 2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645953 2563 flags.go:64] FLAG: --memory-manager-policy="None" Sep 17 07:19:51.651771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645955 2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645958 2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645961 2563 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645964 2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645971 2563 flags.go:64] FLAG: --node-status-max-images="50" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645974 2563 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645977 2563 flags.go:64] FLAG: --oom-score-adj="-999" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645979 2563 flags.go:64] FLAG: --pod-cidr="" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645982 2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645988 2563 flags.go:64] FLAG: --pod-manifest-path="" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645990 2563 flags.go:64] FLAG: --pod-max-pids="-1" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645993 2563 flags.go:64] FLAG: --pods-per-core="0" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645996 2563 flags.go:64] FLAG: --port="10250" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.645999 2563 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646001 2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fa1e102476c56d2f" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646004 2563 flags.go:64] FLAG: --qos-reserved="" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646007 2563 flags.go:64] FLAG: --read-only-port="10255" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646010 2563 flags.go:64] FLAG: --register-node="true" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646012 2563 flags.go:64] FLAG: --register-with-taints="" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646016 2563 flags.go:64] FLAG: --registry-burst="10" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646019 2563 flags.go:64] FLAG: --registry-qps="5" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646022 2563 flags.go:64] FLAG: --reserved-cpus="" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646026 2563 flags.go:64] FLAG: --reserved-memory="" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646030 2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646033 2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 17 07:19:51.652399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646036 2563 flags.go:64] FLAG: --rotate-certificates="false" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646039 2563 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646043 2563 flags.go:64] FLAG: --runonce="false" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646046 2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646049 2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646052 2563 flags.go:64] FLAG: --seccomp-default="false" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646055 2563 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646058 2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646061 2563 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646064 2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646067 2563 flags.go:64] FLAG: --storage-driver-password="root" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646069 2563 flags.go:64] FLAG: --storage-driver-secure="false" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646072 2563 flags.go:64] FLAG: --storage-driver-table="stats" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646075 2563 flags.go:64] FLAG: --storage-driver-user="root" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646078 2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646081 2563 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646084 2563 flags.go:64] FLAG: --system-cgroups="" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646087 2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646093 2563 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646096 2563 flags.go:64] FLAG: --tls-cert-file="" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646098 2563 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646102 2563 flags.go:64] FLAG: --tls-min-version="" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646105 2563 flags.go:64] FLAG: --tls-private-key-file="" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646108 2563 flags.go:64] FLAG: --topology-manager-policy="none" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646111 2563 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 17 07:19:51.653034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646114 2563 flags.go:64] FLAG: --topology-manager-scope="container" Sep 17 07:19:51.653791 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646117 2563 flags.go:64] FLAG: --v="2" Sep 17 07:19:51.653791 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646120 2563 flags.go:64] FLAG: --version="false" Sep 17 07:19:51.653791 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646124 2563 flags.go:64] FLAG: --vmodule="" Sep 17 07:19:51.653791 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646130 2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 17 07:19:51.653791 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.646133 2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 17 07:19:51.653791 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646251 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 07:19:51.653791 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646255 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 07:19:51.653791 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646257 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 07:19:51.653791 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646262 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 07:19:51.653791 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646265 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 17 07:19:51.653791 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646269 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 07:19:51.653791 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646273 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 07:19:51.653791 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646276 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 07:19:51.653791 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646279 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 07:19:51.653791 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646282 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 07:19:51.653791 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646284 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 07:19:51.653791 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646287 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 07:19:51.653791 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646290 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 07:19:51.653791 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646292 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 07:19:51.653791 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646295 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 07:19:51.653791 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646297 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 07:19:51.654699 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646300 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 07:19:51.654699 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646303 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 07:19:51.654699 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646305 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 07:19:51.654699 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646308 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 07:19:51.654699 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646312 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 07:19:51.654699 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646315 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 07:19:51.654699 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646318 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 07:19:51.654699 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646321 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 07:19:51.654699 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646324 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 07:19:51.654699 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646327 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 07:19:51.654699 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646329 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 07:19:51.654699 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646332 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 07:19:51.654699 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646334 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 07:19:51.654699 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646337 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 07:19:51.654699 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646341 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 07:19:51.654699 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646346 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 07:19:51.654699 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646349 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 07:19:51.654699 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646352 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 07:19:51.654699 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646354 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 07:19:51.654699 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646358 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 07:19:51.655576 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646361 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 07:19:51.655576 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646380 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 07:19:51.655576 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646384 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 07:19:51.655576 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646388 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 07:19:51.655576 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646392 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 07:19:51.655576 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646395 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 07:19:51.655576 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646397 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 07:19:51.655576 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646400 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 07:19:51.655576 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646402 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 07:19:51.655576 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646405 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 07:19:51.655576 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646408 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 07:19:51.655576 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646410 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 07:19:51.655576 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646413 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 07:19:51.655576 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646415 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 07:19:51.655576 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646418 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 07:19:51.655576 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646420 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 07:19:51.655576 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646423 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 07:19:51.655576 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646427 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 07:19:51.655576 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646430 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 07:19:51.656186 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646433 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 07:19:51.656186 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646437 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 07:19:51.656186 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646440 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 07:19:51.656186 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646443 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 07:19:51.656186 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646445 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 07:19:51.656186 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646448 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 07:19:51.656186 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646451 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 07:19:51.656186 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646456 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 07:19:51.656186 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646459 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 07:19:51.656186 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646463 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 07:19:51.656186 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646467 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 07:19:51.656186 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646469 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 07:19:51.656186 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646473 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 07:19:51.656186 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646476 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 07:19:51.656186 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646478 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 07:19:51.656186 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646481 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 07:19:51.656186 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646483 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 07:19:51.656186 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646486 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 07:19:51.656186 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646488 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 07:19:51.656742 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646491 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 07:19:51.656742 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646493 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 07:19:51.656742 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646496 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 07:19:51.656742 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646498 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 07:19:51.656742 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646501 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 07:19:51.656742 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646504 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 07:19:51.656742 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646506 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 07:19:51.656742 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646509 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 07:19:51.656742 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646512 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 07:19:51.656742 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646514 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 07:19:51.656742 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646517 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 07:19:51.656742 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646519 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 07:19:51.656742 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646522 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 07:19:51.656742 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646525 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 07:19:51.656742 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646528 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 07:19:51.656742 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646530 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 07:19:51.656742 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646533 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 07:19:51.656742 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646535 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 07:19:51.656742 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646538 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 07:19:51.656742 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646540 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 07:19:51.657242 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646544 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 07:19:51.657242 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646547 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 07:19:51.657242 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646549 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 07:19:51.657242 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646553 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 07:19:51.657242 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.646556 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 07:19:51.657242 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.647254 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 07:19:51.657242 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.655798 2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 17 07:19:51.657242 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.655821 2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 17 07:19:51.657515 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.657500 2563 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 17 07:19:51.657564 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.657522 2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 17 07:19:51.657625 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657614 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 07:19:51.657678 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657626 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 07:19:51.657678 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657632 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 07:19:51.657678 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657637 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 07:19:51.657678 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657641 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 07:19:51.657678 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657646 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 07:19:51.657678 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657650 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 07:19:51.657678 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657655 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 07:19:51.657678 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657659 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 07:19:51.657678 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657663 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 07:19:51.657678 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657667 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 07:19:51.657678 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657674 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 07:19:51.657678 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657681 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 07:19:51.658199 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657687 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 07:19:51.658199 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657692 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 07:19:51.658199 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657697 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 07:19:51.658199 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657701 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 07:19:51.658199 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657706 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 07:19:51.658199 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657710 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 07:19:51.658199 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657715 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 07:19:51.658199 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657719 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 07:19:51.658199 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657724 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 07:19:51.658199 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657728 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 07:19:51.658199 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657733 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 07:19:51.658199 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657737 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 07:19:51.658199 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657741 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 07:19:51.658199 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657745 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 07:19:51.658199 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657749 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 07:19:51.658199 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657753 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 07:19:51.658199 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657759 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 07:19:51.658199 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657763 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 07:19:51.658199 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657767 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 07:19:51.658199 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657772 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 07:19:51.659060 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657776 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 07:19:51.659060 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657780 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 07:19:51.659060 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657784 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 07:19:51.659060 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657788 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 07:19:51.659060 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657792 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 07:19:51.659060 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657796 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 07:19:51.659060 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657800 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 07:19:51.659060 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657805 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 07:19:51.659060 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657809 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 07:19:51.659060 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657813 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 07:19:51.659060 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657817 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 07:19:51.659060 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657821 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 07:19:51.659060 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657825 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 07:19:51.659060 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657830 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 17 07:19:51.659060 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657835 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 07:19:51.659060 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657840 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 07:19:51.659060 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657844 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 07:19:51.659060 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657848 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 07:19:51.659060 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657852 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 07:19:51.659060 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657856 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 07:19:51.659606 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657861 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 07:19:51.659606 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657865 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 07:19:51.659606 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657869 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 07:19:51.659606 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657873 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 07:19:51.659606 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657877 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 07:19:51.659606 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657881 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 07:19:51.659606 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657885 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 07:19:51.659606 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657889 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 07:19:51.659606 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657893 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 07:19:51.659606 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657898 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 07:19:51.659606 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657903 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 07:19:51.659606 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657907 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 07:19:51.659606 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657911 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 07:19:51.659606 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657915 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 07:19:51.659606 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657920 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 07:19:51.659606 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657924 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 07:19:51.659606 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657928 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 07:19:51.659606 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657932 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 07:19:51.659606 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657936 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 07:19:51.659606 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657940 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 07:19:51.660403 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657944 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 07:19:51.660403 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657949 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 07:19:51.660403 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657952 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 07:19:51.660403 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657957 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 07:19:51.660403 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657960 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 07:19:51.660403 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657966 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 07:19:51.660403 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657970 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 07:19:51.660403 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657974 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 07:19:51.660403 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657979 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 07:19:51.660403 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657983 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 07:19:51.660403 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657987 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 07:19:51.660403 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657991 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 07:19:51.660403 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657995 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 07:19:51.660403 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.657999 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 07:19:51.660403 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658003 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 07:19:51.660403 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658007 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 07:19:51.660403 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658012 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 07:19:51.660403 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658015 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 07:19:51.660403 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658020 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 07:19:51.660403 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658024 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 07:19:51.661268 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658029 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 07:19:51.661268 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658033 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 07:19:51.661268 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658045 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 07:19:51.661268 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658049 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 07:19:51.661268 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658056 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 07:19:51.661268 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658061 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 07:19:51.661268 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.658070 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 07:19:51.661268 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658263 2563 feature_gate.go:328] unrecognized feature gate: Example Sep 17 07:19:51.661268 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658272 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 07:19:51.661268 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658278 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 07:19:51.661268 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658283 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 07:19:51.661268 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658287 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 07:19:51.661268 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658291 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 07:19:51.661268 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658295 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 07:19:51.661268 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658299 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 07:19:51.661675 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658303 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 07:19:51.661675 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658308 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 07:19:51.661675 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658312 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 07:19:51.661675 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658317 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 07:19:51.661675 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658321 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 07:19:51.661675 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658326 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 07:19:51.661675 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658330 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 07:19:51.661675 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658334 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 07:19:51.661675 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658338 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 07:19:51.661675 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658342 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 07:19:51.661675 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658346 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 07:19:51.661675 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658350 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 07:19:51.661675 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658354 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 07:19:51.661675 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658358 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 07:19:51.661675 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658381 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 07:19:51.661675 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658386 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 07:19:51.661675 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658390 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 07:19:51.661675 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658394 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 07:19:51.661675 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658398 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 07:19:51.661675 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658402 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 07:19:51.662292 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658411 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 07:19:51.662292 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658416 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 07:19:51.662292 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658420 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 07:19:51.662292 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658424 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 07:19:51.662292 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658428 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 07:19:51.662292 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658432 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 07:19:51.662292 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658437 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 07:19:51.662292 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658442 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 07:19:51.662292 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658446 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 07:19:51.662292 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658450 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 07:19:51.662292 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658454 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 07:19:51.662292 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658459 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 07:19:51.662292 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658465 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 07:19:51.662292 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658471 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 07:19:51.662292 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658476 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 07:19:51.662292 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658480 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 07:19:51.662292 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658484 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 07:19:51.662292 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658488 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 07:19:51.662292 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658495 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 07:19:51.662776 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658500 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 07:19:51.662776 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658505 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 07:19:51.662776 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658509 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 07:19:51.662776 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658514 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 07:19:51.662776 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658519 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 07:19:51.662776 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658523 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 07:19:51.662776 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658528 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 07:19:51.662776 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658532 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 07:19:51.662776 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658536 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 07:19:51.662776 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658540 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 07:19:51.662776 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658544 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 07:19:51.662776 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658549 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 07:19:51.662776 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658553 2563 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 07:19:51.662776 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658558 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 07:19:51.662776 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658563 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 07:19:51.662776 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658567 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 07:19:51.662776 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658571 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 07:19:51.662776 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658575 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 07:19:51.662776 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658579 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 07:19:51.663287 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658586 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 07:19:51.663287 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658591 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 07:19:51.663287 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658597 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 07:19:51.663287 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658602 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 07:19:51.663287 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658607 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 07:19:51.663287 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658612 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 07:19:51.663287 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658616 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 07:19:51.663287 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658621 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 07:19:51.663287 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658625 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 07:19:51.663287 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658630 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 07:19:51.663287 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658635 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 07:19:51.663287 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658640 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 07:19:51.663287 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658644 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 07:19:51.663287 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658649 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 07:19:51.663287 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658653 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 07:19:51.663287 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658658 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 07:19:51.663287 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658663 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 07:19:51.663287 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658667 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 07:19:51.663287 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658671 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 07:19:51.663287 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658675 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 07:19:51.663839 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658679 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 07:19:51.663839 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658684 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 07:19:51.663839 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658688 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 07:19:51.663839 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658692 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 07:19:51.663839 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658696 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 07:19:51.663839 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658700 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 07:19:51.663839 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658705 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 07:19:51.663839 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658710 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 07:19:51.663839 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658715 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 07:19:51.663839 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658719 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 07:19:51.663839 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658723 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 07:19:51.663839 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658727 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 07:19:51.663839 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:51.658731 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 07:19:51.663839 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.658740 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 07:19:51.663839 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.658945 2563 server.go:962] "Client rotation is on, will bootstrap in background" Sep 17 07:19:51.664261 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.662009 2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 17 07:19:51.664261 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.663004 2563 server.go:1019] "Starting client certificate rotation" Sep 17 07:19:51.664261 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.663108 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 17 07:19:51.664261 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.663146 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 17 07:19:51.667308 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.667289 2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 17 07:19:51.668836 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.668814 2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 17 07:19:51.680120 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.680101 2563 log.go:25] "Validated CRI v1 runtime API" Sep 17 07:19:51.683790 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.683776 2563 log.go:25] "Validated CRI v1 image API" Sep 17 07:19:51.684587 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.684572 2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 17 07:19:51.687104 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.687086 2563 fs.go:135] Filesystem UUIDs: map[435a9bc4-e3da-4e1f-a0b6-dae5064306c9:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b440b984-0bfb-4e6d-bfd9-a055934ec7bd:/dev/nvme0n1p4] Sep 17 07:19:51.687155 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.687105 2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 17 07:19:51.692216 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.692099 2563 manager.go:216] Machine: {Timestamp:2026-09-17 07:19:51.690755603 +0000 UTC m=+0.345444287 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098205 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c8bcc311b02b6bc144fd73b1caa30 SystemUUID:ec2c8bcc-311b-02b6-bc14-4fd73b1caa30 BootID:fa059cd9-8121-4f71-b5e3-028940c022b0 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:fb:34:15:2f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:fb:34:15:2f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:a3:03:74:3c:b2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 17 07:19:51.692216 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.692209 2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 17 07:19:51.692344 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.692321 2563 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 17 07:19:51.694759 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.694735 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 17 07:19:51.694955 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.694932 2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 17 07:19:51.695095 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.694959 2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-215.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 17 07:19:51.695816 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.695805 2563 topology_manager.go:138] "Creating topology manager with none policy" Sep 17 07:19:51.695853 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.695819 2563 container_manager_linux.go:309] "Creating device plugin manager" Sep 17 07:19:51.695853 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.695827 2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 17 07:19:51.696544 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.696533 2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 17 07:19:51.698320 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.698309 2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 17 07:19:51.703619 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.703607 2563 state_mem.go:36] "Initialized new in-memory state store" Sep 17 07:19:51.703748 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.703738 2563 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 17 07:19:51.703777 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.703770 2563 kubelet.go:486] "Attempting to sync node with API server" Sep 17 07:19:51.703804 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.703780 2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 17 07:19:51.703804 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.703794 2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 17 07:19:51.703804 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.703802 2563 kubelet.go:398] "Adding apiserver pod source" Sep 17 07:19:51.703896 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.703821 2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 17 07:19:51.704287 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.704266 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 17 07:19:51.704287 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.704289 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 17 07:19:51.707755 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.707727 2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 17 07:19:51.710660 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.710640 2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 17 07:19:51.710707 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.710676 2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 17 07:19:51.712790 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.712779 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 17 07:19:51.712832 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.712796 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 17 07:19:51.712832 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.712802 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 17 07:19:51.712832 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.712808 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 17 07:19:51.712832 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.712816 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 17 07:19:51.712832 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.712822 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 17 07:19:51.712962 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.712835 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 17 07:19:51.712962 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.712841 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 17 07:19:51.712962 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.712851 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 17 07:19:51.712962 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.712857 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 17 07:19:51.712962 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.712866 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 17 07:19:51.712962 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.712871 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 17 07:19:51.714328 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.714311 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 17 07:19:51.714328 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.714325 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 17 07:19:51.715585 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:51.715561 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-215.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 17 07:19:51.715640 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:51.715561 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 17 07:19:51.718447 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.718433 2563 server.go:1268] "Started kubelet" Sep 17 07:19:51.718531 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.718507 2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 17 07:19:51.718619 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.718575 2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 17 07:19:51.718679 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.718642 2563 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 17 07:19:51.719243 ip-10-0-141-215 systemd[1]: Started Kubernetes Kubelet. Sep 17 07:19:51.720059 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.720045 2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 17 07:19:51.720440 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.720420 2563 server.go:310] "Adding debug handlers to kubelet server" Sep 17 07:19:51.724905 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.724878 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x96p6" Sep 17 07:19:51.726004 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:51.725944 2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 17 07:19:51.730931 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.730912 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x96p6" Sep 17 07:19:51.731894 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.731871 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-215.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 17 07:19:51.732033 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.732013 2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 17 07:19:51.732132 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.732038 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 17 07:19:51.732184 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:51.732134 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-215.ec2.internal\" not found" Sep 17 07:19:51.732824 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.732807 2563 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 17 07:19:51.732999 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.732824 2563 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 17 07:19:51.733077 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:51.731913 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-215.ec2.internal.18d60aa1bde5c7b9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-215.ec2.internal,UID:ip-10-0-141-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-215.ec2.internal,},FirstTimestamp:2026-09-17 07:19:51.718315961 +0000 UTC m=+0.373004643,LastTimestamp:2026-09-17 07:19:51.718315961 +0000 UTC m=+0.373004643,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-215.ec2.internal,}" Sep 17 07:19:51.733077 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.732956 2563 factory.go:55] Registering systemd factory Sep 17 07:19:51.733077 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.733069 2563 factory.go:223] Registration of the systemd container factory successfully Sep 17 07:19:51.733214 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.733100 2563 reconstruct.go:97] "Volume reconstruction finished" Sep 17 07:19:51.733214 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.733108 2563 reconciler.go:29] "Reconciler: start to sync state" Sep 17 07:19:51.733214 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.732977 2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 17 07:19:51.733340 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.733316 2563 factory.go:153] Registering CRI-O factory Sep 17 07:19:51.733340 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.733331 2563 factory.go:223] Registration of the crio container factory successfully Sep 17 07:19:51.733465 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.733436 2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 17 07:19:51.733465 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.733456 2563 factory.go:103] Registering Raw factory Sep 17 07:19:51.733559 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.733471 2563 manager.go:1196] Started watching for new ooms in manager Sep 17 07:19:51.733879 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.733864 2563 manager.go:349] Starting recovery of all containers Sep 17 07:19:51.736921 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:51.736893 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 17 07:19:51.743736 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.743717 2563 manager.go:354] Recovery completed Sep 17 07:19:51.746147 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:51.746130 2563 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-215.ec2.internal\" not found" node="ip-10-0-141-215.ec2.internal" Sep 17 07:19:51.747751 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.747732 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:19:51.749597 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.749582 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-215.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:19:51.749681 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.749610 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-215.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:19:51.749681 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.749620 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-215.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:19:51.750117 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.750103 2563 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 17 07:19:51.750189 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.750117 2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 17 07:19:51.750189 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.750136 2563 state_mem.go:36] "Initialized new in-memory state store" Sep 17 07:19:51.750189 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.750163 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:19:51.750823 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.750810 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-215.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:19:51.750883 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.750836 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-215.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:19:51.750883 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.750855 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-215.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:19:51.752255 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.752241 2563 policy_none.go:49] "None policy: Start" Sep 17 07:19:51.752320 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.752261 2563 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 17 07:19:51.752320 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.752274 2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 17 07:19:51.754389 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.754351 2563 policy_none.go:47] "Start" Sep 17 07:19:51.791044 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.791023 2563 manager.go:337] "Starting Device Plugin manager" Sep 17 07:19:51.791151 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:51.791075 2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 17 07:19:51.791151 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.791090 2563 server.go:87] "Starting device plugin registration server" Sep 17 07:19:51.791394 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.791359 2563 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 17 07:19:51.791453 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.791396 2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 17 07:19:51.791600 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.791582 2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 17 07:19:51.791683 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.791670 2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 17 07:19:51.791730 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.791684 2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 17 07:19:51.793396 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:51.792848 2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 17 07:19:51.793396 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:51.792911 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-215.ec2.internal\" not found" Sep 17 07:19:51.837606 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.837572 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 17 07:19:51.839898 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.838982 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 17 07:19:51.839898 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.839032 2563 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 17 07:19:51.839898 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.839059 2563 kubelet.go:2448] "Starting kubelet main sync loop" Sep 17 07:19:51.839898 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:51.839096 2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 17 07:19:51.886349 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.886260 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 07:19:51.892322 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.892307 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:19:51.893348 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.893321 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-215.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:19:51.893433 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.893353 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-215.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:19:51.893433 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.893381 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-215.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:19:51.893433 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.893411 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-215.ec2.internal" Sep 17 07:19:51.904888 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.904871 2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-215.ec2.internal" Sep 17 07:19:51.904933 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:51.904893 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-215.ec2.internal\": node \"ip-10-0-141-215.ec2.internal\" not found" Sep 17 07:19:51.925250 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:51.925231 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-215.ec2.internal\" not found" Sep 17 07:19:51.939689 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.939661 2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-215.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-215.ec2.internal"] Sep 17 07:19:51.939754 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.939748 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:19:51.941398 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.941360 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-215.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:19:51.941476 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.941405 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-215.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:19:51.941476 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.941416 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-215.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:19:51.943710 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.943687 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:19:51.943835 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.943821 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-215.ec2.internal" Sep 17 07:19:51.943871 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.943854 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:19:51.944426 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.944411 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-215.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:19:51.944529 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.944411 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-215.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:19:51.944529 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.944469 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-215.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:19:51.944529 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.944488 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-215.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:19:51.944529 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.944439 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-215.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:19:51.944529 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.944514 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-215.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:19:51.946721 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.946708 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-215.ec2.internal" Sep 17 07:19:51.946773 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.946732 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 07:19:51.947430 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.947417 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-215.ec2.internal" event="NodeHasSufficientMemory" Sep 17 07:19:51.947484 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.947442 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-215.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 07:19:51.947484 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:51.947452 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-215.ec2.internal" event="NodeHasSufficientPID" Sep 17 07:19:51.965940 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:51.965921 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-215.ec2.internal\" not found" node="ip-10-0-141-215.ec2.internal" Sep 17 07:19:51.969305 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:51.969290 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-215.ec2.internal\" not found" node="ip-10-0-141-215.ec2.internal" Sep 17 07:19:52.026248 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:52.026216 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-215.ec2.internal\" not found" Sep 17 07:19:52.035649 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:52.035624 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c5668cb859d7da5d03909e7e9730b526-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-215.ec2.internal\" (UID: \"c5668cb859d7da5d03909e7e9730b526\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-215.ec2.internal" Sep 17 07:19:52.035710 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:52.035657 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5668cb859d7da5d03909e7e9730b526-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-215.ec2.internal\" (UID: \"c5668cb859d7da5d03909e7e9730b526\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-215.ec2.internal" Sep 17 07:19:52.035710 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:52.035687 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/910049e72d6e50062f9693db2cfac120-config\") pod \"kube-apiserver-proxy-ip-10-0-141-215.ec2.internal\" (UID: \"910049e72d6e50062f9693db2cfac120\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-215.ec2.internal" Sep 17 07:19:52.127122 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:52.127093 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-215.ec2.internal\" not found" Sep 17 07:19:52.136493 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:52.136445 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c5668cb859d7da5d03909e7e9730b526-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-215.ec2.internal\" (UID: \"c5668cb859d7da5d03909e7e9730b526\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-215.ec2.internal" Sep 17 07:19:52.136493 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:52.136473 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5668cb859d7da5d03909e7e9730b526-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-215.ec2.internal\" (UID: \"c5668cb859d7da5d03909e7e9730b526\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-215.ec2.internal" Sep 17 07:19:52.136493 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:52.136491 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/910049e72d6e50062f9693db2cfac120-config\") pod \"kube-apiserver-proxy-ip-10-0-141-215.ec2.internal\" (UID: \"910049e72d6e50062f9693db2cfac120\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-215.ec2.internal" Sep 17 07:19:52.136620 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:52.136532 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/910049e72d6e50062f9693db2cfac120-config\") pod \"kube-apiserver-proxy-ip-10-0-141-215.ec2.internal\" (UID: \"910049e72d6e50062f9693db2cfac120\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-215.ec2.internal" Sep 17 07:19:52.136620 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:52.136561 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c5668cb859d7da5d03909e7e9730b526-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-215.ec2.internal\" (UID: \"c5668cb859d7da5d03909e7e9730b526\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-215.ec2.internal" Sep 17 07:19:52.136620 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:52.136562 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5668cb859d7da5d03909e7e9730b526-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-215.ec2.internal\" (UID: \"c5668cb859d7da5d03909e7e9730b526\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-215.ec2.internal" Sep 17 07:19:52.227920 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:52.227873 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-215.ec2.internal\" not found" Sep 17 07:19:52.268343 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:52.268315 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-215.ec2.internal" Sep 17 07:19:52.273921 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:52.273552 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-215.ec2.internal" Sep 17 07:19:52.328909 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:52.328871 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-215.ec2.internal\" not found" Sep 17 07:19:52.429479 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:52.429409 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-215.ec2.internal\" not found" Sep 17 07:19:52.530003 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:52.529964 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-215.ec2.internal\" not found" Sep 17 07:19:52.630663 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:52.630627 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-215.ec2.internal\" not found" Sep 17 07:19:52.663160 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:52.663133 2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 17 07:19:52.663623 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:52.663312 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 17 07:19:52.731697 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:52.731663 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-215.ec2.internal\" not found" Sep 17 07:19:52.735498 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:52.735464 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 07:14:53 +0000 UTC" deadline="2028-05-01 19:36:01.311476245 +0000 UTC" Sep 17 07:19:52.735498 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:52.735491 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14220h16m8.575988038s" Sep 17 07:19:52.823177 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:52.823001 2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 07:19:52.832652 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:52.832634 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-215.ec2.internal" Sep 17 07:19:52.845962 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:52.845943 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 17 07:19:52.847791 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:52.847771 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-215.ec2.internal" Sep 17 07:19:52.857686 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:52.857657 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 17 07:19:52.870430 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:52.870413 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 07:19:52.874776 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:52.874757 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 07:19:52.877859 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:52.877833 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod910049e72d6e50062f9693db2cfac120.slice/crio-cee147b3b072563c88a6c8f2f4c25c4dc3971b61c9b4bba3557ee61f95001978 WatchSource:0}: Error finding container cee147b3b072563c88a6c8f2f4c25c4dc3971b61c9b4bba3557ee61f95001978: Status 404 returned error can't find the container with id cee147b3b072563c88a6c8f2f4c25c4dc3971b61c9b4bba3557ee61f95001978 Sep 17 07:19:52.882231 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:52.882214 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 07:19:55.174507 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.174474 2563 apiserver.go:52] "Watching apiserver" Sep 17 07:19:55.183451 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.183399 2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 17 07:19:55.184968 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.184936 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-k5dtg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-215.ec2.internal","openshift-multus/multus-additional-cni-plugins-2krnr","openshift-multus/network-metrics-daemon-x7484","openshift-network-diagnostics/network-check-target-m6v2q","kube-system/global-pull-secret-syncer-7f254","kube-system/konnectivity-agent-9zrxw","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv","openshift-cluster-node-tuning-operator/tuned-9gz52","openshift-multus/multus-rwlkw","openshift-network-operator/iptables-alerter-b42pw","openshift-ovn-kubernetes/ovnkube-node-68n7z","kube-system/kube-apiserver-proxy-ip-10-0-141-215.ec2.internal","openshift-dns/node-resolver-tnmcf"] Sep 17 07:19:55.187900 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.187880 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9zrxw" Sep 17 07:19:55.190088 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.190067 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b42pw" Sep 17 07:19:55.192315 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.192298 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2krnr" Sep 17 07:19:55.192863 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.192841 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 17 07:19:55.192863 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.192863 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 17 07:19:55.192972 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.192902 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-klcmb\"" Sep 17 07:19:55.194549 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.194531 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:19:55.194642 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:55.194596 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7484" podUID="69c30c82-b237-4fc3-a96c-61840e73b7a7" Sep 17 07:19:55.196733 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.196715 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:19:55.196830 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:55.196781 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v2q" podUID="f5733389-a2ac-47dd-9fb4-8ba44621adb9" Sep 17 07:19:55.198741 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.198718 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 17 07:19:55.198854 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.198812 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 17 07:19:55.198951 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.198930 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 17 07:19:55.199019 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.198951 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7sbtz\"" Sep 17 07:19:55.199087 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.199019 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 17 07:19:55.199143 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.199103 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 17 07:19:55.199257 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.199147 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 17 07:19:55.199257 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.199219 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 17 07:19:55.199403 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.199260 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 17 07:19:55.199473 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.199460 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:19:55.199529 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.199465 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vx6rr\"" Sep 17 07:19:55.199577 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:55.199514 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7f254" podUID="c285d69f-53b9-4e2a-a0d0-c72e4d592566" Sep 17 07:19:55.202127 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.201705 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.202127 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.201806 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 17 07:19:55.204177 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.204158 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tnmcf" Sep 17 07:19:55.206885 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.206864 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" Sep 17 07:19:55.208754 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.208355 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 17 07:19:55.208754 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.208368 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 17 07:19:55.208754 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.208437 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 17 07:19:55.208754 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.208504 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bz7p5\"" Sep 17 07:19:55.208754 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.208610 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6njvq\"" Sep 17 07:19:55.209041 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.208855 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.209389 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.209356 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 17 07:19:55.209491 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.209416 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 17 07:19:55.209552 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.209378 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 17 07:19:55.209552 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.209520 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7rfgk\"" Sep 17 07:19:55.210173 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.210156 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 17 07:19:55.211270 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.211173 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-k5dtg" Sep 17 07:19:55.213929 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.213910 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.214731 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.214709 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd894e06-a948-4955-a07a-dec7e32ca8b0-system-cni-dir\") pod \"multus-additional-cni-plugins-2krnr\" (UID: \"fd894e06-a948-4955-a07a-dec7e32ca8b0\") " pod="openshift-multus/multus-additional-cni-plugins-2krnr" Sep 17 07:19:55.214812 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.214755 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd894e06-a948-4955-a07a-dec7e32ca8b0-cni-binary-copy\") pod \"multus-additional-cni-plugins-2krnr\" (UID: \"fd894e06-a948-4955-a07a-dec7e32ca8b0\") " pod="openshift-multus/multus-additional-cni-plugins-2krnr" Sep 17 07:19:55.214812 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.214792 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd894e06-a948-4955-a07a-dec7e32ca8b0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2krnr\" (UID: \"fd894e06-a948-4955-a07a-dec7e32ca8b0\") " pod="openshift-multus/multus-additional-cni-plugins-2krnr" Sep 17 07:19:55.214934 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.214818 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs\") pod \"network-metrics-daemon-x7484\" (UID: \"69c30c82-b237-4fc3-a96c-61840e73b7a7\") " pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:19:55.214934 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.214841 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-run\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.214934 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.214865 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-host\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.214934 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.214889 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd894e06-a948-4955-a07a-dec7e32ca8b0-cnibin\") pod \"multus-additional-cni-plugins-2krnr\" (UID: \"fd894e06-a948-4955-a07a-dec7e32ca8b0\") " pod="openshift-multus/multus-additional-cni-plugins-2krnr" Sep 17 07:19:55.214934 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.214912 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh47g\" (UniqueName: \"kubernetes.io/projected/69c30c82-b237-4fc3-a96c-61840e73b7a7-kube-api-access-hh47g\") pod \"network-metrics-daemon-x7484\" (UID: \"69c30c82-b237-4fc3-a96c-61840e73b7a7\") " pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:19:55.215073 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.214934 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-etc-systemd\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.215073 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.214959 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-var-lib-kubelet\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.215073 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.214984 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-etc-tuned\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.215073 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.214991 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 17 07:19:55.215073 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215017 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-z5vrw\"" Sep 17 07:19:55.215073 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215010 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/47326f02-5916-4fbe-a826-71c298a22c4b-konnectivity-ca\") pod \"konnectivity-agent-9zrxw\" (UID: \"47326f02-5916-4fbe-a826-71c298a22c4b\") " pod="kube-system/konnectivity-agent-9zrxw" Sep 17 07:19:55.215073 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215051 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fd894e06-a948-4955-a07a-dec7e32ca8b0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2krnr\" (UID: \"fd894e06-a948-4955-a07a-dec7e32ca8b0\") " pod="openshift-multus/multus-additional-cni-plugins-2krnr" Sep 17 07:19:55.215261 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215074 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9c0e9d0a-00c1-479b-a811-68a9ca231e9f-hosts-file\") pod \"node-resolver-tnmcf\" (UID: \"9c0e9d0a-00c1-479b-a811-68a9ca231e9f\") " pod="openshift-dns/node-resolver-tnmcf" Sep 17 07:19:55.215261 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215120 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdx8z\" (UniqueName: \"kubernetes.io/projected/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-kube-api-access-jdx8z\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.215261 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215142 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/47326f02-5916-4fbe-a826-71c298a22c4b-agent-certs\") pod \"konnectivity-agent-9zrxw\" (UID: \"47326f02-5916-4fbe-a826-71c298a22c4b\") " pod="kube-system/konnectivity-agent-9zrxw" Sep 17 07:19:55.215261 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215164 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdcc5\" (UniqueName: \"kubernetes.io/projected/fd894e06-a948-4955-a07a-dec7e32ca8b0-kube-api-access-mdcc5\") pod \"multus-additional-cni-plugins-2krnr\" (UID: \"fd894e06-a948-4955-a07a-dec7e32ca8b0\") " pod="openshift-multus/multus-additional-cni-plugins-2krnr" Sep 17 07:19:55.215261 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215195 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr7f2\" (UniqueName: \"kubernetes.io/projected/f5733389-a2ac-47dd-9fb4-8ba44621adb9-kube-api-access-sr7f2\") pod \"network-check-target-m6v2q\" (UID: \"f5733389-a2ac-47dd-9fb4-8ba44621adb9\") " pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:19:55.215261 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215234 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-etc-sysconfig\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.216177 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215340 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-l4zmh\"" Sep 17 07:19:55.216177 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215424 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-lib-modules\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.216177 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215459 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/197c1697-1d3f-414c-bc8d-5919174461cd-host-slash\") pod \"iptables-alerter-b42pw\" (UID: \"197c1697-1d3f-414c-bc8d-5919174461cd\") " pod="openshift-network-operator/iptables-alerter-b42pw" Sep 17 07:19:55.216177 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215491 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-etc-modprobe-d\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.216177 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215516 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-etc-kubernetes\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.216177 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215539 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-tmp\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.216177 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215561 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/197c1697-1d3f-414c-bc8d-5919174461cd-iptables-alerter-script\") pod \"iptables-alerter-b42pw\" (UID: \"197c1697-1d3f-414c-bc8d-5919174461cd\") " pod="openshift-network-operator/iptables-alerter-b42pw" Sep 17 07:19:55.216177 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215628 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c0e9d0a-00c1-479b-a811-68a9ca231e9f-tmp-dir\") pod \"node-resolver-tnmcf\" (UID: \"9c0e9d0a-00c1-479b-a811-68a9ca231e9f\") " pod="openshift-dns/node-resolver-tnmcf" Sep 17 07:19:55.216177 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215676 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c285d69f-53b9-4e2a-a0d0-c72e4d592566-dbus\") pod \"global-pull-secret-syncer-7f254\" (UID: \"c285d69f-53b9-4e2a-a0d0-c72e4d592566\") " pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:19:55.216177 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215702 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c285d69f-53b9-4e2a-a0d0-c72e4d592566-original-pull-secret\") pod \"global-pull-secret-syncer-7f254\" (UID: \"c285d69f-53b9-4e2a-a0d0-c72e4d592566\") " pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:19:55.216177 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215721 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 17 07:19:55.216177 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215728 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kncs4\" (UniqueName: \"kubernetes.io/projected/197c1697-1d3f-414c-bc8d-5919174461cd-kube-api-access-kncs4\") pod \"iptables-alerter-b42pw\" (UID: \"197c1697-1d3f-414c-bc8d-5919174461cd\") " pod="openshift-network-operator/iptables-alerter-b42pw" Sep 17 07:19:55.216177 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215755 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqzr8\" (UniqueName: \"kubernetes.io/projected/9c0e9d0a-00c1-479b-a811-68a9ca231e9f-kube-api-access-xqzr8\") pod \"node-resolver-tnmcf\" (UID: \"9c0e9d0a-00c1-479b-a811-68a9ca231e9f\") " pod="openshift-dns/node-resolver-tnmcf" Sep 17 07:19:55.216177 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215798 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c285d69f-53b9-4e2a-a0d0-c72e4d592566-kubelet-config\") pod \"global-pull-secret-syncer-7f254\" (UID: \"c285d69f-53b9-4e2a-a0d0-c72e4d592566\") " pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:19:55.216177 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215819 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-etc-sysctl-d\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.216177 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215832 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-etc-sysctl-conf\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.216177 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215849 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fd894e06-a948-4955-a07a-dec7e32ca8b0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2krnr\" (UID: \"fd894e06-a948-4955-a07a-dec7e32ca8b0\") " pod="openshift-multus/multus-additional-cni-plugins-2krnr" Sep 17 07:19:55.217041 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215879 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-sys\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.217041 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215952 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 17 07:19:55.217041 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.215986 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 17 07:19:55.217041 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.216466 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 17 07:19:55.217041 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.217226 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 17 07:19:55.219056 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.219033 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 17 07:19:55.219167 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.219100 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wbdk4\"" Sep 17 07:19:55.219167 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.219113 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 17 07:19:55.219270 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.219191 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 17 07:19:55.219500 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.219484 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 17 07:19:55.223651 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.223629 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 17 07:19:55.248160 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.248140 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-z654h" Sep 17 07:19:55.257146 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.257120 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-z654h" Sep 17 07:19:55.303591 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.303561 2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 17 07:19:55.314831 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.314780 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-215.ec2.internal" event={"ID":"910049e72d6e50062f9693db2cfac120","Type":"ContainerStarted","Data":"cee147b3b072563c88a6c8f2f4c25c4dc3971b61c9b4bba3557ee61f95001978"} Sep 17 07:19:55.316081 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316049 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c285d69f-53b9-4e2a-a0d0-c72e4d592566-kubelet-config\") pod \"global-pull-secret-syncer-7f254\" (UID: \"c285d69f-53b9-4e2a-a0d0-c72e4d592566\") " pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:19:55.316197 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316099 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-etc-openvswitch\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.316197 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316127 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.316197 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316156 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5861ec16-ae09-4421-a393-fe23e69fa0cd-ovn-node-metrics-cert\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.316197 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316169 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c285d69f-53b9-4e2a-a0d0-c72e4d592566-kubelet-config\") pod \"global-pull-secret-syncer-7f254\" (UID: \"c285d69f-53b9-4e2a-a0d0-c72e4d592566\") " pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:19:55.316197 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316182 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-sys\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.316474 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316226 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd894e06-a948-4955-a07a-dec7e32ca8b0-system-cni-dir\") pod \"multus-additional-cni-plugins-2krnr\" (UID: \"fd894e06-a948-4955-a07a-dec7e32ca8b0\") " pod="openshift-multus/multus-additional-cni-plugins-2krnr" Sep 17 07:19:55.316474 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316252 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd894e06-a948-4955-a07a-dec7e32ca8b0-cni-binary-copy\") pod \"multus-additional-cni-plugins-2krnr\" (UID: \"fd894e06-a948-4955-a07a-dec7e32ca8b0\") " pod="openshift-multus/multus-additional-cni-plugins-2krnr" Sep 17 07:19:55.316474 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316260 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-sys\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.316474 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316289 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ddbb72ee-6e56-4549-ac75-f8857a377394-serviceca\") pod \"node-ca-k5dtg\" (UID: \"ddbb72ee-6e56-4549-ac75-f8857a377394\") " pod="openshift-image-registry/node-ca-k5dtg" Sep 17 07:19:55.316474 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316292 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-215.ec2.internal" event={"ID":"c5668cb859d7da5d03909e7e9730b526","Type":"ContainerStarted","Data":"748484dda76cd73c2c07af336d92532ec1ee54fbe12533a4ce4696cdfedb6e8d"} Sep 17 07:19:55.316474 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316307 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-run-ovn\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.316474 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316343 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs\") pod \"network-metrics-daemon-x7484\" (UID: \"69c30c82-b237-4fc3-a96c-61840e73b7a7\") " pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:19:55.316474 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316347 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd894e06-a948-4955-a07a-dec7e32ca8b0-system-cni-dir\") pod \"multus-additional-cni-plugins-2krnr\" (UID: \"fd894e06-a948-4955-a07a-dec7e32ca8b0\") " pod="openshift-multus/multus-additional-cni-plugins-2krnr" Sep 17 07:19:55.316474 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316367 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-var-lib-openvswitch\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.316474 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316408 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z2bz\" (UniqueName: \"kubernetes.io/projected/8c5c5699-5a0e-46f6-8c5a-14cb86164285-kube-api-access-6z2bz\") pod \"aws-ebs-csi-driver-node-nhpgv\" (UID: \"8c5c5699-5a0e-46f6-8c5a-14cb86164285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" Sep 17 07:19:55.316474 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:55.316439 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:19:55.316474 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316441 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-cnibin\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.317026 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:55.316496 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs podName:69c30c82-b237-4fc3-a96c-61840e73b7a7 nodeName:}" failed. No retries permitted until 2026-09-17 07:19:55.816478814 +0000 UTC m=+3.002211764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs") pod "network-metrics-daemon-x7484" (UID: "69c30c82-b237-4fc3-a96c-61840e73b7a7") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:19:55.317026 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316531 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-host-run-netns\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.317026 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316558 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggkg7\" (UniqueName: \"kubernetes.io/projected/0f66f8d4-3fa1-461b-8925-f68c138aafd2-kube-api-access-ggkg7\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.317026 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316592 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-etc-systemd\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.317026 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316618 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-var-lib-kubelet\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.317026 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316644 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-etc-tuned\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.317026 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316648 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-etc-systemd\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.317026 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316673 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/47326f02-5916-4fbe-a826-71c298a22c4b-konnectivity-ca\") pod \"konnectivity-agent-9zrxw\" (UID: \"47326f02-5916-4fbe-a826-71c298a22c4b\") " pod="kube-system/konnectivity-agent-9zrxw" Sep 17 07:19:55.317026 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316700 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9c0e9d0a-00c1-479b-a811-68a9ca231e9f-hosts-file\") pod \"node-resolver-tnmcf\" (UID: \"9c0e9d0a-00c1-479b-a811-68a9ca231e9f\") " pod="openshift-dns/node-resolver-tnmcf" Sep 17 07:19:55.317026 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316728 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-log-socket\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.317026 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316675 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-var-lib-kubelet\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.317026 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316755 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkxt9\" (UniqueName: \"kubernetes.io/projected/5861ec16-ae09-4421-a393-fe23e69fa0cd-kube-api-access-vkxt9\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.317026 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316781 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8c5c5699-5a0e-46f6-8c5a-14cb86164285-device-dir\") pod \"aws-ebs-csi-driver-node-nhpgv\" (UID: \"8c5c5699-5a0e-46f6-8c5a-14cb86164285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" Sep 17 07:19:55.317026 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316810 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr7f2\" (UniqueName: \"kubernetes.io/projected/f5733389-a2ac-47dd-9fb4-8ba44621adb9-kube-api-access-sr7f2\") pod \"network-check-target-m6v2q\" (UID: \"f5733389-a2ac-47dd-9fb4-8ba44621adb9\") " pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:19:55.317026 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316835 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-host-run-netns\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.317026 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316862 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8c5c5699-5a0e-46f6-8c5a-14cb86164285-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nhpgv\" (UID: \"8c5c5699-5a0e-46f6-8c5a-14cb86164285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" Sep 17 07:19:55.317814 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316895 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd894e06-a948-4955-a07a-dec7e32ca8b0-cni-binary-copy\") pod \"multus-additional-cni-plugins-2krnr\" (UID: \"fd894e06-a948-4955-a07a-dec7e32ca8b0\") " pod="openshift-multus/multus-additional-cni-plugins-2krnr" Sep 17 07:19:55.317814 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316912 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-hostroot\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.317814 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316937 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-host-run-multus-certs\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.317814 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.316971 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-node-log\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.317814 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317004 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9c0e9d0a-00c1-479b-a811-68a9ca231e9f-hosts-file\") pod \"node-resolver-tnmcf\" (UID: \"9c0e9d0a-00c1-479b-a811-68a9ca231e9f\") " pod="openshift-dns/node-resolver-tnmcf" Sep 17 07:19:55.317814 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317008 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5861ec16-ae09-4421-a393-fe23e69fa0cd-env-overrides\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.317814 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317045 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-etc-modprobe-d\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.317814 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317070 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-etc-kubernetes\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.317814 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317149 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/47326f02-5916-4fbe-a826-71c298a22c4b-konnectivity-ca\") pod \"konnectivity-agent-9zrxw\" (UID: \"47326f02-5916-4fbe-a826-71c298a22c4b\") " pod="kube-system/konnectivity-agent-9zrxw" Sep 17 07:19:55.317814 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317181 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-etc-kubernetes\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.317814 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317208 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-tmp\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.317814 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317235 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-host-cni-netd\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.317814 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317261 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5861ec16-ae09-4421-a393-fe23e69fa0cd-ovnkube-script-lib\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.317814 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317286 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-host-var-lib-cni-bin\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.317814 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317235 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-etc-modprobe-d\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.317814 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317342 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c285d69f-53b9-4e2a-a0d0-c72e4d592566-original-pull-secret\") pod \"global-pull-secret-syncer-7f254\" (UID: \"c285d69f-53b9-4e2a-a0d0-c72e4d592566\") " pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:19:55.317814 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317371 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kncs4\" (UniqueName: \"kubernetes.io/projected/197c1697-1d3f-414c-bc8d-5919174461cd-kube-api-access-kncs4\") pod \"iptables-alerter-b42pw\" (UID: \"197c1697-1d3f-414c-bc8d-5919174461cd\") " pod="openshift-network-operator/iptables-alerter-b42pw" Sep 17 07:19:55.318630 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317400 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5861ec16-ae09-4421-a393-fe23e69fa0cd-ovnkube-config\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.318630 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317424 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-system-cni-dir\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.318630 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317463 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-etc-sysctl-d\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.318630 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317490 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-etc-sysctl-conf\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.318630 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317535 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fd894e06-a948-4955-a07a-dec7e32ca8b0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2krnr\" (UID: \"fd894e06-a948-4955-a07a-dec7e32ca8b0\") " pod="openshift-multus/multus-additional-cni-plugins-2krnr" Sep 17 07:19:55.318630 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317575 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-host-slash\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.318630 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317600 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-run-openvswitch\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.318630 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317635 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-multus-cni-dir\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.318630 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:55.317656 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:19:55.318630 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317670 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-multus-socket-dir-parent\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.318630 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:55.317703 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c285d69f-53b9-4e2a-a0d0-c72e4d592566-original-pull-secret podName:c285d69f-53b9-4e2a-a0d0-c72e4d592566 nodeName:}" failed. No retries permitted until 2026-09-17 07:19:55.817682364 +0000 UTC m=+3.003415314 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c285d69f-53b9-4e2a-a0d0-c72e4d592566-original-pull-secret") pod "global-pull-secret-syncer-7f254" (UID: "c285d69f-53b9-4e2a-a0d0-c72e4d592566") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:19:55.318630 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317745 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd894e06-a948-4955-a07a-dec7e32ca8b0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2krnr\" (UID: \"fd894e06-a948-4955-a07a-dec7e32ca8b0\") " pod="openshift-multus/multus-additional-cni-plugins-2krnr" Sep 17 07:19:55.318630 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317771 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-host-cni-bin\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.318630 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317788 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-host-var-lib-cni-multus\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.318630 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317789 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-etc-sysctl-conf\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.318630 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317808 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-run\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.318630 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317838 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-host\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.319487 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317863 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd894e06-a948-4955-a07a-dec7e32ca8b0-cnibin\") pod \"multus-additional-cni-plugins-2krnr\" (UID: \"fd894e06-a948-4955-a07a-dec7e32ca8b0\") " pod="openshift-multus/multus-additional-cni-plugins-2krnr" Sep 17 07:19:55.319487 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317889 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-run\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.319487 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317889 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8c5c5699-5a0e-46f6-8c5a-14cb86164285-socket-dir\") pod \"aws-ebs-csi-driver-node-nhpgv\" (UID: \"8c5c5699-5a0e-46f6-8c5a-14cb86164285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" Sep 17 07:19:55.319487 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317918 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8c5c5699-5a0e-46f6-8c5a-14cb86164285-etc-selinux\") pod \"aws-ebs-csi-driver-node-nhpgv\" (UID: \"8c5c5699-5a0e-46f6-8c5a-14cb86164285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" Sep 17 07:19:55.319487 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317927 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-etc-sysctl-d\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.319487 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317934 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-host-run-k8s-cni-cncf-io\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.319487 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317940 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-host\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.319487 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317863 2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 17 07:19:55.319487 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317967 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0f66f8d4-3fa1-461b-8925-f68c138aafd2-multus-daemon-config\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.319487 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.317997 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd894e06-a948-4955-a07a-dec7e32ca8b0-cnibin\") pod \"multus-additional-cni-plugins-2krnr\" (UID: \"fd894e06-a948-4955-a07a-dec7e32ca8b0\") " pod="openshift-multus/multus-additional-cni-plugins-2krnr" Sep 17 07:19:55.319487 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318020 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh47g\" (UniqueName: \"kubernetes.io/projected/69c30c82-b237-4fc3-a96c-61840e73b7a7-kube-api-access-hh47g\") pod \"network-metrics-daemon-x7484\" (UID: \"69c30c82-b237-4fc3-a96c-61840e73b7a7\") " pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:19:55.319487 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318038 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fd894e06-a948-4955-a07a-dec7e32ca8b0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2krnr\" (UID: \"fd894e06-a948-4955-a07a-dec7e32ca8b0\") " pod="openshift-multus/multus-additional-cni-plugins-2krnr" Sep 17 07:19:55.319487 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318037 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd894e06-a948-4955-a07a-dec7e32ca8b0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2krnr\" (UID: \"fd894e06-a948-4955-a07a-dec7e32ca8b0\") " pod="openshift-multus/multus-additional-cni-plugins-2krnr" Sep 17 07:19:55.319487 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318063 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-host-kubelet\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.319487 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318079 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-host-var-lib-kubelet\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.319487 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318099 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdx8z\" (UniqueName: \"kubernetes.io/projected/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-kube-api-access-jdx8z\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.319487 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318114 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/47326f02-5916-4fbe-a826-71c298a22c4b-agent-certs\") pod \"konnectivity-agent-9zrxw\" (UID: \"47326f02-5916-4fbe-a826-71c298a22c4b\") " pod="kube-system/konnectivity-agent-9zrxw" Sep 17 07:19:55.320102 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318131 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdcc5\" (UniqueName: \"kubernetes.io/projected/fd894e06-a948-4955-a07a-dec7e32ca8b0-kube-api-access-mdcc5\") pod \"multus-additional-cni-plugins-2krnr\" (UID: \"fd894e06-a948-4955-a07a-dec7e32ca8b0\") " pod="openshift-multus/multus-additional-cni-plugins-2krnr" Sep 17 07:19:55.320102 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318145 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-etc-sysconfig\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.320102 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318160 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-lib-modules\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.320102 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318191 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/197c1697-1d3f-414c-bc8d-5919174461cd-host-slash\") pod \"iptables-alerter-b42pw\" (UID: \"197c1697-1d3f-414c-bc8d-5919174461cd\") " pod="openshift-network-operator/iptables-alerter-b42pw" Sep 17 07:19:55.320102 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318206 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2bvg\" (UniqueName: \"kubernetes.io/projected/ddbb72ee-6e56-4549-ac75-f8857a377394-kube-api-access-v2bvg\") pod \"node-ca-k5dtg\" (UID: \"ddbb72ee-6e56-4549-ac75-f8857a377394\") " pod="openshift-image-registry/node-ca-k5dtg" Sep 17 07:19:55.320102 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318236 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-systemd-units\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.320102 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318237 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fd894e06-a948-4955-a07a-dec7e32ca8b0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2krnr\" (UID: \"fd894e06-a948-4955-a07a-dec7e32ca8b0\") " pod="openshift-multus/multus-additional-cni-plugins-2krnr" Sep 17 07:19:55.320102 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318255 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-run-systemd\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.320102 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318266 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-lib-modules\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.320102 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318301 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-etc-sysconfig\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.320102 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318366 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.320102 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318402 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-multus-conf-dir\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.320102 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318439 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/197c1697-1d3f-414c-bc8d-5919174461cd-iptables-alerter-script\") pod \"iptables-alerter-b42pw\" (UID: \"197c1697-1d3f-414c-bc8d-5919174461cd\") " pod="openshift-network-operator/iptables-alerter-b42pw" Sep 17 07:19:55.320102 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318545 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/197c1697-1d3f-414c-bc8d-5919174461cd-host-slash\") pod \"iptables-alerter-b42pw\" (UID: \"197c1697-1d3f-414c-bc8d-5919174461cd\") " pod="openshift-network-operator/iptables-alerter-b42pw" Sep 17 07:19:55.320102 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318560 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c0e9d0a-00c1-479b-a811-68a9ca231e9f-tmp-dir\") pod \"node-resolver-tnmcf\" (UID: \"9c0e9d0a-00c1-479b-a811-68a9ca231e9f\") " pod="openshift-dns/node-resolver-tnmcf" Sep 17 07:19:55.320102 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318765 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddbb72ee-6e56-4549-ac75-f8857a377394-host\") pod \"node-ca-k5dtg\" (UID: \"ddbb72ee-6e56-4549-ac75-f8857a377394\") " pod="openshift-image-registry/node-ca-k5dtg" Sep 17 07:19:55.320102 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318832 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8c5c5699-5a0e-46f6-8c5a-14cb86164285-registration-dir\") pod \"aws-ebs-csi-driver-node-nhpgv\" (UID: \"8c5c5699-5a0e-46f6-8c5a-14cb86164285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" Sep 17 07:19:55.320592 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318873 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8c5c5699-5a0e-46f6-8c5a-14cb86164285-sys-fs\") pod \"aws-ebs-csi-driver-node-nhpgv\" (UID: \"8c5c5699-5a0e-46f6-8c5a-14cb86164285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" Sep 17 07:19:55.320592 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.318978 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/197c1697-1d3f-414c-bc8d-5919174461cd-iptables-alerter-script\") pod \"iptables-alerter-b42pw\" (UID: \"197c1697-1d3f-414c-bc8d-5919174461cd\") " pod="openshift-network-operator/iptables-alerter-b42pw" Sep 17 07:19:55.320592 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.319030 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f66f8d4-3fa1-461b-8925-f68c138aafd2-cni-binary-copy\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.320592 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.319057 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c285d69f-53b9-4e2a-a0d0-c72e4d592566-dbus\") pod \"global-pull-secret-syncer-7f254\" (UID: \"c285d69f-53b9-4e2a-a0d0-c72e4d592566\") " pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:19:55.320592 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.319086 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqzr8\" (UniqueName: \"kubernetes.io/projected/9c0e9d0a-00c1-479b-a811-68a9ca231e9f-kube-api-access-xqzr8\") pod \"node-resolver-tnmcf\" (UID: \"9c0e9d0a-00c1-479b-a811-68a9ca231e9f\") " pod="openshift-dns/node-resolver-tnmcf" Sep 17 07:19:55.320592 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.319184 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-etc-kubernetes\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.320592 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.319234 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c0e9d0a-00c1-479b-a811-68a9ca231e9f-tmp-dir\") pod \"node-resolver-tnmcf\" (UID: \"9c0e9d0a-00c1-479b-a811-68a9ca231e9f\") " pod="openshift-dns/node-resolver-tnmcf" Sep 17 07:19:55.320592 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.319259 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fd894e06-a948-4955-a07a-dec7e32ca8b0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2krnr\" (UID: \"fd894e06-a948-4955-a07a-dec7e32ca8b0\") " pod="openshift-multus/multus-additional-cni-plugins-2krnr" Sep 17 07:19:55.320592 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.319401 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c285d69f-53b9-4e2a-a0d0-c72e4d592566-dbus\") pod \"global-pull-secret-syncer-7f254\" (UID: \"c285d69f-53b9-4e2a-a0d0-c72e4d592566\") " pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:19:55.321798 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.321773 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-etc-tuned\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.321884 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.321828 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-tmp\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.321996 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.321976 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/47326f02-5916-4fbe-a826-71c298a22c4b-agent-certs\") pod \"konnectivity-agent-9zrxw\" (UID: \"47326f02-5916-4fbe-a826-71c298a22c4b\") " pod="kube-system/konnectivity-agent-9zrxw" Sep 17 07:19:55.329231 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:55.329206 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:19:55.329383 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:55.329234 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:19:55.329383 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:55.329260 2563 projected.go:196] Error preparing data for projected volume kube-api-access-sr7f2 for pod openshift-network-diagnostics/network-check-target-m6v2q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:19:55.329383 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:55.329333 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5733389-a2ac-47dd-9fb4-8ba44621adb9-kube-api-access-sr7f2 podName:f5733389-a2ac-47dd-9fb4-8ba44621adb9 nodeName:}" failed. No retries permitted until 2026-09-17 07:19:55.829301319 +0000 UTC m=+3.015034285 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sr7f2" (UniqueName: "kubernetes.io/projected/f5733389-a2ac-47dd-9fb4-8ba44621adb9-kube-api-access-sr7f2") pod "network-check-target-m6v2q" (UID: "f5733389-a2ac-47dd-9fb4-8ba44621adb9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:19:55.330055 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.329910 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kncs4\" (UniqueName: \"kubernetes.io/projected/197c1697-1d3f-414c-bc8d-5919174461cd-kube-api-access-kncs4\") pod \"iptables-alerter-b42pw\" (UID: \"197c1697-1d3f-414c-bc8d-5919174461cd\") " pod="openshift-network-operator/iptables-alerter-b42pw" Sep 17 07:19:55.332138 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.332094 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdcc5\" (UniqueName: \"kubernetes.io/projected/fd894e06-a948-4955-a07a-dec7e32ca8b0-kube-api-access-mdcc5\") pod \"multus-additional-cni-plugins-2krnr\" (UID: \"fd894e06-a948-4955-a07a-dec7e32ca8b0\") " pod="openshift-multus/multus-additional-cni-plugins-2krnr" Sep 17 07:19:55.332482 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.332461 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqzr8\" (UniqueName: \"kubernetes.io/projected/9c0e9d0a-00c1-479b-a811-68a9ca231e9f-kube-api-access-xqzr8\") pod \"node-resolver-tnmcf\" (UID: \"9c0e9d0a-00c1-479b-a811-68a9ca231e9f\") " pod="openshift-dns/node-resolver-tnmcf" Sep 17 07:19:55.332752 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.332733 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdx8z\" (UniqueName: \"kubernetes.io/projected/9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3-kube-api-access-jdx8z\") pod \"tuned-9gz52\" (UID: \"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3\") " pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.333911 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.333893 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh47g\" (UniqueName: \"kubernetes.io/projected/69c30c82-b237-4fc3-a96c-61840e73b7a7-kube-api-access-hh47g\") pod \"network-metrics-daemon-x7484\" (UID: \"69c30c82-b237-4fc3-a96c-61840e73b7a7\") " pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:19:55.422462 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.419962 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2bvg\" (UniqueName: \"kubernetes.io/projected/ddbb72ee-6e56-4549-ac75-f8857a377394-kube-api-access-v2bvg\") pod \"node-ca-k5dtg\" (UID: \"ddbb72ee-6e56-4549-ac75-f8857a377394\") " pod="openshift-image-registry/node-ca-k5dtg" Sep 17 07:19:55.422462 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420020 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-systemd-units\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.422462 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420048 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-run-systemd\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.422462 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420078 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.422462 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420184 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-multus-conf-dir\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.422462 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420207 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddbb72ee-6e56-4549-ac75-f8857a377394-host\") pod \"node-ca-k5dtg\" (UID: \"ddbb72ee-6e56-4549-ac75-f8857a377394\") " pod="openshift-image-registry/node-ca-k5dtg" Sep 17 07:19:55.422462 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420235 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8c5c5699-5a0e-46f6-8c5a-14cb86164285-registration-dir\") pod \"aws-ebs-csi-driver-node-nhpgv\" (UID: \"8c5c5699-5a0e-46f6-8c5a-14cb86164285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" Sep 17 07:19:55.422462 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420287 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8c5c5699-5a0e-46f6-8c5a-14cb86164285-sys-fs\") pod \"aws-ebs-csi-driver-node-nhpgv\" (UID: \"8c5c5699-5a0e-46f6-8c5a-14cb86164285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" Sep 17 07:19:55.422462 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420343 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f66f8d4-3fa1-461b-8925-f68c138aafd2-cni-binary-copy\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.422462 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420372 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-etc-kubernetes\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.422462 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420427 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-etc-openvswitch\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.422462 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420455 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.422462 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420483 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5861ec16-ae09-4421-a393-fe23e69fa0cd-ovn-node-metrics-cert\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.422462 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420521 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ddbb72ee-6e56-4549-ac75-f8857a377394-serviceca\") pod \"node-ca-k5dtg\" (UID: \"ddbb72ee-6e56-4549-ac75-f8857a377394\") " pod="openshift-image-registry/node-ca-k5dtg" Sep 17 07:19:55.422462 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420546 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-run-ovn\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.422462 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420593 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-var-lib-openvswitch\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.422462 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420623 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z2bz\" (UniqueName: \"kubernetes.io/projected/8c5c5699-5a0e-46f6-8c5a-14cb86164285-kube-api-access-6z2bz\") pod \"aws-ebs-csi-driver-node-nhpgv\" (UID: \"8c5c5699-5a0e-46f6-8c5a-14cb86164285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" Sep 17 07:19:55.423127 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420648 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-cnibin\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.423127 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420689 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-host-run-netns\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.423127 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420717 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggkg7\" (UniqueName: \"kubernetes.io/projected/0f66f8d4-3fa1-461b-8925-f68c138aafd2-kube-api-access-ggkg7\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.423127 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420751 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-log-socket\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.423127 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420784 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkxt9\" (UniqueName: \"kubernetes.io/projected/5861ec16-ae09-4421-a393-fe23e69fa0cd-kube-api-access-vkxt9\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.423127 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420810 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8c5c5699-5a0e-46f6-8c5a-14cb86164285-device-dir\") pod \"aws-ebs-csi-driver-node-nhpgv\" (UID: \"8c5c5699-5a0e-46f6-8c5a-14cb86164285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" Sep 17 07:19:55.423127 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420856 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-host-run-netns\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.423127 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420898 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8c5c5699-5a0e-46f6-8c5a-14cb86164285-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nhpgv\" (UID: \"8c5c5699-5a0e-46f6-8c5a-14cb86164285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" Sep 17 07:19:55.423127 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420934 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-hostroot\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.423127 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420965 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-host-run-multus-certs\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.423127 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.420994 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-node-log\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.423127 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421028 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5861ec16-ae09-4421-a393-fe23e69fa0cd-env-overrides\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.423127 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421056 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-host-cni-netd\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.423127 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421085 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5861ec16-ae09-4421-a393-fe23e69fa0cd-ovnkube-script-lib\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.423127 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421116 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-host-var-lib-cni-bin\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.423127 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421162 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5861ec16-ae09-4421-a393-fe23e69fa0cd-ovnkube-config\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.423127 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421190 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-system-cni-dir\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.423127 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421216 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-host-slash\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.423959 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421243 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-run-openvswitch\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.423959 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421269 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-multus-cni-dir\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.423959 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421298 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-multus-socket-dir-parent\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.423959 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421347 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-host-cni-bin\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.423959 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421371 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-host-var-lib-cni-multus\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.423959 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421403 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8c5c5699-5a0e-46f6-8c5a-14cb86164285-socket-dir\") pod \"aws-ebs-csi-driver-node-nhpgv\" (UID: \"8c5c5699-5a0e-46f6-8c5a-14cb86164285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" Sep 17 07:19:55.423959 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421435 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8c5c5699-5a0e-46f6-8c5a-14cb86164285-etc-selinux\") pod \"aws-ebs-csi-driver-node-nhpgv\" (UID: \"8c5c5699-5a0e-46f6-8c5a-14cb86164285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" Sep 17 07:19:55.423959 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421561 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.423959 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421576 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-host-run-netns\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.423959 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421504 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-etc-openvswitch\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.423959 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421566 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-run-systemd\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.423959 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421504 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-log-socket\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.423959 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421585 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-host-run-k8s-cni-cncf-io\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.423959 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421652 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8c5c5699-5a0e-46f6-8c5a-14cb86164285-sys-fs\") pod \"aws-ebs-csi-driver-node-nhpgv\" (UID: \"8c5c5699-5a0e-46f6-8c5a-14cb86164285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" Sep 17 07:19:55.423959 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421665 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-run-ovn\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.423959 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421673 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-node-log\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.423959 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421708 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-systemd-units\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.423959 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421733 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-cnibin\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.424847 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421738 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-var-lib-openvswitch\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.424847 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421760 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-host-cni-netd\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.424847 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421860 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-host-run-k8s-cni-cncf-io\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.424847 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421881 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-host-slash\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.424847 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421899 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-multus-cni-dir\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.424847 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421924 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-host-cni-bin\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.424847 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421949 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddbb72ee-6e56-4549-ac75-f8857a377394-host\") pod \"node-ca-k5dtg\" (UID: \"ddbb72ee-6e56-4549-ac75-f8857a377394\") " pod="openshift-image-registry/node-ca-k5dtg" Sep 17 07:19:55.424847 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421974 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8c5c5699-5a0e-46f6-8c5a-14cb86164285-socket-dir\") pod \"aws-ebs-csi-driver-node-nhpgv\" (UID: \"8c5c5699-5a0e-46f6-8c5a-14cb86164285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" Sep 17 07:19:55.424847 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.421504 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-etc-kubernetes\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.424847 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.422040 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-host-var-lib-cni-bin\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.424847 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.422067 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8c5c5699-5a0e-46f6-8c5a-14cb86164285-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nhpgv\" (UID: \"8c5c5699-5a0e-46f6-8c5a-14cb86164285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" Sep 17 07:19:55.424847 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.422072 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0f66f8d4-3fa1-461b-8925-f68c138aafd2-multus-daemon-config\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.424847 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.422079 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ddbb72ee-6e56-4549-ac75-f8857a377394-serviceca\") pod \"node-ca-k5dtg\" (UID: \"ddbb72ee-6e56-4549-ac75-f8857a377394\") " pod="openshift-image-registry/node-ca-k5dtg" Sep 17 07:19:55.424847 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.422112 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-host-kubelet\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.424847 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.422144 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8c5c5699-5a0e-46f6-8c5a-14cb86164285-device-dir\") pod \"aws-ebs-csi-driver-node-nhpgv\" (UID: \"8c5c5699-5a0e-46f6-8c5a-14cb86164285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" Sep 17 07:19:55.424847 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.422143 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-host-var-lib-kubelet\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.424847 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.422180 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-host-var-lib-kubelet\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.424847 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.422401 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-hostroot\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.425710 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.422422 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-multus-socket-dir-parent\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.425710 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.422464 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f66f8d4-3fa1-461b-8925-f68c138aafd2-cni-binary-copy\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.425710 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.422755 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-system-cni-dir\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.425710 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.422874 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.425710 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.422907 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5861ec16-ae09-4421-a393-fe23e69fa0cd-ovnkube-script-lib\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.425710 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.422935 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8c5c5699-5a0e-46f6-8c5a-14cb86164285-etc-selinux\") pod \"aws-ebs-csi-driver-node-nhpgv\" (UID: \"8c5c5699-5a0e-46f6-8c5a-14cb86164285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" Sep 17 07:19:55.425710 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.422991 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5861ec16-ae09-4421-a393-fe23e69fa0cd-ovnkube-config\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.425710 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.422992 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-host-var-lib-cni-multus\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.425710 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.423051 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8c5c5699-5a0e-46f6-8c5a-14cb86164285-registration-dir\") pod \"aws-ebs-csi-driver-node-nhpgv\" (UID: \"8c5c5699-5a0e-46f6-8c5a-14cb86164285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" Sep 17 07:19:55.425710 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.423356 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-run-openvswitch\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.425710 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.423427 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-host-run-netns\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.425710 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.423479 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-multus-conf-dir\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.425710 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.423526 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0f66f8d4-3fa1-461b-8925-f68c138aafd2-host-run-multus-certs\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.425710 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.423570 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5861ec16-ae09-4421-a393-fe23e69fa0cd-host-kubelet\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.425710 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.423683 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0f66f8d4-3fa1-461b-8925-f68c138aafd2-multus-daemon-config\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.425710 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.423763 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5861ec16-ae09-4421-a393-fe23e69fa0cd-env-overrides\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.425710 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.424636 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5861ec16-ae09-4421-a393-fe23e69fa0cd-ovn-node-metrics-cert\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.433498 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.433471 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggkg7\" (UniqueName: \"kubernetes.io/projected/0f66f8d4-3fa1-461b-8925-f68c138aafd2-kube-api-access-ggkg7\") pod \"multus-rwlkw\" (UID: \"0f66f8d4-3fa1-461b-8925-f68c138aafd2\") " pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.433617 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.433501 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkxt9\" (UniqueName: \"kubernetes.io/projected/5861ec16-ae09-4421-a393-fe23e69fa0cd-kube-api-access-vkxt9\") pod \"ovnkube-node-68n7z\" (UID: \"5861ec16-ae09-4421-a393-fe23e69fa0cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.436351 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.436312 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z2bz\" (UniqueName: \"kubernetes.io/projected/8c5c5699-5a0e-46f6-8c5a-14cb86164285-kube-api-access-6z2bz\") pod \"aws-ebs-csi-driver-node-nhpgv\" (UID: \"8c5c5699-5a0e-46f6-8c5a-14cb86164285\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" Sep 17 07:19:55.452817 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.452793 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2bvg\" (UniqueName: \"kubernetes.io/projected/ddbb72ee-6e56-4549-ac75-f8857a377394-kube-api-access-v2bvg\") pod \"node-ca-k5dtg\" (UID: \"ddbb72ee-6e56-4549-ac75-f8857a377394\") " pod="openshift-image-registry/node-ca-k5dtg" Sep 17 07:19:55.488851 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.488826 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 07:19:55.498230 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.498205 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9zrxw" Sep 17 07:19:55.506617 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.506593 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b42pw" Sep 17 07:19:55.519264 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.519241 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2krnr" Sep 17 07:19:55.529138 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.529114 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9gz52" Sep 17 07:19:55.536802 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.536780 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tnmcf" Sep 17 07:19:55.545407 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.545389 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" Sep 17 07:19:55.554031 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.554013 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rwlkw" Sep 17 07:19:55.560718 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.560701 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-k5dtg" Sep 17 07:19:55.566384 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.566365 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:19:55.825696 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.825598 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs\") pod \"network-metrics-daemon-x7484\" (UID: \"69c30c82-b237-4fc3-a96c-61840e73b7a7\") " pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:19:55.825696 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.825668 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c285d69f-53b9-4e2a-a0d0-c72e4d592566-original-pull-secret\") pod \"global-pull-secret-syncer-7f254\" (UID: \"c285d69f-53b9-4e2a-a0d0-c72e4d592566\") " pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:19:55.825925 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:55.825769 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:19:55.825925 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:55.825784 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:19:55.825925 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:55.825846 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs podName:69c30c82-b237-4fc3-a96c-61840e73b7a7 nodeName:}" failed. No retries permitted until 2026-09-17 07:19:56.825826421 +0000 UTC m=+4.011559385 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs") pod "network-metrics-daemon-x7484" (UID: "69c30c82-b237-4fc3-a96c-61840e73b7a7") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:19:55.825925 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:55.825863 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c285d69f-53b9-4e2a-a0d0-c72e4d592566-original-pull-secret podName:c285d69f-53b9-4e2a-a0d0-c72e4d592566 nodeName:}" failed. No retries permitted until 2026-09-17 07:19:56.825855933 +0000 UTC m=+4.011588883 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c285d69f-53b9-4e2a-a0d0-c72e4d592566-original-pull-secret") pod "global-pull-secret-syncer-7f254" (UID: "c285d69f-53b9-4e2a-a0d0-c72e4d592566") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:19:55.926079 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:55.926032 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr7f2\" (UniqueName: \"kubernetes.io/projected/f5733389-a2ac-47dd-9fb4-8ba44621adb9-kube-api-access-sr7f2\") pod \"network-check-target-m6v2q\" (UID: \"f5733389-a2ac-47dd-9fb4-8ba44621adb9\") " pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:19:55.926260 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:55.926187 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:19:55.926260 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:55.926209 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:19:55.926260 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:55.926222 2563 projected.go:196] Error preparing data for projected volume kube-api-access-sr7f2 for pod openshift-network-diagnostics/network-check-target-m6v2q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:19:55.926451 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:55.926286 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5733389-a2ac-47dd-9fb4-8ba44621adb9-kube-api-access-sr7f2 podName:f5733389-a2ac-47dd-9fb4-8ba44621adb9 nodeName:}" failed. No retries permitted until 2026-09-17 07:19:56.926267281 +0000 UTC m=+4.112000254 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sr7f2" (UniqueName: "kubernetes.io/projected/f5733389-a2ac-47dd-9fb4-8ba44621adb9-kube-api-access-sr7f2") pod "network-check-target-m6v2q" (UID: "f5733389-a2ac-47dd-9fb4-8ba44621adb9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:19:56.261010 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:56.260922 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 07:14:55 +0000 UTC" deadline="2028-07-03 17:33:20.629380998 +0000 UTC" Sep 17 07:19:56.261010 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:56.260960 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15730h13m24.368425223s" Sep 17 07:19:56.351471 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:56.351422 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f66f8d4_3fa1_461b_8925_f68c138aafd2.slice/crio-76227f8fb611cd085275b18f22c8ca5b081d29a897a6de199bc4114f82d89372 WatchSource:0}: Error finding container 76227f8fb611cd085275b18f22c8ca5b081d29a897a6de199bc4114f82d89372: Status 404 returned error can't find the container with id 76227f8fb611cd085275b18f22c8ca5b081d29a897a6de199bc4114f82d89372 Sep 17 07:19:56.352991 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:56.352971 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd894e06_a948_4955_a07a_dec7e32ca8b0.slice/crio-56afac3a47d9ffb3b825de80f84852970ef461ed25d518b0c250ccc6cd259df5 WatchSource:0}: Error finding container 56afac3a47d9ffb3b825de80f84852970ef461ed25d518b0c250ccc6cd259df5: Status 404 returned error can't find the container with id 56afac3a47d9ffb3b825de80f84852970ef461ed25d518b0c250ccc6cd259df5 Sep 17 07:19:56.356035 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:56.356011 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c0e9d0a_00c1_479b_a811_68a9ca231e9f.slice/crio-59b60445b76bccfe659df5253133b008d531edd11fd1fd5779a36a8d1000c1ac WatchSource:0}: Error finding container 59b60445b76bccfe659df5253133b008d531edd11fd1fd5779a36a8d1000c1ac: Status 404 returned error can't find the container with id 59b60445b76bccfe659df5253133b008d531edd11fd1fd5779a36a8d1000c1ac Sep 17 07:19:56.357727 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:56.357628 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a1579f7_5f4d_406b_a44b_6d03b5b4bcf3.slice/crio-55172de75badb6f0cb31ca583e1b4b7faf945e5ac93fc7ffb14370c5762d416f WatchSource:0}: Error finding container 55172de75badb6f0cb31ca583e1b4b7faf945e5ac93fc7ffb14370c5762d416f: Status 404 returned error can't find the container with id 55172de75badb6f0cb31ca583e1b4b7faf945e5ac93fc7ffb14370c5762d416f Sep 17 07:19:56.358642 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:19:56.358611 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5861ec16_ae09_4421_a393_fe23e69fa0cd.slice/crio-646fffd32f4bb0ab758d2407be5aad67cac06e47a3c87485a6f3efb92c1e84fb WatchSource:0}: Error finding container 646fffd32f4bb0ab758d2407be5aad67cac06e47a3c87485a6f3efb92c1e84fb: Status 404 returned error can't find the container with id 646fffd32f4bb0ab758d2407be5aad67cac06e47a3c87485a6f3efb92c1e84fb Sep 17 07:19:56.832975 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:56.832936 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs\") pod \"network-metrics-daemon-x7484\" (UID: \"69c30c82-b237-4fc3-a96c-61840e73b7a7\") " pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:19:56.833150 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:56.833014 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c285d69f-53b9-4e2a-a0d0-c72e4d592566-original-pull-secret\") pod \"global-pull-secret-syncer-7f254\" (UID: \"c285d69f-53b9-4e2a-a0d0-c72e4d592566\") " pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:19:56.833241 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:56.833156 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:19:56.833241 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:56.833223 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c285d69f-53b9-4e2a-a0d0-c72e4d592566-original-pull-secret podName:c285d69f-53b9-4e2a-a0d0-c72e4d592566 nodeName:}" failed. No retries permitted until 2026-09-17 07:19:58.833204929 +0000 UTC m=+6.018937896 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c285d69f-53b9-4e2a-a0d0-c72e4d592566-original-pull-secret") pod "global-pull-secret-syncer-7f254" (UID: "c285d69f-53b9-4e2a-a0d0-c72e4d592566") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:19:56.834035 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:56.833651 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:19:56.834035 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:56.833704 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs podName:69c30c82-b237-4fc3-a96c-61840e73b7a7 nodeName:}" failed. No retries permitted until 2026-09-17 07:19:58.833688171 +0000 UTC m=+6.019421135 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs") pod "network-metrics-daemon-x7484" (UID: "69c30c82-b237-4fc3-a96c-61840e73b7a7") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:19:56.933797 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:56.933750 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr7f2\" (UniqueName: \"kubernetes.io/projected/f5733389-a2ac-47dd-9fb4-8ba44621adb9-kube-api-access-sr7f2\") pod \"network-check-target-m6v2q\" (UID: \"f5733389-a2ac-47dd-9fb4-8ba44621adb9\") " pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:19:56.933980 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:56.933913 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:19:56.933980 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:56.933932 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:19:56.933980 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:56.933945 2563 projected.go:196] Error preparing data for projected volume kube-api-access-sr7f2 for pod openshift-network-diagnostics/network-check-target-m6v2q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:19:56.934136 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:56.934023 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5733389-a2ac-47dd-9fb4-8ba44621adb9-kube-api-access-sr7f2 podName:f5733389-a2ac-47dd-9fb4-8ba44621adb9 nodeName:}" failed. No retries permitted until 2026-09-17 07:19:58.934003559 +0000 UTC m=+6.119736515 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sr7f2" (UniqueName: "kubernetes.io/projected/f5733389-a2ac-47dd-9fb4-8ba44621adb9-kube-api-access-sr7f2") pod "network-check-target-m6v2q" (UID: "f5733389-a2ac-47dd-9fb4-8ba44621adb9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:19:57.261342 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:57.261284 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 07:14:55 +0000 UTC" deadline="2028-05-11 03:03:59.149993816 +0000 UTC" Sep 17 07:19:57.261342 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:57.261340 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14443h44m1.888658231s" Sep 17 07:19:57.316467 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:57.316350 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:19:57.316467 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:57.316372 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:19:57.316703 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:57.316513 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v2q" podUID="f5733389-a2ac-47dd-9fb4-8ba44621adb9" Sep 17 07:19:57.316798 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:57.316766 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:19:57.316995 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:57.316899 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7f254" podUID="c285d69f-53b9-4e2a-a0d0-c72e4d592566" Sep 17 07:19:57.316995 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:57.316954 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7484" podUID="69c30c82-b237-4fc3-a96c-61840e73b7a7" Sep 17 07:19:57.332778 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:57.332701 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tnmcf" event={"ID":"9c0e9d0a-00c1-479b-a811-68a9ca231e9f","Type":"ContainerStarted","Data":"59b60445b76bccfe659df5253133b008d531edd11fd1fd5779a36a8d1000c1ac"} Sep 17 07:19:57.355936 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:57.355894 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2krnr" event={"ID":"fd894e06-a948-4955-a07a-dec7e32ca8b0","Type":"ContainerStarted","Data":"56afac3a47d9ffb3b825de80f84852970ef461ed25d518b0c250ccc6cd259df5"} Sep 17 07:19:57.361556 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:57.361486 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rwlkw" event={"ID":"0f66f8d4-3fa1-461b-8925-f68c138aafd2","Type":"ContainerStarted","Data":"76227f8fb611cd085275b18f22c8ca5b081d29a897a6de199bc4114f82d89372"} Sep 17 07:19:57.374611 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:57.373405 2563 generic.go:356] "Generic (PLEG): container finished" podID="c5668cb859d7da5d03909e7e9730b526" containerID="ee35e35f2c559c7b8d9e9886d10ee8708c58d263348c78b28fdea5d6887cacc1" exitCode=0 Sep 17 07:19:57.374611 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:57.374113 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-215.ec2.internal" event={"ID":"c5668cb859d7da5d03909e7e9730b526","Type":"ContainerDied","Data":"ee35e35f2c559c7b8d9e9886d10ee8708c58d263348c78b28fdea5d6887cacc1"} Sep 17 07:19:57.396568 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:57.396534 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" event={"ID":"8c5c5699-5a0e-46f6-8c5a-14cb86164285","Type":"ContainerStarted","Data":"c1a8cee648f457c4547c2423bc67428f74b645f63c272b6939ce085643f8e344"} Sep 17 07:19:57.414282 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:57.414250 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-k5dtg" event={"ID":"ddbb72ee-6e56-4549-ac75-f8857a377394","Type":"ContainerStarted","Data":"e1ef0e272d2719d95d2d2fffcd58d1e8705f5a7f483df4eee2fa41c489394a00"} Sep 17 07:19:57.429085 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:57.429050 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" event={"ID":"5861ec16-ae09-4421-a393-fe23e69fa0cd","Type":"ContainerStarted","Data":"646fffd32f4bb0ab758d2407be5aad67cac06e47a3c87485a6f3efb92c1e84fb"} Sep 17 07:19:57.439500 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:57.439471 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-215.ec2.internal" event={"ID":"910049e72d6e50062f9693db2cfac120","Type":"ContainerStarted","Data":"cdcacba0e10cadd8f6451ec5d1850b18ec190b1c250bec3bb0bbe6df0cc1d52f"} Sep 17 07:19:57.450694 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:57.450599 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b42pw" event={"ID":"197c1697-1d3f-414c-bc8d-5919174461cd","Type":"ContainerStarted","Data":"6e08ed2ce34b80849568487a9af002050b3e3c625b56915099be567e57d82a12"} Sep 17 07:19:57.458475 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:57.458374 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9zrxw" event={"ID":"47326f02-5916-4fbe-a826-71c298a22c4b","Type":"ContainerStarted","Data":"ff17435354fc473a1465a732adebb0e05b8b8b97d0a2fd59276dd7c3fbcf9439"} Sep 17 07:19:57.471092 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:57.471047 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9gz52" event={"ID":"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3","Type":"ContainerStarted","Data":"55172de75badb6f0cb31ca583e1b4b7faf945e5ac93fc7ffb14370c5762d416f"} Sep 17 07:19:58.485457 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:58.485180 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-215.ec2.internal" event={"ID":"c5668cb859d7da5d03909e7e9730b526","Type":"ContainerStarted","Data":"1956d1ade7def969469d55ec925d0ea2a528725745f60f12bc3396eecd10dbfe"} Sep 17 07:19:58.513169 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:58.513107 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-215.ec2.internal" podStartSLOduration=4.513083557 podStartE2EDuration="4.513083557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:19:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:19:57.455039095 +0000 UTC m=+4.640772068" watchObservedRunningTime="2026-09-17 07:19:58.513083557 +0000 UTC m=+5.698816530" Sep 17 07:19:58.855389 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:58.855139 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs\") pod \"network-metrics-daemon-x7484\" (UID: \"69c30c82-b237-4fc3-a96c-61840e73b7a7\") " pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:19:58.855389 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:58.855208 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c285d69f-53b9-4e2a-a0d0-c72e4d592566-original-pull-secret\") pod \"global-pull-secret-syncer-7f254\" (UID: \"c285d69f-53b9-4e2a-a0d0-c72e4d592566\") " pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:19:58.855389 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:58.855316 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:19:58.855645 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:58.855400 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:19:58.855645 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:58.855409 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs podName:69c30c82-b237-4fc3-a96c-61840e73b7a7 nodeName:}" failed. No retries permitted until 2026-09-17 07:20:02.855387551 +0000 UTC m=+10.041120503 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs") pod "network-metrics-daemon-x7484" (UID: "69c30c82-b237-4fc3-a96c-61840e73b7a7") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:19:58.855645 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:58.855457 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c285d69f-53b9-4e2a-a0d0-c72e4d592566-original-pull-secret podName:c285d69f-53b9-4e2a-a0d0-c72e4d592566 nodeName:}" failed. No retries permitted until 2026-09-17 07:20:02.855445108 +0000 UTC m=+10.041178058 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c285d69f-53b9-4e2a-a0d0-c72e4d592566-original-pull-secret") pod "global-pull-secret-syncer-7f254" (UID: "c285d69f-53b9-4e2a-a0d0-c72e4d592566") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:19:58.957210 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:58.956428 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr7f2\" (UniqueName: \"kubernetes.io/projected/f5733389-a2ac-47dd-9fb4-8ba44621adb9-kube-api-access-sr7f2\") pod \"network-check-target-m6v2q\" (UID: \"f5733389-a2ac-47dd-9fb4-8ba44621adb9\") " pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:19:58.957210 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:58.956656 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:19:58.957210 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:58.956684 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:19:58.957210 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:58.956698 2563 projected.go:196] Error preparing data for projected volume kube-api-access-sr7f2 for pod openshift-network-diagnostics/network-check-target-m6v2q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:19:58.957210 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:58.956757 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5733389-a2ac-47dd-9fb4-8ba44621adb9-kube-api-access-sr7f2 podName:f5733389-a2ac-47dd-9fb4-8ba44621adb9 nodeName:}" failed. No retries permitted until 2026-09-17 07:20:02.956739247 +0000 UTC m=+10.142472220 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sr7f2" (UniqueName: "kubernetes.io/projected/f5733389-a2ac-47dd-9fb4-8ba44621adb9-kube-api-access-sr7f2") pod "network-check-target-m6v2q" (UID: "f5733389-a2ac-47dd-9fb4-8ba44621adb9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:19:59.308629 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:59.308593 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:19:59.308816 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:59.308739 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v2q" podUID="f5733389-a2ac-47dd-9fb4-8ba44621adb9" Sep 17 07:19:59.312031 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:59.309127 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:19:59.312031 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:59.309262 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7484" podUID="69c30c82-b237-4fc3-a96c-61840e73b7a7" Sep 17 07:19:59.312031 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:19:59.309344 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:19:59.312031 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:19:59.309421 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7f254" podUID="c285d69f-53b9-4e2a-a0d0-c72e4d592566" Sep 17 07:20:01.309125 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:01.308673 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:20:01.309125 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:01.308673 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:20:01.309125 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:01.308809 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:20:01.309125 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:01.308789 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7f254" podUID="c285d69f-53b9-4e2a-a0d0-c72e4d592566" Sep 17 07:20:01.309125 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:01.308881 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7484" podUID="69c30c82-b237-4fc3-a96c-61840e73b7a7" Sep 17 07:20:01.309125 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:01.308933 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v2q" podUID="f5733389-a2ac-47dd-9fb4-8ba44621adb9" Sep 17 07:20:02.888000 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:02.887959 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs\") pod \"network-metrics-daemon-x7484\" (UID: \"69c30c82-b237-4fc3-a96c-61840e73b7a7\") " pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:20:02.888434 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:02.888041 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c285d69f-53b9-4e2a-a0d0-c72e4d592566-original-pull-secret\") pod \"global-pull-secret-syncer-7f254\" (UID: \"c285d69f-53b9-4e2a-a0d0-c72e4d592566\") " pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:20:02.888434 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:02.888197 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:20:02.888434 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:02.888260 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c285d69f-53b9-4e2a-a0d0-c72e4d592566-original-pull-secret podName:c285d69f-53b9-4e2a-a0d0-c72e4d592566 nodeName:}" failed. No retries permitted until 2026-09-17 07:20:10.888241221 +0000 UTC m=+18.073974194 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c285d69f-53b9-4e2a-a0d0-c72e4d592566-original-pull-secret") pod "global-pull-secret-syncer-7f254" (UID: "c285d69f-53b9-4e2a-a0d0-c72e4d592566") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:20:02.888646 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:02.888627 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:20:02.888701 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:02.888683 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs podName:69c30c82-b237-4fc3-a96c-61840e73b7a7 nodeName:}" failed. No retries permitted until 2026-09-17 07:20:10.888668182 +0000 UTC m=+18.074401146 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs") pod "network-metrics-daemon-x7484" (UID: "69c30c82-b237-4fc3-a96c-61840e73b7a7") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:20:02.989296 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:02.989255 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr7f2\" (UniqueName: \"kubernetes.io/projected/f5733389-a2ac-47dd-9fb4-8ba44621adb9-kube-api-access-sr7f2\") pod \"network-check-target-m6v2q\" (UID: \"f5733389-a2ac-47dd-9fb4-8ba44621adb9\") " pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:20:02.989500 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:02.989446 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:20:02.989500 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:02.989466 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:20:02.989500 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:02.989480 2563 projected.go:196] Error preparing data for projected volume kube-api-access-sr7f2 for pod openshift-network-diagnostics/network-check-target-m6v2q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:20:02.989660 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:02.989556 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5733389-a2ac-47dd-9fb4-8ba44621adb9-kube-api-access-sr7f2 podName:f5733389-a2ac-47dd-9fb4-8ba44621adb9 nodeName:}" failed. No retries permitted until 2026-09-17 07:20:10.989530302 +0000 UTC m=+18.175263269 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sr7f2" (UniqueName: "kubernetes.io/projected/f5733389-a2ac-47dd-9fb4-8ba44621adb9-kube-api-access-sr7f2") pod "network-check-target-m6v2q" (UID: "f5733389-a2ac-47dd-9fb4-8ba44621adb9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:20:03.309607 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:03.309571 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:20:03.309796 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:03.309578 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:20:03.309796 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:03.309683 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v2q" podUID="f5733389-a2ac-47dd-9fb4-8ba44621adb9" Sep 17 07:20:03.309796 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:03.309783 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7484" podUID="69c30c82-b237-4fc3-a96c-61840e73b7a7" Sep 17 07:20:03.309953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:03.309852 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:20:03.310010 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:03.309961 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7f254" podUID="c285d69f-53b9-4e2a-a0d0-c72e4d592566" Sep 17 07:20:05.308765 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:05.308237 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:20:05.308765 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:05.308237 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:20:05.308765 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:05.308393 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:20:05.308765 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:05.308415 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7484" podUID="69c30c82-b237-4fc3-a96c-61840e73b7a7" Sep 17 07:20:05.308765 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:05.308472 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7f254" podUID="c285d69f-53b9-4e2a-a0d0-c72e4d592566" Sep 17 07:20:05.308765 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:05.308566 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v2q" podUID="f5733389-a2ac-47dd-9fb4-8ba44621adb9" Sep 17 07:20:07.308454 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:07.308411 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:20:07.308891 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:07.308423 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:20:07.308891 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:07.308573 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v2q" podUID="f5733389-a2ac-47dd-9fb4-8ba44621adb9" Sep 17 07:20:07.308891 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:07.308591 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:20:07.308891 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:07.308664 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7484" podUID="69c30c82-b237-4fc3-a96c-61840e73b7a7" Sep 17 07:20:07.308891 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:07.308731 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7f254" podUID="c285d69f-53b9-4e2a-a0d0-c72e4d592566" Sep 17 07:20:09.308441 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:09.308403 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:20:09.308441 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:09.308429 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:20:09.308939 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:09.308403 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:20:09.308939 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:09.308548 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7484" podUID="69c30c82-b237-4fc3-a96c-61840e73b7a7" Sep 17 07:20:09.308939 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:09.308617 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7f254" podUID="c285d69f-53b9-4e2a-a0d0-c72e4d592566" Sep 17 07:20:09.308939 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:09.308689 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v2q" podUID="f5733389-a2ac-47dd-9fb4-8ba44621adb9" Sep 17 07:20:10.951694 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:10.951651 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs\") pod \"network-metrics-daemon-x7484\" (UID: \"69c30c82-b237-4fc3-a96c-61840e73b7a7\") " pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:20:10.952197 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:10.951717 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c285d69f-53b9-4e2a-a0d0-c72e4d592566-original-pull-secret\") pod \"global-pull-secret-syncer-7f254\" (UID: \"c285d69f-53b9-4e2a-a0d0-c72e4d592566\") " pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:20:10.952197 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:10.951817 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:20:10.952197 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:10.951817 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:20:10.952197 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:10.951885 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs podName:69c30c82-b237-4fc3-a96c-61840e73b7a7 nodeName:}" failed. No retries permitted until 2026-09-17 07:20:26.951865982 +0000 UTC m=+34.137598935 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs") pod "network-metrics-daemon-x7484" (UID: "69c30c82-b237-4fc3-a96c-61840e73b7a7") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:20:10.952197 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:10.951899 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c285d69f-53b9-4e2a-a0d0-c72e4d592566-original-pull-secret podName:c285d69f-53b9-4e2a-a0d0-c72e4d592566 nodeName:}" failed. No retries permitted until 2026-09-17 07:20:26.951893372 +0000 UTC m=+34.137626321 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c285d69f-53b9-4e2a-a0d0-c72e4d592566-original-pull-secret") pod "global-pull-secret-syncer-7f254" (UID: "c285d69f-53b9-4e2a-a0d0-c72e4d592566") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:20:11.052853 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:11.052819 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr7f2\" (UniqueName: \"kubernetes.io/projected/f5733389-a2ac-47dd-9fb4-8ba44621adb9-kube-api-access-sr7f2\") pod \"network-check-target-m6v2q\" (UID: \"f5733389-a2ac-47dd-9fb4-8ba44621adb9\") " pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:20:11.053069 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:11.053020 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:20:11.053069 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:11.053048 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:20:11.053069 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:11.053062 2563 projected.go:196] Error preparing data for projected volume kube-api-access-sr7f2 for pod openshift-network-diagnostics/network-check-target-m6v2q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:20:11.053234 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:11.053122 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5733389-a2ac-47dd-9fb4-8ba44621adb9-kube-api-access-sr7f2 podName:f5733389-a2ac-47dd-9fb4-8ba44621adb9 nodeName:}" failed. No retries permitted until 2026-09-17 07:20:27.053107172 +0000 UTC m=+34.238840135 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sr7f2" (UniqueName: "kubernetes.io/projected/f5733389-a2ac-47dd-9fb4-8ba44621adb9-kube-api-access-sr7f2") pod "network-check-target-m6v2q" (UID: "f5733389-a2ac-47dd-9fb4-8ba44621adb9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:20:11.308739 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:11.308315 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:20:11.308739 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:11.308335 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:20:11.308739 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:11.308442 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v2q" podUID="f5733389-a2ac-47dd-9fb4-8ba44621adb9" Sep 17 07:20:11.308739 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:11.308461 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:20:11.308739 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:11.308586 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7484" podUID="69c30c82-b237-4fc3-a96c-61840e73b7a7" Sep 17 07:20:11.308739 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:11.308689 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7f254" podUID="c285d69f-53b9-4e2a-a0d0-c72e4d592566" Sep 17 07:20:13.309299 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:13.309264 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:20:13.309776 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:13.309387 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v2q" podUID="f5733389-a2ac-47dd-9fb4-8ba44621adb9" Sep 17 07:20:13.309776 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:13.309484 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:20:13.309776 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:13.309600 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7f254" podUID="c285d69f-53b9-4e2a-a0d0-c72e4d592566" Sep 17 07:20:13.309776 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:13.309646 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:20:13.309776 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:13.309718 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7484" podUID="69c30c82-b237-4fc3-a96c-61840e73b7a7" Sep 17 07:20:14.539309 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:14.539061 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9zrxw" event={"ID":"47326f02-5916-4fbe-a826-71c298a22c4b","Type":"ContainerStarted","Data":"9d9ba3a0968f46535fd3de6a6b74f73ef6d448184be45923d7d748001837a576"} Sep 17 07:20:14.540572 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:14.540539 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9gz52" event={"ID":"9a1579f7-5f4d-406b-a44b-6d03b5b4bcf3","Type":"ContainerStarted","Data":"b197bd92927cf7ca80de24e122b819d0c3e6fb114c2d8a0d1789a8d424712c38"} Sep 17 07:20:14.541955 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:14.541925 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tnmcf" event={"ID":"9c0e9d0a-00c1-479b-a811-68a9ca231e9f","Type":"ContainerStarted","Data":"c2be119c46bd7402d77f4efcf9a68c80fe2f8f85c371d1d2fdf001c89039b715"} Sep 17 07:20:14.543427 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:14.543402 2563 generic.go:356] "Generic (PLEG): container finished" podID="fd894e06-a948-4955-a07a-dec7e32ca8b0" containerID="7a5bc59f4bd64df8d7df851e91bf100e6ec32c7cfd8604e68c3e318f8240087a" exitCode=0 Sep 17 07:20:14.543533 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:14.543475 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2krnr" event={"ID":"fd894e06-a948-4955-a07a-dec7e32ca8b0","Type":"ContainerDied","Data":"7a5bc59f4bd64df8d7df851e91bf100e6ec32c7cfd8604e68c3e318f8240087a"} Sep 17 07:20:14.545256 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:14.544957 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rwlkw" event={"ID":"0f66f8d4-3fa1-461b-8925-f68c138aafd2","Type":"ContainerStarted","Data":"0331561ed4b9d5eecbb3e70a70c0982c46ef794d78cffa5eb3a5da631bd443c4"} Sep 17 07:20:14.547901 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:14.547874 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" event={"ID":"8c5c5699-5a0e-46f6-8c5a-14cb86164285","Type":"ContainerStarted","Data":"b396509c9e2d541627fc8cf795fc46df5e58e4316ad50d3a0c73faa579d63bc0"} Sep 17 07:20:14.549879 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:14.549856 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-k5dtg" event={"ID":"ddbb72ee-6e56-4549-ac75-f8857a377394","Type":"ContainerStarted","Data":"73e4198b2a597ed6c88763c87aac3b75ebaed3cfcd880ef03002cdaedbd3ba84"} Sep 17 07:20:14.552585 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:14.552566 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" event={"ID":"5861ec16-ae09-4421-a393-fe23e69fa0cd","Type":"ContainerStarted","Data":"0b66f05cf98e3a6e1a64521c896a186d8b0827ee34bbd7faf253f236945168bd"} Sep 17 07:20:14.552669 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:14.552617 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" event={"ID":"5861ec16-ae09-4421-a393-fe23e69fa0cd","Type":"ContainerStarted","Data":"7d2f2ee6fddb8eaf22343f5de11407347809d3708ac77dcc8171947e8ecf44df"} Sep 17 07:20:14.552669 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:14.552645 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" event={"ID":"5861ec16-ae09-4421-a393-fe23e69fa0cd","Type":"ContainerStarted","Data":"b6d65cc194f2e4aef0d507afeb8ef6547b33f730c00a5bf18671ccb3a79cb4cc"} Sep 17 07:20:14.552736 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:14.552672 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" event={"ID":"5861ec16-ae09-4421-a393-fe23e69fa0cd","Type":"ContainerStarted","Data":"74b0e2fc4541c08010e98380007485fa1f1553669470acc18f1a770007dd14d6"} Sep 17 07:20:14.563688 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:14.563644 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-215.ec2.internal" podStartSLOduration=20.563630795 podStartE2EDuration="20.563630795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:19:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:19:58.513974896 +0000 UTC m=+5.699707870" watchObservedRunningTime="2026-09-17 07:20:14.563630795 +0000 UTC m=+21.749363767" Sep 17 07:20:14.588670 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:14.588631 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9zrxw" podStartSLOduration=3.871928213 podStartE2EDuration="21.588615202s" totalImagesPullingTime="17.716686989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:20:14.563897531 +0000 UTC m=+21.749630503" watchObservedRunningTime="2026-09-17 07:20:14.588615202 +0000 UTC m=+21.774348177" Sep 17 07:20:14.588774 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:14.588758 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9gz52" podStartSLOduration=3.868053139 podStartE2EDuration="21.588753313s" totalImagesPullingTime="17.720700174s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:20:14.588421072 +0000 UTC m=+21.774154036" watchObservedRunningTime="2026-09-17 07:20:14.588753313 +0000 UTC m=+21.774486285" Sep 17 07:20:14.603401 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:14.603369 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9zrxw" Sep 17 07:20:14.604015 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:14.603996 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9zrxw" Sep 17 07:20:14.608083 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:14.608036 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-tnmcf" podStartSLOduration=3.885734882 podStartE2EDuration="21.608025904s" totalImagesPullingTime="17.722291022s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:20:14.608009914 +0000 UTC m=+21.793742886" watchObservedRunningTime="2026-09-17 07:20:14.608025904 +0000 UTC m=+21.793758860" Sep 17 07:20:14.656727 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:14.656669 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rwlkw" podStartSLOduration=3.873898212 podStartE2EDuration="21.656655981s" totalImagesPullingTime="17.782757769s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:20:14.656496396 +0000 UTC m=+21.842229368" watchObservedRunningTime="2026-09-17 07:20:14.656655981 +0000 UTC m=+21.842388953" Sep 17 07:20:14.673501 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:14.673450 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-k5dtg" podStartSLOduration=7.793199514 podStartE2EDuration="21.673431025s" totalImagesPullingTime="13.880231511s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:20:14.672994364 +0000 UTC m=+21.858727340" watchObservedRunningTime="2026-09-17 07:20:14.673431025 +0000 UTC m=+21.859164000" Sep 17 07:20:15.309514 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:15.309252 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:20:15.309718 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:15.309252 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:20:15.309718 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:15.309290 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:20:15.309718 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:15.309694 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7484" podUID="69c30c82-b237-4fc3-a96c-61840e73b7a7" Sep 17 07:20:15.309872 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:15.309837 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7f254" podUID="c285d69f-53b9-4e2a-a0d0-c72e4d592566" Sep 17 07:20:15.309948 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:15.309928 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v2q" podUID="f5733389-a2ac-47dd-9fb4-8ba44621adb9" Sep 17 07:20:15.559782 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:15.559747 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" event={"ID":"5861ec16-ae09-4421-a393-fe23e69fa0cd","Type":"ContainerStarted","Data":"d47efa13925509e87f431054d03894f7685a0fdb50f3855b29740e9f8cf051bf"} Sep 17 07:20:15.560208 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:15.559852 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" event={"ID":"5861ec16-ae09-4421-a393-fe23e69fa0cd","Type":"ContainerStarted","Data":"e7e5ff6a31364b7b879a0e0ce2a4be48ed529e42e9c41c194f82b10954c72541"} Sep 17 07:20:15.562816 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:15.562786 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b42pw" event={"ID":"197c1697-1d3f-414c-bc8d-5919174461cd","Type":"ContainerStarted","Data":"a49237e3672f76acb6bac0e55c70b7c937b9702f9494785a2923c6dae315a82b"} Sep 17 07:20:15.563375 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:15.563350 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9zrxw" Sep 17 07:20:15.563799 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:15.563778 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9zrxw" Sep 17 07:20:15.586170 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:15.586056 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-b42pw" podStartSLOduration=8.707001096 podStartE2EDuration="22.586034503s" totalImagesPullingTime="13.879033407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:20:15.585687067 +0000 UTC m=+22.771420039" watchObservedRunningTime="2026-09-17 07:20:15.586034503 +0000 UTC m=+22.771767478" Sep 17 07:20:15.651105 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:15.651077 2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 17 07:20:16.272195 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:16.272043 2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T07:20:15.65109887Z","UUID":"98dacefe-9712-4491-9fb8-d642cf5b4e29","Handler":null,"Name":"","Endpoint":""} Sep 17 07:20:16.274160 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:16.274123 2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 17 07:20:16.274302 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:16.274168 2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 17 07:20:16.566884 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:16.566807 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" event={"ID":"8c5c5699-5a0e-46f6-8c5a-14cb86164285","Type":"ContainerStarted","Data":"126009d304605a766f337984feac1da9e677a12b450eb7117f04ded7b3382a35"} Sep 17 07:20:17.309315 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:17.309280 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:20:17.309315 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:17.309280 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:20:17.309602 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:17.309338 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:20:17.309602 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:17.309435 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7484" podUID="69c30c82-b237-4fc3-a96c-61840e73b7a7" Sep 17 07:20:17.309602 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:17.309506 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v2q" podUID="f5733389-a2ac-47dd-9fb4-8ba44621adb9" Sep 17 07:20:17.309602 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:17.309568 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7f254" podUID="c285d69f-53b9-4e2a-a0d0-c72e4d592566" Sep 17 07:20:17.571695 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:17.571604 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" event={"ID":"8c5c5699-5a0e-46f6-8c5a-14cb86164285","Type":"ContainerStarted","Data":"56f45b37f42af2654c563572c317fb492ae7c1bfac2cc51cb61d37972b7729c5"} Sep 17 07:20:17.575716 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:17.575680 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" event={"ID":"5861ec16-ae09-4421-a393-fe23e69fa0cd","Type":"ContainerStarted","Data":"2042101893da7266163ae9a170f9d7b8ce954bc16a8dd143c88263d2db25c601"} Sep 17 07:20:17.594270 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:17.594223 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpgv" podStartSLOduration=4.586673009 podStartE2EDuration="24.59420802s" totalImagesPullingTime="20.007535011s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:19:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:20:17.592999287 +0000 UTC m=+24.778732259" watchObservedRunningTime="2026-09-17 07:20:17.59420802 +0000 UTC m=+24.779940992" Sep 17 07:20:19.309042 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:19.309000 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:20:19.309042 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:19.309029 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:20:19.309574 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:19.309007 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:20:19.309574 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:19.309143 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v2q" podUID="f5733389-a2ac-47dd-9fb4-8ba44621adb9" Sep 17 07:20:19.309574 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:19.309263 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7484" podUID="69c30c82-b237-4fc3-a96c-61840e73b7a7" Sep 17 07:20:19.309574 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:19.309375 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7f254" podUID="c285d69f-53b9-4e2a-a0d0-c72e4d592566" Sep 17 07:20:20.584974 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:20.584796 2563 generic.go:356] "Generic (PLEG): container finished" podID="fd894e06-a948-4955-a07a-dec7e32ca8b0" containerID="3ce4a8db965149927e402b9615ce795b7008a7649015ff805312329fcbfc7f3b" exitCode=0 Sep 17 07:20:20.585691 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:20.584877 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2krnr" event={"ID":"fd894e06-a948-4955-a07a-dec7e32ca8b0","Type":"ContainerDied","Data":"3ce4a8db965149927e402b9615ce795b7008a7649015ff805312329fcbfc7f3b"} Sep 17 07:20:20.589002 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:20.588983 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" event={"ID":"5861ec16-ae09-4421-a393-fe23e69fa0cd","Type":"ContainerStarted","Data":"9c929c291a9424c4f1290d81dea2e3416860a1c31a3e4184626e1d2348836776"} Sep 17 07:20:20.589244 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:20.589219 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:20:20.589382 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:20.589293 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:20:20.589382 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:20.589362 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:20:20.605036 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:20.605009 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:20:20.605139 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:20.605109 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:20:20.641621 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:20.641578 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" podStartSLOduration=9.872733496 podStartE2EDuration="27.64156221s" totalImagesPullingTime="17.768828714s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:20:20.64012535 +0000 UTC m=+27.825858321" watchObservedRunningTime="2026-09-17 07:20:20.64156221 +0000 UTC m=+27.827295164" Sep 17 07:20:21.309206 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:21.309172 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:20:21.309206 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:21.309194 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:20:21.309476 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:21.309255 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:20:21.309476 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:21.309365 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v2q" podUID="f5733389-a2ac-47dd-9fb4-8ba44621adb9" Sep 17 07:20:21.309548 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:21.309487 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7f254" podUID="c285d69f-53b9-4e2a-a0d0-c72e4d592566" Sep 17 07:20:21.309587 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:21.309573 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7484" podUID="69c30c82-b237-4fc3-a96c-61840e73b7a7" Sep 17 07:20:22.141158 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:22.140957 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-x7484"] Sep 17 07:20:22.141676 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:22.141238 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:20:22.141676 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:22.141358 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7484" podUID="69c30c82-b237-4fc3-a96c-61840e73b7a7" Sep 17 07:20:22.145132 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:22.145108 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7f254"] Sep 17 07:20:22.145245 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:22.145192 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:20:22.145292 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:22.145276 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7f254" podUID="c285d69f-53b9-4e2a-a0d0-c72e4d592566" Sep 17 07:20:22.145676 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:22.145653 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m6v2q"] Sep 17 07:20:22.145786 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:22.145739 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:20:22.145857 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:22.145829 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v2q" podUID="f5733389-a2ac-47dd-9fb4-8ba44621adb9" Sep 17 07:20:22.595652 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:22.595593 2563 generic.go:356] "Generic (PLEG): container finished" podID="fd894e06-a948-4955-a07a-dec7e32ca8b0" containerID="6f486caa38ac2350aa3bd4a3a23809395b862b98c63beb70ab90781adf58e63a" exitCode=0 Sep 17 07:20:22.595794 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:22.595657 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2krnr" event={"ID":"fd894e06-a948-4955-a07a-dec7e32ca8b0","Type":"ContainerDied","Data":"6f486caa38ac2350aa3bd4a3a23809395b862b98c63beb70ab90781adf58e63a"} Sep 17 07:20:24.308673 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:24.308634 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:20:24.309056 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:24.308634 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:20:24.309056 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:24.308746 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7f254" podUID="c285d69f-53b9-4e2a-a0d0-c72e4d592566" Sep 17 07:20:24.309056 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:24.308635 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:20:24.309056 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:24.308821 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v2q" podUID="f5733389-a2ac-47dd-9fb4-8ba44621adb9" Sep 17 07:20:24.309056 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:24.308887 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7484" podUID="69c30c82-b237-4fc3-a96c-61840e73b7a7" Sep 17 07:20:24.606335 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:24.606234 2563 generic.go:356] "Generic (PLEG): container finished" podID="fd894e06-a948-4955-a07a-dec7e32ca8b0" containerID="1b1aaabe1dd780bc6ff044bcc56712575803cb2ef378f96b690cf4bb03bd196f" exitCode=0 Sep 17 07:20:24.606335 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:24.606295 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2krnr" event={"ID":"fd894e06-a948-4955-a07a-dec7e32ca8b0","Type":"ContainerDied","Data":"1b1aaabe1dd780bc6ff044bcc56712575803cb2ef378f96b690cf4bb03bd196f"} Sep 17 07:20:26.308474 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:26.308418 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:20:26.308474 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:26.308448 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:20:26.308976 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:26.308455 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:20:26.308976 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:26.308561 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7f254" podUID="c285d69f-53b9-4e2a-a0d0-c72e4d592566" Sep 17 07:20:26.308976 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:26.308625 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m6v2q" podUID="f5733389-a2ac-47dd-9fb4-8ba44621adb9" Sep 17 07:20:26.308976 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:26.308709 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x7484" podUID="69c30c82-b237-4fc3-a96c-61840e73b7a7" Sep 17 07:20:26.976813 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:26.976782 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c285d69f-53b9-4e2a-a0d0-c72e4d592566-original-pull-secret\") pod \"global-pull-secret-syncer-7f254\" (UID: \"c285d69f-53b9-4e2a-a0d0-c72e4d592566\") " pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:20:26.977091 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:26.976855 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs\") pod \"network-metrics-daemon-x7484\" (UID: \"69c30c82-b237-4fc3-a96c-61840e73b7a7\") " pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:20:26.977091 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:26.976950 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 07:20:26.977091 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:26.976963 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:20:26.977091 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:26.977023 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs podName:69c30c82-b237-4fc3-a96c-61840e73b7a7 nodeName:}" failed. No retries permitted until 2026-09-17 07:20:58.977003577 +0000 UTC m=+66.162736545 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs") pod "network-metrics-daemon-x7484" (UID: "69c30c82-b237-4fc3-a96c-61840e73b7a7") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 07:20:26.977091 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:26.977039 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c285d69f-53b9-4e2a-a0d0-c72e4d592566-original-pull-secret podName:c285d69f-53b9-4e2a-a0d0-c72e4d592566 nodeName:}" failed. No retries permitted until 2026-09-17 07:20:58.977031602 +0000 UTC m=+66.162764552 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c285d69f-53b9-4e2a-a0d0-c72e4d592566-original-pull-secret") pod "global-pull-secret-syncer-7f254" (UID: "c285d69f-53b9-4e2a-a0d0-c72e4d592566") : object "kube-system"/"original-pull-secret" not registered Sep 17 07:20:27.078099 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.078068 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr7f2\" (UniqueName: \"kubernetes.io/projected/f5733389-a2ac-47dd-9fb4-8ba44621adb9-kube-api-access-sr7f2\") pod \"network-check-target-m6v2q\" (UID: \"f5733389-a2ac-47dd-9fb4-8ba44621adb9\") " pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:20:27.078296 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:27.078241 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 07:20:27.078296 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:27.078272 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 07:20:27.078296 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:27.078287 2563 projected.go:196] Error preparing data for projected volume kube-api-access-sr7f2 for pod openshift-network-diagnostics/network-check-target-m6v2q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:20:27.078428 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:27.078375 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5733389-a2ac-47dd-9fb4-8ba44621adb9-kube-api-access-sr7f2 podName:f5733389-a2ac-47dd-9fb4-8ba44621adb9 nodeName:}" failed. No retries permitted until 2026-09-17 07:20:59.078355084 +0000 UTC m=+66.264088043 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sr7f2" (UniqueName: "kubernetes.io/projected/f5733389-a2ac-47dd-9fb4-8ba44621adb9-kube-api-access-sr7f2") pod "network-check-target-m6v2q" (UID: "f5733389-a2ac-47dd-9fb4-8ba44621adb9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 07:20:27.078820 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.078802 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-215.ec2.internal" event="NodeReady" Sep 17 07:20:27.080475 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.079258 2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 17 07:20:27.153762 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.153724 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hll2c"] Sep 17 07:20:27.175202 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.175165 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tr629"] Sep 17 07:20:27.175399 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.175289 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hll2c" Sep 17 07:20:27.179915 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.179755 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 17 07:20:27.180104 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.179767 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 17 07:20:27.181034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.181014 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9p6qq\"" Sep 17 07:20:27.186668 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.186648 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 17 07:20:27.193379 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.193348 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tr629"] Sep 17 07:20:27.193489 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.193421 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tr629" Sep 17 07:20:27.193556 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.193425 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hll2c"] Sep 17 07:20:27.196371 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.196353 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 17 07:20:27.196705 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.196688 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tth88\"" Sep 17 07:20:27.200504 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.200487 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 17 07:20:27.279992 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.279892 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3305365-b6fc-4a79-97c8-63209d1d71ea-tmp-dir\") pod \"dns-default-tr629\" (UID: \"f3305365-b6fc-4a79-97c8-63209d1d71ea\") " pod="openshift-dns/dns-default-tr629" Sep 17 07:20:27.279992 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.279944 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drvql\" (UniqueName: \"kubernetes.io/projected/f3305365-b6fc-4a79-97c8-63209d1d71ea-kube-api-access-drvql\") pod \"dns-default-tr629\" (UID: \"f3305365-b6fc-4a79-97c8-63209d1d71ea\") " pod="openshift-dns/dns-default-tr629" Sep 17 07:20:27.280278 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.280052 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls\") pod \"dns-default-tr629\" (UID: \"f3305365-b6fc-4a79-97c8-63209d1d71ea\") " pod="openshift-dns/dns-default-tr629" Sep 17 07:20:27.280278 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.280106 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcb4z\" (UniqueName: \"kubernetes.io/projected/79621566-efae-43dd-9386-30a4a2aed04b-kube-api-access-qcb4z\") pod \"ingress-canary-hll2c\" (UID: \"79621566-efae-43dd-9386-30a4a2aed04b\") " pod="openshift-ingress-canary/ingress-canary-hll2c" Sep 17 07:20:27.280278 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.280164 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79621566-efae-43dd-9386-30a4a2aed04b-cert\") pod \"ingress-canary-hll2c\" (UID: \"79621566-efae-43dd-9386-30a4a2aed04b\") " pod="openshift-ingress-canary/ingress-canary-hll2c" Sep 17 07:20:27.280278 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.280224 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f3305365-b6fc-4a79-97c8-63209d1d71ea-config-volume\") pod \"dns-default-tr629\" (UID: \"f3305365-b6fc-4a79-97c8-63209d1d71ea\") " pod="openshift-dns/dns-default-tr629" Sep 17 07:20:27.381275 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.381233 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcb4z\" (UniqueName: \"kubernetes.io/projected/79621566-efae-43dd-9386-30a4a2aed04b-kube-api-access-qcb4z\") pod \"ingress-canary-hll2c\" (UID: \"79621566-efae-43dd-9386-30a4a2aed04b\") " pod="openshift-ingress-canary/ingress-canary-hll2c" Sep 17 07:20:27.381791 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.381287 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79621566-efae-43dd-9386-30a4a2aed04b-cert\") pod \"ingress-canary-hll2c\" (UID: \"79621566-efae-43dd-9386-30a4a2aed04b\") " pod="openshift-ingress-canary/ingress-canary-hll2c" Sep 17 07:20:27.381791 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.381357 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f3305365-b6fc-4a79-97c8-63209d1d71ea-config-volume\") pod \"dns-default-tr629\" (UID: \"f3305365-b6fc-4a79-97c8-63209d1d71ea\") " pod="openshift-dns/dns-default-tr629" Sep 17 07:20:27.381791 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.381387 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3305365-b6fc-4a79-97c8-63209d1d71ea-tmp-dir\") pod \"dns-default-tr629\" (UID: \"f3305365-b6fc-4a79-97c8-63209d1d71ea\") " pod="openshift-dns/dns-default-tr629" Sep 17 07:20:27.381791 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.381411 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drvql\" (UniqueName: \"kubernetes.io/projected/f3305365-b6fc-4a79-97c8-63209d1d71ea-kube-api-access-drvql\") pod \"dns-default-tr629\" (UID: \"f3305365-b6fc-4a79-97c8-63209d1d71ea\") " pod="openshift-dns/dns-default-tr629" Sep 17 07:20:27.381791 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.381470 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls\") pod \"dns-default-tr629\" (UID: \"f3305365-b6fc-4a79-97c8-63209d1d71ea\") " pod="openshift-dns/dns-default-tr629" Sep 17 07:20:27.381791 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:27.381585 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:20:27.381791 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:27.381641 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls podName:f3305365-b6fc-4a79-97c8-63209d1d71ea nodeName:}" failed. No retries permitted until 2026-09-17 07:20:27.881625258 +0000 UTC m=+35.067358225 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls") pod "dns-default-tr629" (UID: "f3305365-b6fc-4a79-97c8-63209d1d71ea") : secret "dns-default-metrics-tls" not found Sep 17 07:20:27.382050 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.381921 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3305365-b6fc-4a79-97c8-63209d1d71ea-tmp-dir\") pod \"dns-default-tr629\" (UID: \"f3305365-b6fc-4a79-97c8-63209d1d71ea\") " pod="openshift-dns/dns-default-tr629" Sep 17 07:20:27.382050 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.381940 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f3305365-b6fc-4a79-97c8-63209d1d71ea-config-volume\") pod \"dns-default-tr629\" (UID: \"f3305365-b6fc-4a79-97c8-63209d1d71ea\") " pod="openshift-dns/dns-default-tr629" Sep 17 07:20:27.386205 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.386173 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79621566-efae-43dd-9386-30a4a2aed04b-cert\") pod \"ingress-canary-hll2c\" (UID: \"79621566-efae-43dd-9386-30a4a2aed04b\") " pod="openshift-ingress-canary/ingress-canary-hll2c" Sep 17 07:20:27.402713 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.402685 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drvql\" (UniqueName: \"kubernetes.io/projected/f3305365-b6fc-4a79-97c8-63209d1d71ea-kube-api-access-drvql\") pod \"dns-default-tr629\" (UID: \"f3305365-b6fc-4a79-97c8-63209d1d71ea\") " pod="openshift-dns/dns-default-tr629" Sep 17 07:20:27.402713 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.402698 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcb4z\" (UniqueName: \"kubernetes.io/projected/79621566-efae-43dd-9386-30a4a2aed04b-kube-api-access-qcb4z\") pod \"ingress-canary-hll2c\" (UID: \"79621566-efae-43dd-9386-30a4a2aed04b\") " pod="openshift-ingress-canary/ingress-canary-hll2c" Sep 17 07:20:27.487716 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.487679 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hll2c" Sep 17 07:20:27.886146 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:27.886109 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls\") pod \"dns-default-tr629\" (UID: \"f3305365-b6fc-4a79-97c8-63209d1d71ea\") " pod="openshift-dns/dns-default-tr629" Sep 17 07:20:27.886378 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:27.886251 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:20:27.886378 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:27.886338 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls podName:f3305365-b6fc-4a79-97c8-63209d1d71ea nodeName:}" failed. No retries permitted until 2026-09-17 07:20:28.886308771 +0000 UTC m=+36.072041734 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls") pod "dns-default-tr629" (UID: "f3305365-b6fc-4a79-97c8-63209d1d71ea") : secret "dns-default-metrics-tls" not found Sep 17 07:20:28.309150 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:28.309115 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:20:28.309362 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:28.309115 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:20:28.309362 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:28.309115 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:20:28.312339 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:28.312299 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 17 07:20:28.312467 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:28.312312 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lkdnc\"" Sep 17 07:20:28.313361 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:28.313339 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 17 07:20:28.313657 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:28.313640 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-krbdq\"" Sep 17 07:20:28.313741 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:28.313679 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 17 07:20:28.313741 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:28.313696 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 17 07:20:28.894810 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:28.894788 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls\") pod \"dns-default-tr629\" (UID: \"f3305365-b6fc-4a79-97c8-63209d1d71ea\") " pod="openshift-dns/dns-default-tr629" Sep 17 07:20:28.895620 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:28.894956 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:20:28.895620 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:28.895048 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls podName:f3305365-b6fc-4a79-97c8-63209d1d71ea nodeName:}" failed. No retries permitted until 2026-09-17 07:20:30.895025463 +0000 UTC m=+38.080758435 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls") pod "dns-default-tr629" (UID: "f3305365-b6fc-4a79-97c8-63209d1d71ea") : secret "dns-default-metrics-tls" not found Sep 17 07:20:28.987938 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:28.987911 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hll2c"] Sep 17 07:20:28.989842 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:20:28.989818 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79621566_efae_43dd_9386_30a4a2aed04b.slice/crio-b0c3b5f91cf9d48c72a4f98696f3bf8ea0b52de0991825c32a2f4bfdeacb50e6 WatchSource:0}: Error finding container b0c3b5f91cf9d48c72a4f98696f3bf8ea0b52de0991825c32a2f4bfdeacb50e6: Status 404 returned error can't find the container with id b0c3b5f91cf9d48c72a4f98696f3bf8ea0b52de0991825c32a2f4bfdeacb50e6 Sep 17 07:20:29.620894 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:29.620699 2563 generic.go:356] "Generic (PLEG): container finished" podID="fd894e06-a948-4955-a07a-dec7e32ca8b0" containerID="fc9e90450f7cef8eaf345e9d1e54fb3209eebb4d4b5dcdc5450c7878c3131541" exitCode=0 Sep 17 07:20:29.621075 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:29.620791 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2krnr" event={"ID":"fd894e06-a948-4955-a07a-dec7e32ca8b0","Type":"ContainerDied","Data":"fc9e90450f7cef8eaf345e9d1e54fb3209eebb4d4b5dcdc5450c7878c3131541"} Sep 17 07:20:29.624429 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:29.624390 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hll2c" event={"ID":"79621566-efae-43dd-9386-30a4a2aed04b","Type":"ContainerStarted","Data":"b0c3b5f91cf9d48c72a4f98696f3bf8ea0b52de0991825c32a2f4bfdeacb50e6"} Sep 17 07:20:30.631334 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:30.631280 2563 generic.go:356] "Generic (PLEG): container finished" podID="fd894e06-a948-4955-a07a-dec7e32ca8b0" containerID="7daf01ebe47a71251136c23bac2fff2423535aee7dac0a1849b15a7bda1c8aee" exitCode=0 Sep 17 07:20:30.631736 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:30.631366 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2krnr" event={"ID":"fd894e06-a948-4955-a07a-dec7e32ca8b0","Type":"ContainerDied","Data":"7daf01ebe47a71251136c23bac2fff2423535aee7dac0a1849b15a7bda1c8aee"} Sep 17 07:20:30.913268 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:30.913169 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls\") pod \"dns-default-tr629\" (UID: \"f3305365-b6fc-4a79-97c8-63209d1d71ea\") " pod="openshift-dns/dns-default-tr629" Sep 17 07:20:30.913468 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:30.913293 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:20:30.913468 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:30.913391 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls podName:f3305365-b6fc-4a79-97c8-63209d1d71ea nodeName:}" failed. No retries permitted until 2026-09-17 07:20:34.913367712 +0000 UTC m=+42.099100684 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls") pod "dns-default-tr629" (UID: "f3305365-b6fc-4a79-97c8-63209d1d71ea") : secret "dns-default-metrics-tls" not found Sep 17 07:20:31.637845 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:31.637809 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2krnr" event={"ID":"fd894e06-a948-4955-a07a-dec7e32ca8b0","Type":"ContainerStarted","Data":"23995e3a806f91a128a9f026a8c79cfad51d2771c2e53ca8598f811198cf90b9"} Sep 17 07:20:31.665451 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:31.665297 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2krnr" podStartSLOduration=8.743218347 podStartE2EDuration="38.66527546s" totalImagesPullingTime="29.922057113s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:19:53 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:20:31.663801843 +0000 UTC m=+38.849534814" watchObservedRunningTime="2026-09-17 07:20:31.66527546 +0000 UTC m=+38.851008434" Sep 17 07:20:32.641406 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:32.641367 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hll2c" event={"ID":"79621566-efae-43dd-9386-30a4a2aed04b","Type":"ContainerStarted","Data":"ff9c925cd935f9c34b01fe434a6a1d78622983512f2cf68f6d3f5b9bf9e52c7f"} Sep 17 07:20:32.660979 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:32.660929 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hll2c" podStartSLOduration=2.795670542 podStartE2EDuration="5.660914626s" totalImagesPullingTime="2.865244084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:20:32.659484262 +0000 UTC m=+39.845217235" watchObservedRunningTime="2026-09-17 07:20:32.660914626 +0000 UTC m=+39.846647592" Sep 17 07:20:34.943055 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:34.943013 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls\") pod \"dns-default-tr629\" (UID: \"f3305365-b6fc-4a79-97c8-63209d1d71ea\") " pod="openshift-dns/dns-default-tr629" Sep 17 07:20:34.943467 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:34.943157 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:20:34.943467 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:34.943223 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls podName:f3305365-b6fc-4a79-97c8-63209d1d71ea nodeName:}" failed. No retries permitted until 2026-09-17 07:20:42.943207461 +0000 UTC m=+50.128940415 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls") pod "dns-default-tr629" (UID: "f3305365-b6fc-4a79-97c8-63209d1d71ea") : secret "dns-default-metrics-tls" not found Sep 17 07:20:42.997626 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:42.997586 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls\") pod \"dns-default-tr629\" (UID: \"f3305365-b6fc-4a79-97c8-63209d1d71ea\") " pod="openshift-dns/dns-default-tr629" Sep 17 07:20:42.998088 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:42.997724 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:20:42.998088 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:42.997795 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls podName:f3305365-b6fc-4a79-97c8-63209d1d71ea nodeName:}" failed. No retries permitted until 2026-09-17 07:20:58.997776012 +0000 UTC m=+66.183508966 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls") pod "dns-default-tr629" (UID: "f3305365-b6fc-4a79-97c8-63209d1d71ea") : secret "dns-default-metrics-tls" not found Sep 17 07:20:52.615621 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:52.615589 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-68n7z" Sep 17 07:20:59.006999 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:59.006952 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs\") pod \"network-metrics-daemon-x7484\" (UID: \"69c30c82-b237-4fc3-a96c-61840e73b7a7\") " pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:20:59.007555 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:59.007016 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls\") pod \"dns-default-tr629\" (UID: \"f3305365-b6fc-4a79-97c8-63209d1d71ea\") " pod="openshift-dns/dns-default-tr629" Sep 17 07:20:59.007555 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:59.007041 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c285d69f-53b9-4e2a-a0d0-c72e4d592566-original-pull-secret\") pod \"global-pull-secret-syncer-7f254\" (UID: \"c285d69f-53b9-4e2a-a0d0-c72e4d592566\") " pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:20:59.007555 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:59.007170 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:20:59.007555 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:59.007265 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls podName:f3305365-b6fc-4a79-97c8-63209d1d71ea nodeName:}" failed. No retries permitted until 2026-09-17 07:21:31.007241436 +0000 UTC m=+98.192974390 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls") pod "dns-default-tr629" (UID: "f3305365-b6fc-4a79-97c8-63209d1d71ea") : secret "dns-default-metrics-tls" not found Sep 17 07:20:59.010772 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:59.010752 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 17 07:20:59.011227 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:59.011214 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 17 07:20:59.017497 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:59.017481 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 17 07:20:59.017569 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:20:59.017559 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs podName:69c30c82-b237-4fc3-a96c-61840e73b7a7 nodeName:}" failed. No retries permitted until 2026-09-17 07:22:03.017538187 +0000 UTC m=+130.203271139 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs") pod "network-metrics-daemon-x7484" (UID: "69c30c82-b237-4fc3-a96c-61840e73b7a7") : secret "metrics-daemon-secret" not found Sep 17 07:20:59.020776 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:59.020754 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c285d69f-53b9-4e2a-a0d0-c72e4d592566-original-pull-secret\") pod \"global-pull-secret-syncer-7f254\" (UID: \"c285d69f-53b9-4e2a-a0d0-c72e4d592566\") " pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:20:59.108211 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:59.108187 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr7f2\" (UniqueName: \"kubernetes.io/projected/f5733389-a2ac-47dd-9fb4-8ba44621adb9-kube-api-access-sr7f2\") pod \"network-check-target-m6v2q\" (UID: \"f5733389-a2ac-47dd-9fb4-8ba44621adb9\") " pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:20:59.111687 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:59.111669 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 17 07:20:59.122410 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:59.122392 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 17 07:20:59.132721 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:59.132696 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr7f2\" (UniqueName: \"kubernetes.io/projected/f5733389-a2ac-47dd-9fb4-8ba44621adb9-kube-api-access-sr7f2\") pod \"network-check-target-m6v2q\" (UID: \"f5733389-a2ac-47dd-9fb4-8ba44621adb9\") " pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:20:59.232665 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:59.232646 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-krbdq\"" Sep 17 07:20:59.234737 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:59.234720 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7f254" Sep 17 07:20:59.239526 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:59.239513 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:20:59.509270 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:59.509230 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7f254"] Sep 17 07:20:59.512291 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:20:59.512249 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc285d69f_53b9_4e2a_a0d0_c72e4d592566.slice/crio-ea9e03261d1f886c193b0cc8cbfae002a019824ed7a0920e9bb4d535073a727f WatchSource:0}: Error finding container ea9e03261d1f886c193b0cc8cbfae002a019824ed7a0920e9bb4d535073a727f: Status 404 returned error can't find the container with id ea9e03261d1f886c193b0cc8cbfae002a019824ed7a0920e9bb4d535073a727f Sep 17 07:20:59.515570 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:59.514062 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m6v2q"] Sep 17 07:20:59.516949 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:20:59.516925 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5733389_a2ac_47dd_9fb4_8ba44621adb9.slice/crio-5066003e107dd69e0cf7725fcaf8902d9f9fe5ccfebe916977215582edfa3325 WatchSource:0}: Error finding container 5066003e107dd69e0cf7725fcaf8902d9f9fe5ccfebe916977215582edfa3325: Status 404 returned error can't find the container with id 5066003e107dd69e0cf7725fcaf8902d9f9fe5ccfebe916977215582edfa3325 Sep 17 07:20:59.707990 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:59.707961 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m6v2q" event={"ID":"f5733389-a2ac-47dd-9fb4-8ba44621adb9","Type":"ContainerStarted","Data":"5066003e107dd69e0cf7725fcaf8902d9f9fe5ccfebe916977215582edfa3325"} Sep 17 07:20:59.708950 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:20:59.708933 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7f254" event={"ID":"c285d69f-53b9-4e2a-a0d0-c72e4d592566","Type":"ContainerStarted","Data":"ea9e03261d1f886c193b0cc8cbfae002a019824ed7a0920e9bb4d535073a727f"} Sep 17 07:21:05.727897 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:05.727857 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m6v2q" event={"ID":"f5733389-a2ac-47dd-9fb4-8ba44621adb9","Type":"ContainerStarted","Data":"7eb60a2c8c37a8900a522ae0b4f50bb8be45708f8cce55ecc87a805adb052a67"} Sep 17 07:21:05.728388 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:05.728004 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:21:05.729374 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:05.729351 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7f254" event={"ID":"c285d69f-53b9-4e2a-a0d0-c72e4d592566","Type":"ContainerStarted","Data":"e68c8e28deea2e391ba742f31d62103c6e5adfa02787a125b7d21343d607ac19"} Sep 17 07:21:05.748584 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:05.748537 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-m6v2q" podStartSLOduration=67.485389531 podStartE2EDuration="1m12.748522233s" totalImagesPullingTime="5.263132702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:21:05.74698868 +0000 UTC m=+72.932721652" watchObservedRunningTime="2026-09-17 07:21:05.748522233 +0000 UTC m=+72.934255205" Sep 17 07:21:05.765231 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:05.765179 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7f254" podStartSLOduration=67.493637191 podStartE2EDuration="1m12.765160061s" totalImagesPullingTime="5.27152287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:21:05.76504965 +0000 UTC m=+72.950782622" watchObservedRunningTime="2026-09-17 07:21:05.765160061 +0000 UTC m=+72.950893035" Sep 17 07:21:31.026964 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:31.026835 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls\") pod \"dns-default-tr629\" (UID: \"f3305365-b6fc-4a79-97c8-63209d1d71ea\") " pod="openshift-dns/dns-default-tr629" Sep 17 07:21:31.027376 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:31.026995 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 07:21:31.027376 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:31.027079 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls podName:f3305365-b6fc-4a79-97c8-63209d1d71ea nodeName:}" failed. No retries permitted until 2026-09-17 07:22:35.027055636 +0000 UTC m=+162.212788587 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls") pod "dns-default-tr629" (UID: "f3305365-b6fc-4a79-97c8-63209d1d71ea") : secret "dns-default-metrics-tls" not found Sep 17 07:21:36.734359 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:36.734311 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-m6v2q" Sep 17 07:21:41.638531 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.638495 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-b4rsw"] Sep 17 07:21:41.641334 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.641305 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b4rsw" Sep 17 07:21:41.647387 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.646984 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 17 07:21:41.647387 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.647031 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2vz69\"" Sep 17 07:21:41.647387 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.647048 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 17 07:21:41.647387 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.647263 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 17 07:21:41.647627 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.647470 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 17 07:21:41.662956 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.662924 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-b4rsw"] Sep 17 07:21:41.700470 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.700435 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d07236c4-ae38-4ffc-b1ca-288072181c8d-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-b4rsw\" (UID: \"d07236c4-ae38-4ffc-b1ca-288072181c8d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b4rsw" Sep 17 07:21:41.700636 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.700472 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d07236c4-ae38-4ffc-b1ca-288072181c8d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b4rsw\" (UID: \"d07236c4-ae38-4ffc-b1ca-288072181c8d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b4rsw" Sep 17 07:21:41.700636 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.700548 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snxmm\" (UniqueName: \"kubernetes.io/projected/d07236c4-ae38-4ffc-b1ca-288072181c8d-kube-api-access-snxmm\") pod \"cluster-monitoring-operator-6cd954b897-b4rsw\" (UID: \"d07236c4-ae38-4ffc-b1ca-288072181c8d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b4rsw" Sep 17 07:21:41.751280 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.751248 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-sjghs"] Sep 17 07:21:41.753213 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.753199 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-sjghs" Sep 17 07:21:41.757014 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.756993 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 17 07:21:41.757387 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.757371 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-s5pjm\"" Sep 17 07:21:41.757550 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.757533 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-fwkfn"] Sep 17 07:21:41.757826 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.757811 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 17 07:21:41.759444 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.759424 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-fwkfn" Sep 17 07:21:41.763781 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.763762 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 17 07:21:41.764161 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.764146 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 17 07:21:41.764262 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.764237 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 17 07:21:41.764488 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.764474 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-lv5fp\"" Sep 17 07:21:41.764676 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.764663 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 17 07:21:41.770863 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.770844 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 17 07:21:41.776162 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.776143 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-sjghs"] Sep 17 07:21:41.786752 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.786726 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-fwkfn"] Sep 17 07:21:41.800851 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.800826 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d07236c4-ae38-4ffc-b1ca-288072181c8d-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-b4rsw\" (UID: \"d07236c4-ae38-4ffc-b1ca-288072181c8d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b4rsw" Sep 17 07:21:41.800991 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.800855 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d07236c4-ae38-4ffc-b1ca-288072181c8d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b4rsw\" (UID: \"d07236c4-ae38-4ffc-b1ca-288072181c8d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b4rsw" Sep 17 07:21:41.800991 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.800892 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snxmm\" (UniqueName: \"kubernetes.io/projected/d07236c4-ae38-4ffc-b1ca-288072181c8d-kube-api-access-snxmm\") pod \"cluster-monitoring-operator-6cd954b897-b4rsw\" (UID: \"d07236c4-ae38-4ffc-b1ca-288072181c8d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b4rsw" Sep 17 07:21:41.801065 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:41.800994 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 07:21:41.801065 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:41.801061 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d07236c4-ae38-4ffc-b1ca-288072181c8d-cluster-monitoring-operator-tls podName:d07236c4-ae38-4ffc-b1ca-288072181c8d nodeName:}" failed. No retries permitted until 2026-09-17 07:21:42.301040403 +0000 UTC m=+109.486773367 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d07236c4-ae38-4ffc-b1ca-288072181c8d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-b4rsw" (UID: "d07236c4-ae38-4ffc-b1ca-288072181c8d") : secret "cluster-monitoring-operator-tls" not found Sep 17 07:21:41.801503 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.801487 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d07236c4-ae38-4ffc-b1ca-288072181c8d-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-b4rsw\" (UID: \"d07236c4-ae38-4ffc-b1ca-288072181c8d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b4rsw" Sep 17 07:21:41.820574 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.820552 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snxmm\" (UniqueName: \"kubernetes.io/projected/d07236c4-ae38-4ffc-b1ca-288072181c8d-kube-api-access-snxmm\") pod \"cluster-monitoring-operator-6cd954b897-b4rsw\" (UID: \"d07236c4-ae38-4ffc-b1ca-288072181c8d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b4rsw" Sep 17 07:21:41.901763 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.901666 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c7192a-8c88-4409-ba0a-811cccbc2112-trusted-ca-bundle\") pod \"insights-operator-5844b856db-fwkfn\" (UID: \"59c7192a-8c88-4409-ba0a-811cccbc2112\") " pod="openshift-insights/insights-operator-5844b856db-fwkfn" Sep 17 07:21:41.901763 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.901731 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59c7192a-8c88-4409-ba0a-811cccbc2112-tmp\") pod \"insights-operator-5844b856db-fwkfn\" (UID: \"59c7192a-8c88-4409-ba0a-811cccbc2112\") " pod="openshift-insights/insights-operator-5844b856db-fwkfn" Sep 17 07:21:41.901946 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.901770 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vldz2\" (UniqueName: \"kubernetes.io/projected/59c7192a-8c88-4409-ba0a-811cccbc2112-kube-api-access-vldz2\") pod \"insights-operator-5844b856db-fwkfn\" (UID: \"59c7192a-8c88-4409-ba0a-811cccbc2112\") " pod="openshift-insights/insights-operator-5844b856db-fwkfn" Sep 17 07:21:41.901946 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.901869 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5ksw\" (UniqueName: \"kubernetes.io/projected/cd69a0d7-2a67-4f2e-9331-10d0874d1063-kube-api-access-b5ksw\") pod \"volume-data-source-validator-86695b7cc7-sjghs\" (UID: \"cd69a0d7-2a67-4f2e-9331-10d0874d1063\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-sjghs" Sep 17 07:21:41.901946 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.901901 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c7192a-8c88-4409-ba0a-811cccbc2112-service-ca-bundle\") pod \"insights-operator-5844b856db-fwkfn\" (UID: \"59c7192a-8c88-4409-ba0a-811cccbc2112\") " pod="openshift-insights/insights-operator-5844b856db-fwkfn" Sep 17 07:21:41.901946 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.901927 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/59c7192a-8c88-4409-ba0a-811cccbc2112-snapshots\") pod \"insights-operator-5844b856db-fwkfn\" (UID: \"59c7192a-8c88-4409-ba0a-811cccbc2112\") " pod="openshift-insights/insights-operator-5844b856db-fwkfn" Sep 17 07:21:41.902108 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:41.901969 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/59c7192a-8c88-4409-ba0a-811cccbc2112-serving-cert\") pod \"insights-operator-5844b856db-fwkfn\" (UID: \"59c7192a-8c88-4409-ba0a-811cccbc2112\") " pod="openshift-insights/insights-operator-5844b856db-fwkfn" Sep 17 07:21:42.003260 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:42.003206 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/59c7192a-8c88-4409-ba0a-811cccbc2112-serving-cert\") pod \"insights-operator-5844b856db-fwkfn\" (UID: \"59c7192a-8c88-4409-ba0a-811cccbc2112\") " pod="openshift-insights/insights-operator-5844b856db-fwkfn" Sep 17 07:21:42.003260 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:42.003261 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c7192a-8c88-4409-ba0a-811cccbc2112-trusted-ca-bundle\") pod \"insights-operator-5844b856db-fwkfn\" (UID: \"59c7192a-8c88-4409-ba0a-811cccbc2112\") " pod="openshift-insights/insights-operator-5844b856db-fwkfn" Sep 17 07:21:42.003503 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:42.003301 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59c7192a-8c88-4409-ba0a-811cccbc2112-tmp\") pod \"insights-operator-5844b856db-fwkfn\" (UID: \"59c7192a-8c88-4409-ba0a-811cccbc2112\") " pod="openshift-insights/insights-operator-5844b856db-fwkfn" Sep 17 07:21:42.003503 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:42.003317 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vldz2\" (UniqueName: \"kubernetes.io/projected/59c7192a-8c88-4409-ba0a-811cccbc2112-kube-api-access-vldz2\") pod \"insights-operator-5844b856db-fwkfn\" (UID: \"59c7192a-8c88-4409-ba0a-811cccbc2112\") " pod="openshift-insights/insights-operator-5844b856db-fwkfn" Sep 17 07:21:42.003503 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:42.003363 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5ksw\" (UniqueName: \"kubernetes.io/projected/cd69a0d7-2a67-4f2e-9331-10d0874d1063-kube-api-access-b5ksw\") pod \"volume-data-source-validator-86695b7cc7-sjghs\" (UID: \"cd69a0d7-2a67-4f2e-9331-10d0874d1063\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-sjghs" Sep 17 07:21:42.003503 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:42.003377 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c7192a-8c88-4409-ba0a-811cccbc2112-service-ca-bundle\") pod \"insights-operator-5844b856db-fwkfn\" (UID: \"59c7192a-8c88-4409-ba0a-811cccbc2112\") " pod="openshift-insights/insights-operator-5844b856db-fwkfn" Sep 17 07:21:42.003503 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:42.003399 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/59c7192a-8c88-4409-ba0a-811cccbc2112-snapshots\") pod \"insights-operator-5844b856db-fwkfn\" (UID: \"59c7192a-8c88-4409-ba0a-811cccbc2112\") " pod="openshift-insights/insights-operator-5844b856db-fwkfn" Sep 17 07:21:42.004011 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:42.003989 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59c7192a-8c88-4409-ba0a-811cccbc2112-tmp\") pod \"insights-operator-5844b856db-fwkfn\" (UID: \"59c7192a-8c88-4409-ba0a-811cccbc2112\") " pod="openshift-insights/insights-operator-5844b856db-fwkfn" Sep 17 07:21:42.004093 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:42.004028 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/59c7192a-8c88-4409-ba0a-811cccbc2112-snapshots\") pod \"insights-operator-5844b856db-fwkfn\" (UID: \"59c7192a-8c88-4409-ba0a-811cccbc2112\") " pod="openshift-insights/insights-operator-5844b856db-fwkfn" Sep 17 07:21:42.004194 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:42.004096 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c7192a-8c88-4409-ba0a-811cccbc2112-service-ca-bundle\") pod \"insights-operator-5844b856db-fwkfn\" (UID: \"59c7192a-8c88-4409-ba0a-811cccbc2112\") " pod="openshift-insights/insights-operator-5844b856db-fwkfn" Sep 17 07:21:42.004293 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:42.004277 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59c7192a-8c88-4409-ba0a-811cccbc2112-trusted-ca-bundle\") pod \"insights-operator-5844b856db-fwkfn\" (UID: \"59c7192a-8c88-4409-ba0a-811cccbc2112\") " pod="openshift-insights/insights-operator-5844b856db-fwkfn" Sep 17 07:21:42.005844 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:42.005818 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/59c7192a-8c88-4409-ba0a-811cccbc2112-serving-cert\") pod \"insights-operator-5844b856db-fwkfn\" (UID: \"59c7192a-8c88-4409-ba0a-811cccbc2112\") " pod="openshift-insights/insights-operator-5844b856db-fwkfn" Sep 17 07:21:42.013708 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:42.013683 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vldz2\" (UniqueName: \"kubernetes.io/projected/59c7192a-8c88-4409-ba0a-811cccbc2112-kube-api-access-vldz2\") pod \"insights-operator-5844b856db-fwkfn\" (UID: \"59c7192a-8c88-4409-ba0a-811cccbc2112\") " pod="openshift-insights/insights-operator-5844b856db-fwkfn" Sep 17 07:21:42.013798 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:42.013713 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5ksw\" (UniqueName: \"kubernetes.io/projected/cd69a0d7-2a67-4f2e-9331-10d0874d1063-kube-api-access-b5ksw\") pod \"volume-data-source-validator-86695b7cc7-sjghs\" (UID: \"cd69a0d7-2a67-4f2e-9331-10d0874d1063\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-sjghs" Sep 17 07:21:42.062700 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:42.062661 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-sjghs" Sep 17 07:21:42.068628 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:42.068605 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-fwkfn" Sep 17 07:21:42.305946 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:42.305903 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d07236c4-ae38-4ffc-b1ca-288072181c8d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b4rsw\" (UID: \"d07236c4-ae38-4ffc-b1ca-288072181c8d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b4rsw" Sep 17 07:21:42.306114 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:42.306071 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 07:21:42.306161 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:42.306155 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d07236c4-ae38-4ffc-b1ca-288072181c8d-cluster-monitoring-operator-tls podName:d07236c4-ae38-4ffc-b1ca-288072181c8d nodeName:}" failed. No retries permitted until 2026-09-17 07:21:43.306138815 +0000 UTC m=+110.491871765 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d07236c4-ae38-4ffc-b1ca-288072181c8d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-b4rsw" (UID: "d07236c4-ae38-4ffc-b1ca-288072181c8d") : secret "cluster-monitoring-operator-tls" not found Sep 17 07:21:42.348955 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:42.348923 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-sjghs"] Sep 17 07:21:42.354440 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:21:42.354415 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd69a0d7_2a67_4f2e_9331_10d0874d1063.slice/crio-33b16698466f896e47223357db78d40d999cbccfad1bd84a875eeaa5598e145d WatchSource:0}: Error finding container 33b16698466f896e47223357db78d40d999cbccfad1bd84a875eeaa5598e145d: Status 404 returned error can't find the container with id 33b16698466f896e47223357db78d40d999cbccfad1bd84a875eeaa5598e145d Sep 17 07:21:42.366440 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:42.366415 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-fwkfn"] Sep 17 07:21:42.369235 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:21:42.369213 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59c7192a_8c88_4409_ba0a_811cccbc2112.slice/crio-73e4aab4a84413632127a7cb3089431fae1c16fa68f47bc796e599cbd1a9baf8 WatchSource:0}: Error finding container 73e4aab4a84413632127a7cb3089431fae1c16fa68f47bc796e599cbd1a9baf8: Status 404 returned error can't find the container with id 73e4aab4a84413632127a7cb3089431fae1c16fa68f47bc796e599cbd1a9baf8 Sep 17 07:21:42.821821 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:42.821779 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-sjghs" event={"ID":"cd69a0d7-2a67-4f2e-9331-10d0874d1063","Type":"ContainerStarted","Data":"33b16698466f896e47223357db78d40d999cbccfad1bd84a875eeaa5598e145d"} Sep 17 07:21:42.822918 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:42.822897 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-fwkfn" event={"ID":"59c7192a-8c88-4409-ba0a-811cccbc2112","Type":"ContainerStarted","Data":"73e4aab4a84413632127a7cb3089431fae1c16fa68f47bc796e599cbd1a9baf8"} Sep 17 07:21:43.313409 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:43.313372 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d07236c4-ae38-4ffc-b1ca-288072181c8d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b4rsw\" (UID: \"d07236c4-ae38-4ffc-b1ca-288072181c8d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b4rsw" Sep 17 07:21:43.313582 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:43.313497 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 07:21:43.313582 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:43.313558 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d07236c4-ae38-4ffc-b1ca-288072181c8d-cluster-monitoring-operator-tls podName:d07236c4-ae38-4ffc-b1ca-288072181c8d nodeName:}" failed. No retries permitted until 2026-09-17 07:21:45.313542714 +0000 UTC m=+112.499275666 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d07236c4-ae38-4ffc-b1ca-288072181c8d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-b4rsw" (UID: "d07236c4-ae38-4ffc-b1ca-288072181c8d") : secret "cluster-monitoring-operator-tls" not found Sep 17 07:21:43.828221 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:43.828185 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-sjghs" event={"ID":"cd69a0d7-2a67-4f2e-9331-10d0874d1063","Type":"ContainerStarted","Data":"d6e685b7bc2414d3a5a7801cbfd1ac1664b39b03bea10cc65cd8855c0235111e"} Sep 17 07:21:44.832505 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:44.832460 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-fwkfn" event={"ID":"59c7192a-8c88-4409-ba0a-811cccbc2112","Type":"ContainerStarted","Data":"6577e0426941d17790ba2170603f28ab717183eebecff7add3a1b57add291ec1"} Sep 17 07:21:44.855790 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:44.855723 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-sjghs" podStartSLOduration=2.474113976 podStartE2EDuration="3.855701041s" totalImagesPullingTime="1.381587065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:21:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:21:43.852840558 +0000 UTC m=+111.038573529" watchObservedRunningTime="2026-09-17 07:21:44.855701041 +0000 UTC m=+112.041434015" Sep 17 07:21:44.856137 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:44.856110 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-fwkfn" podStartSLOduration=1.848236915 podStartE2EDuration="3.856100656s" totalImagesPullingTime="2.007863741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:21:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:21:44.855509511 +0000 UTC m=+112.041242493" watchObservedRunningTime="2026-09-17 07:21:44.856100656 +0000 UTC m=+112.041833627" Sep 17 07:21:45.131061 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:45.130984 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lzhlj"] Sep 17 07:21:45.133388 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:45.133367 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:21:45.136799 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:45.136775 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 17 07:21:45.136916 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:45.136813 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 17 07:21:45.136986 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:45.136955 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ghp9r\"" Sep 17 07:21:45.150649 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:45.150623 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lzhlj"] Sep 17 07:21:45.228742 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:45.228701 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/552cd700-3fb9-4411-9eda-3374a359f156-crio-socket\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:21:45.228742 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:45.228739 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/552cd700-3fb9-4411-9eda-3374a359f156-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:21:45.229007 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:45.228783 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjd7d\" (UniqueName: \"kubernetes.io/projected/552cd700-3fb9-4411-9eda-3374a359f156-kube-api-access-jjd7d\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:21:45.229007 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:45.228886 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/552cd700-3fb9-4411-9eda-3374a359f156-data-volume\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:21:45.229007 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:45.228931 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/552cd700-3fb9-4411-9eda-3374a359f156-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:21:45.329736 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:45.329690 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjd7d\" (UniqueName: \"kubernetes.io/projected/552cd700-3fb9-4411-9eda-3374a359f156-kube-api-access-jjd7d\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:21:45.329915 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:45.329753 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d07236c4-ae38-4ffc-b1ca-288072181c8d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b4rsw\" (UID: \"d07236c4-ae38-4ffc-b1ca-288072181c8d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b4rsw" Sep 17 07:21:45.329915 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:45.329777 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/552cd700-3fb9-4411-9eda-3374a359f156-data-volume\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:21:45.329915 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:45.329806 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/552cd700-3fb9-4411-9eda-3374a359f156-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:21:45.329915 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:45.329866 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/552cd700-3fb9-4411-9eda-3374a359f156-crio-socket\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:21:45.329915 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:45.329889 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/552cd700-3fb9-4411-9eda-3374a359f156-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:21:45.330159 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:45.329924 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 07:21:45.330159 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:45.329990 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 07:21:45.330159 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:45.330001 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d07236c4-ae38-4ffc-b1ca-288072181c8d-cluster-monitoring-operator-tls podName:d07236c4-ae38-4ffc-b1ca-288072181c8d nodeName:}" failed. No retries permitted until 2026-09-17 07:21:49.329980187 +0000 UTC m=+116.515713141 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d07236c4-ae38-4ffc-b1ca-288072181c8d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-b4rsw" (UID: "d07236c4-ae38-4ffc-b1ca-288072181c8d") : secret "cluster-monitoring-operator-tls" not found Sep 17 07:21:45.330159 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:45.330007 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 07:21:45.330159 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:45.330008 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/552cd700-3fb9-4411-9eda-3374a359f156-crio-socket\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:21:45.330159 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:45.330052 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/552cd700-3fb9-4411-9eda-3374a359f156-kube-rbac-proxy-cm podName:552cd700-3fb9-4411-9eda-3374a359f156 nodeName:}" failed. No retries permitted until 2026-09-17 07:21:45.830035764 +0000 UTC m=+113.015768717 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/552cd700-3fb9-4411-9eda-3374a359f156-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lzhlj" (UID: "552cd700-3fb9-4411-9eda-3374a359f156") : configmap "kube-rbac-proxy" not found Sep 17 07:21:45.330159 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:45.330073 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/552cd700-3fb9-4411-9eda-3374a359f156-insights-runtime-extractor-tls podName:552cd700-3fb9-4411-9eda-3374a359f156 nodeName:}" failed. No retries permitted until 2026-09-17 07:21:45.830063208 +0000 UTC m=+113.015796158 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/552cd700-3fb9-4411-9eda-3374a359f156-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lzhlj" (UID: "552cd700-3fb9-4411-9eda-3374a359f156") : secret "insights-runtime-extractor-tls" not found Sep 17 07:21:45.330413 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:45.330162 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/552cd700-3fb9-4411-9eda-3374a359f156-data-volume\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:21:45.344132 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:45.344105 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjd7d\" (UniqueName: \"kubernetes.io/projected/552cd700-3fb9-4411-9eda-3374a359f156-kube-api-access-jjd7d\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:21:45.833678 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:45.833642 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/552cd700-3fb9-4411-9eda-3374a359f156-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:21:45.834118 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:45.833708 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/552cd700-3fb9-4411-9eda-3374a359f156-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:21:45.834118 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:45.833799 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 07:21:45.834118 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:45.833821 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 07:21:45.834118 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:45.833879 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/552cd700-3fb9-4411-9eda-3374a359f156-kube-rbac-proxy-cm podName:552cd700-3fb9-4411-9eda-3374a359f156 nodeName:}" failed. No retries permitted until 2026-09-17 07:21:46.833861539 +0000 UTC m=+114.019594489 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/552cd700-3fb9-4411-9eda-3374a359f156-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lzhlj" (UID: "552cd700-3fb9-4411-9eda-3374a359f156") : configmap "kube-rbac-proxy" not found Sep 17 07:21:45.834118 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:45.833896 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/552cd700-3fb9-4411-9eda-3374a359f156-insights-runtime-extractor-tls podName:552cd700-3fb9-4411-9eda-3374a359f156 nodeName:}" failed. No retries permitted until 2026-09-17 07:21:46.833888258 +0000 UTC m=+114.019621216 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/552cd700-3fb9-4411-9eda-3374a359f156-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lzhlj" (UID: "552cd700-3fb9-4411-9eda-3374a359f156") : secret "insights-runtime-extractor-tls" not found Sep 17 07:21:46.627057 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:46.627021 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gxzj"] Sep 17 07:21:46.629441 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:46.629411 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gxzj" Sep 17 07:21:46.632372 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:46.632347 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 17 07:21:46.632582 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:46.632565 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 17 07:21:46.633471 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:46.633451 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 17 07:21:46.633581 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:46.633535 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 17 07:21:46.633581 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:46.633536 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-8l84h\"" Sep 17 07:21:46.642588 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:46.642553 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gxzj"] Sep 17 07:21:46.741660 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:46.741629 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b4e5f925-07d2-41f5-b94b-f275630921ed-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-4gxzj\" (UID: \"b4e5f925-07d2-41f5-b94b-f275630921ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gxzj" Sep 17 07:21:46.741845 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:46.741703 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dmgz\" (UniqueName: \"kubernetes.io/projected/b4e5f925-07d2-41f5-b94b-f275630921ed-kube-api-access-9dmgz\") pod \"kube-storage-version-migrator-operator-6c8f84f897-4gxzj\" (UID: \"b4e5f925-07d2-41f5-b94b-f275630921ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gxzj" Sep 17 07:21:46.741845 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:46.741730 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b4e5f925-07d2-41f5-b94b-f275630921ed-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-4gxzj\" (UID: \"b4e5f925-07d2-41f5-b94b-f275630921ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gxzj" Sep 17 07:21:46.842829 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:46.842793 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b4e5f925-07d2-41f5-b94b-f275630921ed-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-4gxzj\" (UID: \"b4e5f925-07d2-41f5-b94b-f275630921ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gxzj" Sep 17 07:21:46.843290 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:46.842837 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/552cd700-3fb9-4411-9eda-3374a359f156-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:21:46.843290 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:46.843024 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 07:21:46.843290 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:46.843088 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/552cd700-3fb9-4411-9eda-3374a359f156-insights-runtime-extractor-tls podName:552cd700-3fb9-4411-9eda-3374a359f156 nodeName:}" failed. No retries permitted until 2026-09-17 07:21:48.84306878 +0000 UTC m=+116.028801733 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/552cd700-3fb9-4411-9eda-3374a359f156-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lzhlj" (UID: "552cd700-3fb9-4411-9eda-3374a359f156") : secret "insights-runtime-extractor-tls" not found Sep 17 07:21:46.843290 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:46.843156 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/552cd700-3fb9-4411-9eda-3374a359f156-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:21:46.843290 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:46.843182 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dmgz\" (UniqueName: \"kubernetes.io/projected/b4e5f925-07d2-41f5-b94b-f275630921ed-kube-api-access-9dmgz\") pod \"kube-storage-version-migrator-operator-6c8f84f897-4gxzj\" (UID: \"b4e5f925-07d2-41f5-b94b-f275630921ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gxzj" Sep 17 07:21:46.843290 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:46.843214 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b4e5f925-07d2-41f5-b94b-f275630921ed-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-4gxzj\" (UID: \"b4e5f925-07d2-41f5-b94b-f275630921ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gxzj" Sep 17 07:21:46.843290 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:46.843252 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 07:21:46.843686 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:46.843344 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/552cd700-3fb9-4411-9eda-3374a359f156-kube-rbac-proxy-cm podName:552cd700-3fb9-4411-9eda-3374a359f156 nodeName:}" failed. No retries permitted until 2026-09-17 07:21:48.843304982 +0000 UTC m=+116.029037932 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/552cd700-3fb9-4411-9eda-3374a359f156-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lzhlj" (UID: "552cd700-3fb9-4411-9eda-3374a359f156") : configmap "kube-rbac-proxy" not found Sep 17 07:21:46.843863 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:46.843842 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b4e5f925-07d2-41f5-b94b-f275630921ed-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-4gxzj\" (UID: \"b4e5f925-07d2-41f5-b94b-f275630921ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gxzj" Sep 17 07:21:46.845404 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:46.845387 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b4e5f925-07d2-41f5-b94b-f275630921ed-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-4gxzj\" (UID: \"b4e5f925-07d2-41f5-b94b-f275630921ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gxzj" Sep 17 07:21:46.857345 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:46.857311 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dmgz\" (UniqueName: \"kubernetes.io/projected/b4e5f925-07d2-41f5-b94b-f275630921ed-kube-api-access-9dmgz\") pod \"kube-storage-version-migrator-operator-6c8f84f897-4gxzj\" (UID: \"b4e5f925-07d2-41f5-b94b-f275630921ed\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gxzj" Sep 17 07:21:46.938591 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:46.938499 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gxzj" Sep 17 07:21:47.215591 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.215565 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gxzj"] Sep 17 07:21:47.217273 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:21:47.217246 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4e5f925_07d2_41f5_b94b_f275630921ed.slice/crio-dc9eb240a5398b3a9d7f972e204c3cc3c893b9019f7ca187df4e887f42265c86 WatchSource:0}: Error finding container dc9eb240a5398b3a9d7f972e204c3cc3c893b9019f7ca187df4e887f42265c86: Status 404 returned error can't find the container with id dc9eb240a5398b3a9d7f972e204c3cc3c893b9019f7ca187df4e887f42265c86 Sep 17 07:21:47.252296 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.252267 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tnmcf_9c0e9d0a-00c1-479b-a811-68a9ca231e9f/dns-node-resolver/0.log" Sep 17 07:21:47.417963 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.417932 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7cc57c98c5-9ftmq"] Sep 17 07:21:47.421135 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.421119 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:47.424852 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.424826 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 17 07:21:47.424978 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.424860 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 17 07:21:47.424978 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.424904 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-n9snn\"" Sep 17 07:21:47.424978 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.424951 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 17 07:21:47.431206 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.431188 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 17 07:21:47.446371 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.446347 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cc57c98c5-9ftmq"] Sep 17 07:21:47.549274 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.549182 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/23825dc0-31ec-4002-8389-addd925bf1d9-registry-certificates\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:47.549274 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.549232 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-bound-sa-token\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:47.549527 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.549315 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/23825dc0-31ec-4002-8389-addd925bf1d9-image-registry-private-configuration\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:47.549527 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.549386 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/23825dc0-31ec-4002-8389-addd925bf1d9-ca-trust-extracted\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:47.549527 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.549412 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23825dc0-31ec-4002-8389-addd925bf1d9-trusted-ca\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:47.549527 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.549468 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-registry-tls\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:47.549527 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.549483 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4jlb\" (UniqueName: \"kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-kube-api-access-t4jlb\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:47.549527 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.549503 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/23825dc0-31ec-4002-8389-addd925bf1d9-installation-pull-secrets\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:47.648651 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.648620 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-k5dtg_ddbb72ee-6e56-4549-ac75-f8857a377394/node-ca/0.log" Sep 17 07:21:47.649892 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.649870 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/23825dc0-31ec-4002-8389-addd925bf1d9-registry-certificates\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:47.649995 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.649908 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-bound-sa-token\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:47.649995 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.649935 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/23825dc0-31ec-4002-8389-addd925bf1d9-image-registry-private-configuration\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:47.649995 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.649955 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/23825dc0-31ec-4002-8389-addd925bf1d9-ca-trust-extracted\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:47.650124 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.649999 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23825dc0-31ec-4002-8389-addd925bf1d9-trusted-ca\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:47.650124 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.650083 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-registry-tls\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:47.650223 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.650107 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4jlb\" (UniqueName: \"kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-kube-api-access-t4jlb\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:47.650223 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.650168 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/23825dc0-31ec-4002-8389-addd925bf1d9-installation-pull-secrets\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:47.650312 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:47.650297 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 07:21:47.650390 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:47.650314 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cc57c98c5-9ftmq: secret "image-registry-tls" not found Sep 17 07:21:47.650390 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.650367 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/23825dc0-31ec-4002-8389-addd925bf1d9-ca-trust-extracted\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:47.650490 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:47.650396 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-registry-tls podName:23825dc0-31ec-4002-8389-addd925bf1d9 nodeName:}" failed. No retries permitted until 2026-09-17 07:21:48.150377516 +0000 UTC m=+115.336110483 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-registry-tls") pod "image-registry-7cc57c98c5-9ftmq" (UID: "23825dc0-31ec-4002-8389-addd925bf1d9") : secret "image-registry-tls" not found Sep 17 07:21:47.650643 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.650623 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/23825dc0-31ec-4002-8389-addd925bf1d9-registry-certificates\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:47.651095 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.651075 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23825dc0-31ec-4002-8389-addd925bf1d9-trusted-ca\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:47.652815 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.652789 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/23825dc0-31ec-4002-8389-addd925bf1d9-image-registry-private-configuration\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:47.652897 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.652844 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/23825dc0-31ec-4002-8389-addd925bf1d9-installation-pull-secrets\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:47.661468 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.661449 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-bound-sa-token\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:47.661533 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.661490 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4jlb\" (UniqueName: \"kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-kube-api-access-t4jlb\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:47.841200 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:47.841113 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gxzj" event={"ID":"b4e5f925-07d2-41f5-b94b-f275630921ed","Type":"ContainerStarted","Data":"dc9eb240a5398b3a9d7f972e204c3cc3c893b9019f7ca187df4e887f42265c86"} Sep 17 07:21:48.154051 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:48.153968 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-registry-tls\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:48.154516 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:48.154149 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 07:21:48.154516 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:48.154170 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cc57c98c5-9ftmq: secret "image-registry-tls" not found Sep 17 07:21:48.154516 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:48.154225 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-registry-tls podName:23825dc0-31ec-4002-8389-addd925bf1d9 nodeName:}" failed. No retries permitted until 2026-09-17 07:21:49.154207373 +0000 UTC m=+116.339940338 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-registry-tls") pod "image-registry-7cc57c98c5-9ftmq" (UID: "23825dc0-31ec-4002-8389-addd925bf1d9") : secret "image-registry-tls" not found Sep 17 07:21:48.649613 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:48.649580 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hll2c_79621566-efae-43dd-9386-30a4a2aed04b/serve-healthcheck-canary/0.log" Sep 17 07:21:48.860199 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:48.860164 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/552cd700-3fb9-4411-9eda-3374a359f156-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:21:48.860412 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:48.860224 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/552cd700-3fb9-4411-9eda-3374a359f156-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:21:48.860412 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:48.860289 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 07:21:48.860412 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:48.860354 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 07:21:48.860412 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:48.860397 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/552cd700-3fb9-4411-9eda-3374a359f156-kube-rbac-proxy-cm podName:552cd700-3fb9-4411-9eda-3374a359f156 nodeName:}" failed. No retries permitted until 2026-09-17 07:21:52.860358326 +0000 UTC m=+120.046091300 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/552cd700-3fb9-4411-9eda-3374a359f156-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lzhlj" (UID: "552cd700-3fb9-4411-9eda-3374a359f156") : configmap "kube-rbac-proxy" not found Sep 17 07:21:48.860412 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:48.860413 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/552cd700-3fb9-4411-9eda-3374a359f156-insights-runtime-extractor-tls podName:552cd700-3fb9-4411-9eda-3374a359f156 nodeName:}" failed. No retries permitted until 2026-09-17 07:21:52.860406964 +0000 UTC m=+120.046139913 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/552cd700-3fb9-4411-9eda-3374a359f156-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lzhlj" (UID: "552cd700-3fb9-4411-9eda-3374a359f156") : secret "insights-runtime-extractor-tls" not found Sep 17 07:21:49.162212 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:49.162177 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-registry-tls\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:49.162603 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:49.162368 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 07:21:49.162603 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:49.162392 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cc57c98c5-9ftmq: secret "image-registry-tls" not found Sep 17 07:21:49.162603 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:49.162471 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-registry-tls podName:23825dc0-31ec-4002-8389-addd925bf1d9 nodeName:}" failed. No retries permitted until 2026-09-17 07:21:51.162445943 +0000 UTC m=+118.348178910 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-registry-tls") pod "image-registry-7cc57c98c5-9ftmq" (UID: "23825dc0-31ec-4002-8389-addd925bf1d9") : secret "image-registry-tls" not found Sep 17 07:21:49.364130 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:49.364092 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d07236c4-ae38-4ffc-b1ca-288072181c8d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b4rsw\" (UID: \"d07236c4-ae38-4ffc-b1ca-288072181c8d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b4rsw" Sep 17 07:21:49.364284 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:49.364134 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 07:21:49.364284 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:49.364210 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d07236c4-ae38-4ffc-b1ca-288072181c8d-cluster-monitoring-operator-tls podName:d07236c4-ae38-4ffc-b1ca-288072181c8d nodeName:}" failed. No retries permitted until 2026-09-17 07:21:57.364190763 +0000 UTC m=+124.549923719 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d07236c4-ae38-4ffc-b1ca-288072181c8d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-b4rsw" (UID: "d07236c4-ae38-4ffc-b1ca-288072181c8d") : secret "cluster-monitoring-operator-tls" not found Sep 17 07:21:49.850745 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:49.850709 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gxzj" event={"ID":"b4e5f925-07d2-41f5-b94b-f275630921ed","Type":"ContainerStarted","Data":"f555c742315b8665f30ee7ea17102b0d8b7dc6574e72c5ff635dfd1c4ba08de9"} Sep 17 07:21:49.871723 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:49.871676 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gxzj" podStartSLOduration=1.8809785909999999 podStartE2EDuration="3.871661966s" totalImagesPullingTime="1.990683375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:21:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:21:49.87068418 +0000 UTC m=+117.056417151" watchObservedRunningTime="2026-09-17 07:21:49.871661966 +0000 UTC m=+117.057394932" Sep 17 07:21:51.176771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:51.176739 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-registry-tls\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:51.177150 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:51.176875 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 07:21:51.177150 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:51.176892 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cc57c98c5-9ftmq: secret "image-registry-tls" not found Sep 17 07:21:51.177150 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:51.176944 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-registry-tls podName:23825dc0-31ec-4002-8389-addd925bf1d9 nodeName:}" failed. No retries permitted until 2026-09-17 07:21:55.176929094 +0000 UTC m=+122.362662043 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-registry-tls") pod "image-registry-7cc57c98c5-9ftmq" (UID: "23825dc0-31ec-4002-8389-addd925bf1d9") : secret "image-registry-tls" not found Sep 17 07:21:52.889011 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:52.888984 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/552cd700-3fb9-4411-9eda-3374a359f156-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:21:52.889513 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:52.889043 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/552cd700-3fb9-4411-9eda-3374a359f156-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:21:52.889513 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:52.889102 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 07:21:52.889513 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:52.889139 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 07:21:52.889513 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:52.889170 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/552cd700-3fb9-4411-9eda-3374a359f156-kube-rbac-proxy-cm podName:552cd700-3fb9-4411-9eda-3374a359f156 nodeName:}" failed. No retries permitted until 2026-09-17 07:22:00.889149737 +0000 UTC m=+128.074882688 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/552cd700-3fb9-4411-9eda-3374a359f156-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lzhlj" (UID: "552cd700-3fb9-4411-9eda-3374a359f156") : configmap "kube-rbac-proxy" not found Sep 17 07:21:52.889513 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:52.889188 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/552cd700-3fb9-4411-9eda-3374a359f156-insights-runtime-extractor-tls podName:552cd700-3fb9-4411-9eda-3374a359f156 nodeName:}" failed. No retries permitted until 2026-09-17 07:22:00.889175998 +0000 UTC m=+128.074908948 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/552cd700-3fb9-4411-9eda-3374a359f156-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lzhlj" (UID: "552cd700-3fb9-4411-9eda-3374a359f156") : secret "insights-runtime-extractor-tls" not found Sep 17 07:21:53.081948 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:53.081917 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-cqgql"] Sep 17 07:21:53.084917 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:53.084901 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-cqgql" Sep 17 07:21:53.087662 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:53.087635 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 17 07:21:53.087662 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:53.087637 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-ll9jf\"" Sep 17 07:21:53.088628 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:53.088609 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 17 07:21:53.101118 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:53.101096 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-cqgql"] Sep 17 07:21:53.190883 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:53.190825 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8de810bd-a229-40bc-b8e5-9d02eeb9a501-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-cqgql\" (UID: \"8de810bd-a229-40bc-b8e5-9d02eeb9a501\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-cqgql" Sep 17 07:21:53.190883 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:53.190860 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8de810bd-a229-40bc-b8e5-9d02eeb9a501-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-cqgql\" (UID: \"8de810bd-a229-40bc-b8e5-9d02eeb9a501\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-cqgql" Sep 17 07:21:53.292070 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:53.292045 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8de810bd-a229-40bc-b8e5-9d02eeb9a501-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-cqgql\" (UID: \"8de810bd-a229-40bc-b8e5-9d02eeb9a501\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-cqgql" Sep 17 07:21:53.292202 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:53.292085 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8de810bd-a229-40bc-b8e5-9d02eeb9a501-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-cqgql\" (UID: \"8de810bd-a229-40bc-b8e5-9d02eeb9a501\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-cqgql" Sep 17 07:21:53.295216 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:53.295200 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 17 07:21:53.295706 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:53.295694 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 17 07:21:53.303124 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:53.303107 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 07:21:53.303176 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:53.303162 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8de810bd-a229-40bc-b8e5-9d02eeb9a501-networking-console-plugin-cert podName:8de810bd-a229-40bc-b8e5-9d02eeb9a501 nodeName:}" failed. No retries permitted until 2026-09-17 07:21:53.803145949 +0000 UTC m=+120.988878912 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8de810bd-a229-40bc-b8e5-9d02eeb9a501-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-cqgql" (UID: "8de810bd-a229-40bc-b8e5-9d02eeb9a501") : secret "networking-console-plugin-cert" not found Sep 17 07:21:53.303578 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:53.303562 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8de810bd-a229-40bc-b8e5-9d02eeb9a501-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-cqgql\" (UID: \"8de810bd-a229-40bc-b8e5-9d02eeb9a501\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-cqgql" Sep 17 07:21:53.896149 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:53.896109 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8de810bd-a229-40bc-b8e5-9d02eeb9a501-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-cqgql\" (UID: \"8de810bd-a229-40bc-b8e5-9d02eeb9a501\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-cqgql" Sep 17 07:21:53.896558 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:53.896271 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 07:21:53.896558 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:53.896345 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8de810bd-a229-40bc-b8e5-9d02eeb9a501-networking-console-plugin-cert podName:8de810bd-a229-40bc-b8e5-9d02eeb9a501 nodeName:}" failed. No retries permitted until 2026-09-17 07:21:54.896315161 +0000 UTC m=+122.082048117 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8de810bd-a229-40bc-b8e5-9d02eeb9a501-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-cqgql" (UID: "8de810bd-a229-40bc-b8e5-9d02eeb9a501") : secret "networking-console-plugin-cert" not found Sep 17 07:21:54.903852 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:54.903823 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8de810bd-a229-40bc-b8e5-9d02eeb9a501-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-cqgql\" (UID: \"8de810bd-a229-40bc-b8e5-9d02eeb9a501\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-cqgql" Sep 17 07:21:54.904210 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:54.903991 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 07:21:54.904210 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:54.904065 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8de810bd-a229-40bc-b8e5-9d02eeb9a501-networking-console-plugin-cert podName:8de810bd-a229-40bc-b8e5-9d02eeb9a501 nodeName:}" failed. No retries permitted until 2026-09-17 07:21:56.904046227 +0000 UTC m=+124.089779192 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8de810bd-a229-40bc-b8e5-9d02eeb9a501-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-cqgql" (UID: "8de810bd-a229-40bc-b8e5-9d02eeb9a501") : secret "networking-console-plugin-cert" not found Sep 17 07:21:55.206234 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:55.206206 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-registry-tls\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:21:55.206447 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:55.206394 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 07:21:55.206447 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:55.206413 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cc57c98c5-9ftmq: secret "image-registry-tls" not found Sep 17 07:21:55.206532 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:55.206472 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-registry-tls podName:23825dc0-31ec-4002-8389-addd925bf1d9 nodeName:}" failed. No retries permitted until 2026-09-17 07:22:03.206451993 +0000 UTC m=+130.392184944 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-registry-tls") pod "image-registry-7cc57c98c5-9ftmq" (UID: "23825dc0-31ec-4002-8389-addd925bf1d9") : secret "image-registry-tls" not found Sep 17 07:21:56.014759 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:56.014729 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-d6lpc"] Sep 17 07:21:56.017947 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:56.017931 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-d6lpc" Sep 17 07:21:56.022612 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:56.022591 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 17 07:21:56.023205 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:56.023186 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 17 07:21:56.023308 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:56.023234 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 17 07:21:56.024168 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:56.024136 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 17 07:21:56.024230 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:56.024218 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-swrcc\"" Sep 17 07:21:56.031826 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:56.031808 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-d6lpc"] Sep 17 07:21:56.113245 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:56.113218 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/84bbb407-a50d-4fc1-a17d-e338a59ed4b8-signing-key\") pod \"service-ca-568869fdc5-d6lpc\" (UID: \"84bbb407-a50d-4fc1-a17d-e338a59ed4b8\") " pod="openshift-service-ca/service-ca-568869fdc5-d6lpc" Sep 17 07:21:56.113357 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:56.113254 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zld8\" (UniqueName: \"kubernetes.io/projected/84bbb407-a50d-4fc1-a17d-e338a59ed4b8-kube-api-access-7zld8\") pod \"service-ca-568869fdc5-d6lpc\" (UID: \"84bbb407-a50d-4fc1-a17d-e338a59ed4b8\") " pod="openshift-service-ca/service-ca-568869fdc5-d6lpc" Sep 17 07:21:56.113357 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:56.113315 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/84bbb407-a50d-4fc1-a17d-e338a59ed4b8-signing-cabundle\") pod \"service-ca-568869fdc5-d6lpc\" (UID: \"84bbb407-a50d-4fc1-a17d-e338a59ed4b8\") " pod="openshift-service-ca/service-ca-568869fdc5-d6lpc" Sep 17 07:21:56.214596 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:56.214573 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/84bbb407-a50d-4fc1-a17d-e338a59ed4b8-signing-cabundle\") pod \"service-ca-568869fdc5-d6lpc\" (UID: \"84bbb407-a50d-4fc1-a17d-e338a59ed4b8\") " pod="openshift-service-ca/service-ca-568869fdc5-d6lpc" Sep 17 07:21:56.214708 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:56.214684 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/84bbb407-a50d-4fc1-a17d-e338a59ed4b8-signing-key\") pod \"service-ca-568869fdc5-d6lpc\" (UID: \"84bbb407-a50d-4fc1-a17d-e338a59ed4b8\") " pod="openshift-service-ca/service-ca-568869fdc5-d6lpc" Sep 17 07:21:56.214748 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:56.214718 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zld8\" (UniqueName: \"kubernetes.io/projected/84bbb407-a50d-4fc1-a17d-e338a59ed4b8-kube-api-access-7zld8\") pod \"service-ca-568869fdc5-d6lpc\" (UID: \"84bbb407-a50d-4fc1-a17d-e338a59ed4b8\") " pod="openshift-service-ca/service-ca-568869fdc5-d6lpc" Sep 17 07:21:56.215241 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:56.215212 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/84bbb407-a50d-4fc1-a17d-e338a59ed4b8-signing-cabundle\") pod \"service-ca-568869fdc5-d6lpc\" (UID: \"84bbb407-a50d-4fc1-a17d-e338a59ed4b8\") " pod="openshift-service-ca/service-ca-568869fdc5-d6lpc" Sep 17 07:21:56.217340 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:56.217306 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/84bbb407-a50d-4fc1-a17d-e338a59ed4b8-signing-key\") pod \"service-ca-568869fdc5-d6lpc\" (UID: \"84bbb407-a50d-4fc1-a17d-e338a59ed4b8\") " pod="openshift-service-ca/service-ca-568869fdc5-d6lpc" Sep 17 07:21:56.223956 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:56.223928 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zld8\" (UniqueName: \"kubernetes.io/projected/84bbb407-a50d-4fc1-a17d-e338a59ed4b8-kube-api-access-7zld8\") pod \"service-ca-568869fdc5-d6lpc\" (UID: \"84bbb407-a50d-4fc1-a17d-e338a59ed4b8\") " pod="openshift-service-ca/service-ca-568869fdc5-d6lpc" Sep 17 07:21:56.326662 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:56.326588 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-d6lpc" Sep 17 07:21:56.605294 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:56.605233 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-d6lpc"] Sep 17 07:21:56.608707 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:21:56.608679 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84bbb407_a50d_4fc1_a17d_e338a59ed4b8.slice/crio-fd8b75ce23ff421f56805d99c467c7123cfc753b40eb124848f6adeae4edecbc WatchSource:0}: Error finding container fd8b75ce23ff421f56805d99c467c7123cfc753b40eb124848f6adeae4edecbc: Status 404 returned error can't find the container with id fd8b75ce23ff421f56805d99c467c7123cfc753b40eb124848f6adeae4edecbc Sep 17 07:21:56.878071 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:56.877995 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-d6lpc" event={"ID":"84bbb407-a50d-4fc1-a17d-e338a59ed4b8","Type":"ContainerStarted","Data":"fd8b75ce23ff421f56805d99c467c7123cfc753b40eb124848f6adeae4edecbc"} Sep 17 07:21:56.921017 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:56.920994 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8de810bd-a229-40bc-b8e5-9d02eeb9a501-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-cqgql\" (UID: \"8de810bd-a229-40bc-b8e5-9d02eeb9a501\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-cqgql" Sep 17 07:21:56.921112 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:56.921095 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 07:21:56.921153 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:56.921145 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8de810bd-a229-40bc-b8e5-9d02eeb9a501-networking-console-plugin-cert podName:8de810bd-a229-40bc-b8e5-9d02eeb9a501 nodeName:}" failed. No retries permitted until 2026-09-17 07:22:00.921131335 +0000 UTC m=+128.106864298 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8de810bd-a229-40bc-b8e5-9d02eeb9a501-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-cqgql" (UID: "8de810bd-a229-40bc-b8e5-9d02eeb9a501") : secret "networking-console-plugin-cert" not found Sep 17 07:21:57.424922 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:57.424890 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d07236c4-ae38-4ffc-b1ca-288072181c8d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b4rsw\" (UID: \"d07236c4-ae38-4ffc-b1ca-288072181c8d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b4rsw" Sep 17 07:21:57.425371 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:57.425076 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 07:21:57.425371 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:21:57.425158 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d07236c4-ae38-4ffc-b1ca-288072181c8d-cluster-monitoring-operator-tls podName:d07236c4-ae38-4ffc-b1ca-288072181c8d nodeName:}" failed. No retries permitted until 2026-09-17 07:22:13.425135527 +0000 UTC m=+140.610868486 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d07236c4-ae38-4ffc-b1ca-288072181c8d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-b4rsw" (UID: "d07236c4-ae38-4ffc-b1ca-288072181c8d") : secret "cluster-monitoring-operator-tls" not found Sep 17 07:21:58.884651 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:58.884564 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-d6lpc" event={"ID":"84bbb407-a50d-4fc1-a17d-e338a59ed4b8","Type":"ContainerStarted","Data":"d3c8cb60225df97d0a0db3c6063a399d13ae81afa71fc18d0be7c6e700c95ca2"} Sep 17 07:21:58.908233 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:21:58.908190 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-d6lpc" podStartSLOduration=2.036821271 podStartE2EDuration="3.908175629s" totalImagesPullingTime="1.871354358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:21:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:21:58.906439722 +0000 UTC m=+126.092172693" watchObservedRunningTime="2026-09-17 07:21:58.908175629 +0000 UTC m=+126.093908600" Sep 17 07:22:00.955484 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:00.955454 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/552cd700-3fb9-4411-9eda-3374a359f156-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:22:00.955838 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:00.955502 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8de810bd-a229-40bc-b8e5-9d02eeb9a501-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-cqgql\" (UID: \"8de810bd-a229-40bc-b8e5-9d02eeb9a501\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-cqgql" Sep 17 07:22:00.955838 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:00.955536 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/552cd700-3fb9-4411-9eda-3374a359f156-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:22:00.955838 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:22:00.955622 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 07:22:00.955838 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:22:00.955622 2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 07:22:00.955838 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:22:00.955689 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/552cd700-3fb9-4411-9eda-3374a359f156-insights-runtime-extractor-tls podName:552cd700-3fb9-4411-9eda-3374a359f156 nodeName:}" failed. No retries permitted until 2026-09-17 07:22:16.955670191 +0000 UTC m=+144.141403144 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/552cd700-3fb9-4411-9eda-3374a359f156-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lzhlj" (UID: "552cd700-3fb9-4411-9eda-3374a359f156") : secret "insights-runtime-extractor-tls" not found Sep 17 07:22:00.955838 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:22:00.955707 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8de810bd-a229-40bc-b8e5-9d02eeb9a501-networking-console-plugin-cert podName:8de810bd-a229-40bc-b8e5-9d02eeb9a501 nodeName:}" failed. No retries permitted until 2026-09-17 07:22:08.955698082 +0000 UTC m=+136.141431032 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8de810bd-a229-40bc-b8e5-9d02eeb9a501-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-cqgql" (UID: "8de810bd-a229-40bc-b8e5-9d02eeb9a501") : secret "networking-console-plugin-cert" not found Sep 17 07:22:00.956072 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:00.956050 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/552cd700-3fb9-4411-9eda-3374a359f156-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:22:03.072073 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:03.072037 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs\") pod \"network-metrics-daemon-x7484\" (UID: \"69c30c82-b237-4fc3-a96c-61840e73b7a7\") " pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:22:03.072508 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:22:03.072148 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 17 07:22:03.072508 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:22:03.072198 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs podName:69c30c82-b237-4fc3-a96c-61840e73b7a7 nodeName:}" failed. No retries permitted until 2026-09-17 07:24:05.072183945 +0000 UTC m=+252.257916907 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs") pod "network-metrics-daemon-x7484" (UID: "69c30c82-b237-4fc3-a96c-61840e73b7a7") : secret "metrics-daemon-secret" not found Sep 17 07:22:03.273593 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:03.273563 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-registry-tls\") pod \"image-registry-7cc57c98c5-9ftmq\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:22:03.273757 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:22:03.273739 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 07:22:03.273826 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:22:03.273761 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cc57c98c5-9ftmq: secret "image-registry-tls" not found Sep 17 07:22:03.273881 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:22:03.273848 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-registry-tls podName:23825dc0-31ec-4002-8389-addd925bf1d9 nodeName:}" failed. No retries permitted until 2026-09-17 07:22:19.273826019 +0000 UTC m=+146.459558994 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-registry-tls") pod "image-registry-7cc57c98c5-9ftmq" (UID: "23825dc0-31ec-4002-8389-addd925bf1d9") : secret "image-registry-tls" not found Sep 17 07:22:09.016423 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:09.016388 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8de810bd-a229-40bc-b8e5-9d02eeb9a501-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-cqgql\" (UID: \"8de810bd-a229-40bc-b8e5-9d02eeb9a501\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-cqgql" Sep 17 07:22:09.018833 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:09.018810 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8de810bd-a229-40bc-b8e5-9d02eeb9a501-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-cqgql\" (UID: \"8de810bd-a229-40bc-b8e5-9d02eeb9a501\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-cqgql" Sep 17 07:22:09.298934 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:09.298854 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-ll9jf\"" Sep 17 07:22:09.304256 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:09.304240 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-cqgql" Sep 17 07:22:09.603619 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:09.603539 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-cqgql"] Sep 17 07:22:09.605595 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:22:09.605561 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8de810bd_a229_40bc_b8e5_9d02eeb9a501.slice/crio-7ccb6a1f179ab6d3cb3a4f388fdc0b73d982782845e64344157fa2236fff26e2 WatchSource:0}: Error finding container 7ccb6a1f179ab6d3cb3a4f388fdc0b73d982782845e64344157fa2236fff26e2: Status 404 returned error can't find the container with id 7ccb6a1f179ab6d3cb3a4f388fdc0b73d982782845e64344157fa2236fff26e2 Sep 17 07:22:09.917985 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:09.917909 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-cqgql" event={"ID":"8de810bd-a229-40bc-b8e5-9d02eeb9a501","Type":"ContainerStarted","Data":"7ccb6a1f179ab6d3cb3a4f388fdc0b73d982782845e64344157fa2236fff26e2"} Sep 17 07:22:11.924865 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:11.924832 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-cqgql" event={"ID":"8de810bd-a229-40bc-b8e5-9d02eeb9a501","Type":"ContainerStarted","Data":"7ecfbb6d3b46a844f0da7ef40017b02346bdfa4b20d3a99235141827303cd89b"} Sep 17 07:22:11.951921 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:11.951876 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-cqgql" podStartSLOduration=17.688345123 podStartE2EDuration="18.951862609s" totalImagesPullingTime="1.263517486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:21:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:22:11.951195414 +0000 UTC m=+139.136928388" watchObservedRunningTime="2026-09-17 07:22:11.951862609 +0000 UTC m=+139.137595581" Sep 17 07:22:13.449098 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:13.449064 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d07236c4-ae38-4ffc-b1ca-288072181c8d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b4rsw\" (UID: \"d07236c4-ae38-4ffc-b1ca-288072181c8d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b4rsw" Sep 17 07:22:13.451525 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:13.451506 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d07236c4-ae38-4ffc-b1ca-288072181c8d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b4rsw\" (UID: \"d07236c4-ae38-4ffc-b1ca-288072181c8d\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b4rsw" Sep 17 07:22:13.752772 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:13.752744 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2vz69\"" Sep 17 07:22:13.760346 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:13.760316 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b4rsw" Sep 17 07:22:14.039908 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:14.039834 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-b4rsw"] Sep 17 07:22:14.042938 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:22:14.042912 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd07236c4_ae38_4ffc_b1ca_288072181c8d.slice/crio-708e7117d795744fb5b14629752a358783f7c213792b42f133969573095ae81c WatchSource:0}: Error finding container 708e7117d795744fb5b14629752a358783f7c213792b42f133969573095ae81c: Status 404 returned error can't find the container with id 708e7117d795744fb5b14629752a358783f7c213792b42f133969573095ae81c Sep 17 07:22:14.936040 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:14.936004 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b4rsw" event={"ID":"d07236c4-ae38-4ffc-b1ca-288072181c8d","Type":"ContainerStarted","Data":"708e7117d795744fb5b14629752a358783f7c213792b42f133969573095ae81c"} Sep 17 07:22:15.114933 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.114895 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cc57c98c5-9ftmq"] Sep 17 07:22:15.115178 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:22:15.115155 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" podUID="23825dc0-31ec-4002-8389-addd925bf1d9" Sep 17 07:22:15.175973 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.175940 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7685554dd6-cwqw2"] Sep 17 07:22:15.179041 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.179019 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.264242 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.264213 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc-image-registry-private-configuration\") pod \"image-registry-7685554dd6-cwqw2\" (UID: \"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc\") " pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.264372 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.264245 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc-ca-trust-extracted\") pod \"image-registry-7685554dd6-cwqw2\" (UID: \"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc\") " pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.264372 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.264268 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc-trusted-ca\") pod \"image-registry-7685554dd6-cwqw2\" (UID: \"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc\") " pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.264476 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.264368 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc-bound-sa-token\") pod \"image-registry-7685554dd6-cwqw2\" (UID: \"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc\") " pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.264476 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.264398 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc-installation-pull-secrets\") pod \"image-registry-7685554dd6-cwqw2\" (UID: \"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc\") " pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.264476 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.264414 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz2qz\" (UniqueName: \"kubernetes.io/projected/8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc-kube-api-access-tz2qz\") pod \"image-registry-7685554dd6-cwqw2\" (UID: \"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc\") " pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.264476 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.264435 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc-registry-certificates\") pod \"image-registry-7685554dd6-cwqw2\" (UID: \"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc\") " pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.264644 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.264575 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc-registry-tls\") pod \"image-registry-7685554dd6-cwqw2\" (UID: \"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc\") " pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.290734 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.290707 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7685554dd6-cwqw2"] Sep 17 07:22:15.365973 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.365946 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc-image-registry-private-configuration\") pod \"image-registry-7685554dd6-cwqw2\" (UID: \"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc\") " pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.366131 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.365978 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc-ca-trust-extracted\") pod \"image-registry-7685554dd6-cwqw2\" (UID: \"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc\") " pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.366131 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.365999 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc-trusted-ca\") pod \"image-registry-7685554dd6-cwqw2\" (UID: \"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc\") " pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.366131 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.366072 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc-bound-sa-token\") pod \"image-registry-7685554dd6-cwqw2\" (UID: \"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc\") " pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.366131 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.366103 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc-installation-pull-secrets\") pod \"image-registry-7685554dd6-cwqw2\" (UID: \"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc\") " pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.366131 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.366132 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz2qz\" (UniqueName: \"kubernetes.io/projected/8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc-kube-api-access-tz2qz\") pod \"image-registry-7685554dd6-cwqw2\" (UID: \"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc\") " pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.366400 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.366153 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc-registry-certificates\") pod \"image-registry-7685554dd6-cwqw2\" (UID: \"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc\") " pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.366400 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.366343 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc-registry-tls\") pod \"image-registry-7685554dd6-cwqw2\" (UID: \"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc\") " pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.366490 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.366422 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc-ca-trust-extracted\") pod \"image-registry-7685554dd6-cwqw2\" (UID: \"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc\") " pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.367087 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.367044 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc-registry-certificates\") pod \"image-registry-7685554dd6-cwqw2\" (UID: \"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc\") " pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.367434 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.367413 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc-trusted-ca\") pod \"image-registry-7685554dd6-cwqw2\" (UID: \"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc\") " pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.368649 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.368627 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc-image-registry-private-configuration\") pod \"image-registry-7685554dd6-cwqw2\" (UID: \"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc\") " pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.369047 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.369028 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc-registry-tls\") pod \"image-registry-7685554dd6-cwqw2\" (UID: \"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc\") " pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.369156 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.369137 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc-installation-pull-secrets\") pod \"image-registry-7685554dd6-cwqw2\" (UID: \"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc\") " pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.382846 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.382822 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc-bound-sa-token\") pod \"image-registry-7685554dd6-cwqw2\" (UID: \"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc\") " pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.391421 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.391402 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz2qz\" (UniqueName: \"kubernetes.io/projected/8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc-kube-api-access-tz2qz\") pod \"image-registry-7685554dd6-cwqw2\" (UID: \"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc\") " pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.494104 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.494071 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-n9snn\"" Sep 17 07:22:15.500368 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.500342 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:15.939693 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.939665 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:22:15.945955 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.945935 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:22:15.970101 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.970079 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/23825dc0-31ec-4002-8389-addd925bf1d9-registry-certificates\") pod \"23825dc0-31ec-4002-8389-addd925bf1d9\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " Sep 17 07:22:15.970192 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.970140 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/23825dc0-31ec-4002-8389-addd925bf1d9-installation-pull-secrets\") pod \"23825dc0-31ec-4002-8389-addd925bf1d9\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " Sep 17 07:22:15.970192 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.970174 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/23825dc0-31ec-4002-8389-addd925bf1d9-image-registry-private-configuration\") pod \"23825dc0-31ec-4002-8389-addd925bf1d9\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " Sep 17 07:22:15.970277 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.970208 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-bound-sa-token\") pod \"23825dc0-31ec-4002-8389-addd925bf1d9\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " Sep 17 07:22:15.970277 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.970239 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23825dc0-31ec-4002-8389-addd925bf1d9-trusted-ca\") pod \"23825dc0-31ec-4002-8389-addd925bf1d9\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " Sep 17 07:22:15.970277 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.970264 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4jlb\" (UniqueName: \"kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-kube-api-access-t4jlb\") pod \"23825dc0-31ec-4002-8389-addd925bf1d9\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " Sep 17 07:22:15.970437 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.970307 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/23825dc0-31ec-4002-8389-addd925bf1d9-ca-trust-extracted\") pod \"23825dc0-31ec-4002-8389-addd925bf1d9\" (UID: \"23825dc0-31ec-4002-8389-addd925bf1d9\") " Sep 17 07:22:15.970497 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.970468 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23825dc0-31ec-4002-8389-addd925bf1d9-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "23825dc0-31ec-4002-8389-addd925bf1d9" (UID: "23825dc0-31ec-4002-8389-addd925bf1d9"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:22:15.970621 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.970599 2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/23825dc0-31ec-4002-8389-addd925bf1d9-registry-certificates\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:22:15.970758 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.970691 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23825dc0-31ec-4002-8389-addd925bf1d9-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "23825dc0-31ec-4002-8389-addd925bf1d9" (UID: "23825dc0-31ec-4002-8389-addd925bf1d9"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:22:15.970975 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.970837 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23825dc0-31ec-4002-8389-addd925bf1d9-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "23825dc0-31ec-4002-8389-addd925bf1d9" (UID: "23825dc0-31ec-4002-8389-addd925bf1d9"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:22:15.973757 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.973735 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23825dc0-31ec-4002-8389-addd925bf1d9-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "23825dc0-31ec-4002-8389-addd925bf1d9" (UID: "23825dc0-31ec-4002-8389-addd925bf1d9"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:22:15.978176 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.978143 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23825dc0-31ec-4002-8389-addd925bf1d9-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "23825dc0-31ec-4002-8389-addd925bf1d9" (UID: "23825dc0-31ec-4002-8389-addd925bf1d9"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:22:15.978269 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.978204 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-kube-api-access-t4jlb" (OuterVolumeSpecName: "kube-api-access-t4jlb") pod "23825dc0-31ec-4002-8389-addd925bf1d9" (UID: "23825dc0-31ec-4002-8389-addd925bf1d9"). InnerVolumeSpecName "kube-api-access-t4jlb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:22:15.978269 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:15.978221 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "23825dc0-31ec-4002-8389-addd925bf1d9" (UID: "23825dc0-31ec-4002-8389-addd925bf1d9"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:22:16.071398 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:16.071375 2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/23825dc0-31ec-4002-8389-addd925bf1d9-installation-pull-secrets\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:22:16.071398 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:16.071397 2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/23825dc0-31ec-4002-8389-addd925bf1d9-image-registry-private-configuration\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:22:16.071398 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:16.071406 2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-bound-sa-token\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:22:16.071591 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:16.071414 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/23825dc0-31ec-4002-8389-addd925bf1d9-trusted-ca\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:22:16.071591 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:16.071422 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4jlb\" (UniqueName: \"kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-kube-api-access-t4jlb\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:22:16.071591 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:16.071430 2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/23825dc0-31ec-4002-8389-addd925bf1d9-ca-trust-extracted\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:22:16.209376 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:16.209348 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7685554dd6-cwqw2"] Sep 17 07:22:16.211965 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:22:16.211943 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d8627ac_e7f3_41f9_bbc4_1a09bd285ddc.slice/crio-2d10941f690f0b59ac61a860aeb3fbcf513765fd78c85135bbceaf3827c6a9b7 WatchSource:0}: Error finding container 2d10941f690f0b59ac61a860aeb3fbcf513765fd78c85135bbceaf3827c6a9b7: Status 404 returned error can't find the container with id 2d10941f690f0b59ac61a860aeb3fbcf513765fd78c85135bbceaf3827c6a9b7 Sep 17 07:22:16.944591 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:16.944549 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" event={"ID":"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc","Type":"ContainerStarted","Data":"7060d9a082589044e874a4812bcd465de49e3f2c1c54231cf8bc3b5cbe2c9462"} Sep 17 07:22:16.945082 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:16.944622 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" event={"ID":"8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc","Type":"ContainerStarted","Data":"2d10941f690f0b59ac61a860aeb3fbcf513765fd78c85135bbceaf3827c6a9b7"} Sep 17 07:22:16.945082 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:16.944651 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:16.946181 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:16.946156 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b4rsw" event={"ID":"d07236c4-ae38-4ffc-b1ca-288072181c8d","Type":"ContainerStarted","Data":"9ca04ed48324da617658fc0f4425ec3b83c9c81e1cd0cca24ce3a4ebcfd45934"} Sep 17 07:22:16.946277 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:16.946208 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cc57c98c5-9ftmq" Sep 17 07:22:16.974470 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:16.974433 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" podStartSLOduration=1.974420568 podStartE2EDuration="1.974420568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:22:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:22:16.972546703 +0000 UTC m=+144.158279674" watchObservedRunningTime="2026-09-17 07:22:16.974420568 +0000 UTC m=+144.160153539" Sep 17 07:22:16.979382 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:16.979354 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/552cd700-3fb9-4411-9eda-3374a359f156-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:22:16.981937 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:16.981915 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/552cd700-3fb9-4411-9eda-3374a359f156-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzhlj\" (UID: \"552cd700-3fb9-4411-9eda-3374a359f156\") " pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:22:16.994446 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:16.994407 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b4rsw" podStartSLOduration=34.07334073 podStartE2EDuration="35.994394219s" totalImagesPullingTime="1.921053489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:21:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:22:16.993429781 +0000 UTC m=+144.179162753" watchObservedRunningTime="2026-09-17 07:22:16.994394219 +0000 UTC m=+144.180127191" Sep 17 07:22:17.040125 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:17.040101 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cc57c98c5-9ftmq"] Sep 17 07:22:17.070379 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:17.070354 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7cc57c98c5-9ftmq"] Sep 17 07:22:17.180444 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:17.180416 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/23825dc0-31ec-4002-8389-addd925bf1d9-registry-tls\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:22:17.245218 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:17.245191 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ghp9r\"" Sep 17 07:22:17.252295 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:17.252272 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lzhlj" Sep 17 07:22:17.314606 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:17.314579 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23825dc0-31ec-4002-8389-addd925bf1d9" path="/var/lib/kubelet/pods/23825dc0-31ec-4002-8389-addd925bf1d9/volumes" Sep 17 07:22:17.597866 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:22:17.597838 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod552cd700_3fb9_4411_9eda_3374a359f156.slice/crio-fd6760709193aee2fd82406a8d7c388bc2cb4570631f146c1bc6f27dcff13c33 WatchSource:0}: Error finding container fd6760709193aee2fd82406a8d7c388bc2cb4570631f146c1bc6f27dcff13c33: Status 404 returned error can't find the container with id fd6760709193aee2fd82406a8d7c388bc2cb4570631f146c1bc6f27dcff13c33 Sep 17 07:22:17.601880 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:17.601844 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lzhlj"] Sep 17 07:22:17.950398 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:17.950365 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lzhlj" event={"ID":"552cd700-3fb9-4411-9eda-3374a359f156","Type":"ContainerStarted","Data":"1794aea50a1c7dfb045dca7193dccbf58374545fe2b71b5006d035709b5eb8f8"} Sep 17 07:22:17.950738 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:17.950425 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lzhlj" event={"ID":"552cd700-3fb9-4411-9eda-3374a359f156","Type":"ContainerStarted","Data":"fd6760709193aee2fd82406a8d7c388bc2cb4570631f146c1bc6f27dcff13c33"} Sep 17 07:22:18.954770 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:18.954739 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lzhlj" event={"ID":"552cd700-3fb9-4411-9eda-3374a359f156","Type":"ContainerStarted","Data":"79ea4224eb809c58544165991ec1706edd06416201caa652e63760e569aa41aa"} Sep 17 07:22:19.822845 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:19.822807 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-whl5j"] Sep 17 07:22:19.826391 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:19.826370 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-whl5j" Sep 17 07:22:19.829371 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:19.829346 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 17 07:22:19.829509 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:19.829479 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-mww2q\"" Sep 17 07:22:19.830720 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:19.830687 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 17 07:22:19.830822 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:19.830743 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 17 07:22:19.845894 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:19.845871 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-whl5j"] Sep 17 07:22:19.904144 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:19.904118 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gn5j\" (UniqueName: \"kubernetes.io/projected/37cc8e33-52ac-48a9-b2df-58b931c58dc2-kube-api-access-7gn5j\") pod \"prometheus-operator-68bcd7f987-whl5j\" (UID: \"37cc8e33-52ac-48a9-b2df-58b931c58dc2\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-whl5j" Sep 17 07:22:19.904293 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:19.904155 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37cc8e33-52ac-48a9-b2df-58b931c58dc2-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-whl5j\" (UID: \"37cc8e33-52ac-48a9-b2df-58b931c58dc2\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-whl5j" Sep 17 07:22:19.904293 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:19.904195 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/37cc8e33-52ac-48a9-b2df-58b931c58dc2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-whl5j\" (UID: \"37cc8e33-52ac-48a9-b2df-58b931c58dc2\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-whl5j" Sep 17 07:22:19.904428 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:19.904352 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/37cc8e33-52ac-48a9-b2df-58b931c58dc2-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-whl5j\" (UID: \"37cc8e33-52ac-48a9-b2df-58b931c58dc2\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-whl5j" Sep 17 07:22:20.005559 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:20.005528 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/37cc8e33-52ac-48a9-b2df-58b931c58dc2-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-whl5j\" (UID: \"37cc8e33-52ac-48a9-b2df-58b931c58dc2\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-whl5j" Sep 17 07:22:20.005991 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:20.005591 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gn5j\" (UniqueName: \"kubernetes.io/projected/37cc8e33-52ac-48a9-b2df-58b931c58dc2-kube-api-access-7gn5j\") pod \"prometheus-operator-68bcd7f987-whl5j\" (UID: \"37cc8e33-52ac-48a9-b2df-58b931c58dc2\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-whl5j" Sep 17 07:22:20.005991 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:22:20.005775 2563 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Sep 17 07:22:20.005991 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:20.005822 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37cc8e33-52ac-48a9-b2df-58b931c58dc2-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-whl5j\" (UID: \"37cc8e33-52ac-48a9-b2df-58b931c58dc2\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-whl5j" Sep 17 07:22:20.005991 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:22:20.005845 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37cc8e33-52ac-48a9-b2df-58b931c58dc2-prometheus-operator-tls podName:37cc8e33-52ac-48a9-b2df-58b931c58dc2 nodeName:}" failed. No retries permitted until 2026-09-17 07:22:20.505823235 +0000 UTC m=+147.691556198 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/37cc8e33-52ac-48a9-b2df-58b931c58dc2-prometheus-operator-tls") pod "prometheus-operator-68bcd7f987-whl5j" (UID: "37cc8e33-52ac-48a9-b2df-58b931c58dc2") : secret "prometheus-operator-tls" not found Sep 17 07:22:20.005991 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:20.005892 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/37cc8e33-52ac-48a9-b2df-58b931c58dc2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-whl5j\" (UID: \"37cc8e33-52ac-48a9-b2df-58b931c58dc2\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-whl5j" Sep 17 07:22:20.006580 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:20.006555 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/37cc8e33-52ac-48a9-b2df-58b931c58dc2-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-whl5j\" (UID: \"37cc8e33-52ac-48a9-b2df-58b931c58dc2\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-whl5j" Sep 17 07:22:20.008886 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:20.008859 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/37cc8e33-52ac-48a9-b2df-58b931c58dc2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-whl5j\" (UID: \"37cc8e33-52ac-48a9-b2df-58b931c58dc2\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-whl5j" Sep 17 07:22:20.019548 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:20.019522 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gn5j\" (UniqueName: \"kubernetes.io/projected/37cc8e33-52ac-48a9-b2df-58b931c58dc2-kube-api-access-7gn5j\") pod \"prometheus-operator-68bcd7f987-whl5j\" (UID: \"37cc8e33-52ac-48a9-b2df-58b931c58dc2\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-whl5j" Sep 17 07:22:20.509572 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:20.509538 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/37cc8e33-52ac-48a9-b2df-58b931c58dc2-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-whl5j\" (UID: \"37cc8e33-52ac-48a9-b2df-58b931c58dc2\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-whl5j" Sep 17 07:22:20.511994 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:20.511973 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/37cc8e33-52ac-48a9-b2df-58b931c58dc2-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-whl5j\" (UID: \"37cc8e33-52ac-48a9-b2df-58b931c58dc2\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-whl5j" Sep 17 07:22:20.737049 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:20.737022 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-whl5j" Sep 17 07:22:20.963749 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:20.963718 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lzhlj" event={"ID":"552cd700-3fb9-4411-9eda-3374a359f156","Type":"ContainerStarted","Data":"68e1e02d2f7254c42ec39e2ff75f49cb77591831838554ace79a97341c428dd3"} Sep 17 07:22:21.025398 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:21.025359 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lzhlj" podStartSLOduration=33.547866586 podStartE2EDuration="36.025312663s" totalImagesPullingTime="2.477446077s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:21:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:22:21.000423157 +0000 UTC m=+148.186156130" watchObservedRunningTime="2026-09-17 07:22:21.025312663 +0000 UTC m=+148.211045634" Sep 17 07:22:21.026270 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:21.026253 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-whl5j"] Sep 17 07:22:21.029073 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:22:21.029052 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37cc8e33_52ac_48a9_b2df_58b931c58dc2.slice/crio-15a9ff131cd555aff9c89f39e9ca7cba1ad81ccda7e57d198f090e90a6874ce7 WatchSource:0}: Error finding container 15a9ff131cd555aff9c89f39e9ca7cba1ad81ccda7e57d198f090e90a6874ce7: Status 404 returned error can't find the container with id 15a9ff131cd555aff9c89f39e9ca7cba1ad81ccda7e57d198f090e90a6874ce7 Sep 17 07:22:21.969035 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:21.968996 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-whl5j" event={"ID":"37cc8e33-52ac-48a9-b2df-58b931c58dc2","Type":"ContainerStarted","Data":"15a9ff131cd555aff9c89f39e9ca7cba1ad81ccda7e57d198f090e90a6874ce7"} Sep 17 07:22:22.975111 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:22.975075 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-whl5j" event={"ID":"37cc8e33-52ac-48a9-b2df-58b931c58dc2","Type":"ContainerStarted","Data":"9c36f50c2b75d107e4a7c21ca4206cadb368b6f518ac80575352c25b6bf8db01"} Sep 17 07:22:22.975606 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:22.975136 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-whl5j" event={"ID":"37cc8e33-52ac-48a9-b2df-58b931c58dc2","Type":"ContainerStarted","Data":"e7f15eff6059c18970b3ce92d8422b5899e3d9ff862776153913198d02c1fd0f"} Sep 17 07:22:22.995119 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:22.995075 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-whl5j" podStartSLOduration=2.666229717 podStartE2EDuration="3.995060565s" totalImagesPullingTime="1.328830848s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:22:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:22:22.994373297 +0000 UTC m=+150.180106268" watchObservedRunningTime="2026-09-17 07:22:22.995060565 +0000 UTC m=+150.180793536" Sep 17 07:22:25.353469 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.353438 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89"] Sep 17 07:22:25.358114 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.358096 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" Sep 17 07:22:25.360980 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.360960 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 17 07:22:25.361654 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.361622 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-vtc97\"" Sep 17 07:22:25.362811 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.362792 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 17 07:22:25.362913 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.362829 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 17 07:22:25.398546 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.398525 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89"] Sep 17 07:22:25.416412 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.416391 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hcrn9"] Sep 17 07:22:25.420522 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.420508 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.424372 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.424355 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 17 07:22:25.424717 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.424704 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 17 07:22:25.424858 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.424840 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 17 07:22:25.427912 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.427895 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-l75mw\"" Sep 17 07:22:25.445435 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.445414 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nfmm\" (UniqueName: \"kubernetes.io/projected/52adbf5f-ede1-4f89-a64e-3afe3078f954-kube-api-access-9nfmm\") pod \"kube-state-metrics-59fbdd77d-w2d89\" (UID: \"52adbf5f-ede1-4f89-a64e-3afe3078f954\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" Sep 17 07:22:25.445524 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.445449 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/52adbf5f-ede1-4f89-a64e-3afe3078f954-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-w2d89\" (UID: \"52adbf5f-ede1-4f89-a64e-3afe3078f954\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" Sep 17 07:22:25.445524 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.445480 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/52adbf5f-ede1-4f89-a64e-3afe3078f954-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-w2d89\" (UID: \"52adbf5f-ede1-4f89-a64e-3afe3078f954\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" Sep 17 07:22:25.445610 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.445540 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52adbf5f-ede1-4f89-a64e-3afe3078f954-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-w2d89\" (UID: \"52adbf5f-ede1-4f89-a64e-3afe3078f954\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" Sep 17 07:22:25.445610 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.445599 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52adbf5f-ede1-4f89-a64e-3afe3078f954-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-w2d89\" (UID: \"52adbf5f-ede1-4f89-a64e-3afe3078f954\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" Sep 17 07:22:25.445693 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.445623 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/52adbf5f-ede1-4f89-a64e-3afe3078f954-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-w2d89\" (UID: \"52adbf5f-ede1-4f89-a64e-3afe3078f954\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" Sep 17 07:22:25.546751 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.546726 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8465f6f8-337f-44a6-9277-bb24929f2158-node-exporter-textfile\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.546863 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.546767 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nfmm\" (UniqueName: \"kubernetes.io/projected/52adbf5f-ede1-4f89-a64e-3afe3078f954-kube-api-access-9nfmm\") pod \"kube-state-metrics-59fbdd77d-w2d89\" (UID: \"52adbf5f-ede1-4f89-a64e-3afe3078f954\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" Sep 17 07:22:25.546863 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.546790 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/52adbf5f-ede1-4f89-a64e-3afe3078f954-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-w2d89\" (UID: \"52adbf5f-ede1-4f89-a64e-3afe3078f954\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" Sep 17 07:22:25.546863 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.546849 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8465f6f8-337f-44a6-9277-bb24929f2158-metrics-client-ca\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.547018 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.546888 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/52adbf5f-ede1-4f89-a64e-3afe3078f954-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-w2d89\" (UID: \"52adbf5f-ede1-4f89-a64e-3afe3078f954\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" Sep 17 07:22:25.547018 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.546928 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8465f6f8-337f-44a6-9277-bb24929f2158-sys\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.547018 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.546951 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8465f6f8-337f-44a6-9277-bb24929f2158-root\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.547123 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.547026 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hns7\" (UniqueName: \"kubernetes.io/projected/8465f6f8-337f-44a6-9277-bb24929f2158-kube-api-access-9hns7\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.547123 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.547097 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8465f6f8-337f-44a6-9277-bb24929f2158-node-exporter-accelerators-collector-config\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.547187 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.547138 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52adbf5f-ede1-4f89-a64e-3afe3078f954-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-w2d89\" (UID: \"52adbf5f-ede1-4f89-a64e-3afe3078f954\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" Sep 17 07:22:25.547187 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.547164 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8465f6f8-337f-44a6-9277-bb24929f2158-node-exporter-wtmp\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.547285 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.547193 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8465f6f8-337f-44a6-9277-bb24929f2158-node-exporter-tls\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.547285 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.547234 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52adbf5f-ede1-4f89-a64e-3afe3078f954-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-w2d89\" (UID: \"52adbf5f-ede1-4f89-a64e-3afe3078f954\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" Sep 17 07:22:25.547285 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.547265 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8465f6f8-337f-44a6-9277-bb24929f2158-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.547469 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.547298 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/52adbf5f-ede1-4f89-a64e-3afe3078f954-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-w2d89\" (UID: \"52adbf5f-ede1-4f89-a64e-3afe3078f954\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" Sep 17 07:22:25.547529 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.547477 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/52adbf5f-ede1-4f89-a64e-3afe3078f954-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-w2d89\" (UID: \"52adbf5f-ede1-4f89-a64e-3afe3078f954\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" Sep 17 07:22:25.547669 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.547650 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/52adbf5f-ede1-4f89-a64e-3afe3078f954-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-w2d89\" (UID: \"52adbf5f-ede1-4f89-a64e-3afe3078f954\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" Sep 17 07:22:25.547812 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.547793 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52adbf5f-ede1-4f89-a64e-3afe3078f954-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-w2d89\" (UID: \"52adbf5f-ede1-4f89-a64e-3afe3078f954\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" Sep 17 07:22:25.549513 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.549492 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/52adbf5f-ede1-4f89-a64e-3afe3078f954-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-w2d89\" (UID: \"52adbf5f-ede1-4f89-a64e-3afe3078f954\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" Sep 17 07:22:25.549751 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.549734 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52adbf5f-ede1-4f89-a64e-3afe3078f954-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-w2d89\" (UID: \"52adbf5f-ede1-4f89-a64e-3afe3078f954\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" Sep 17 07:22:25.561559 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.561537 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nfmm\" (UniqueName: \"kubernetes.io/projected/52adbf5f-ede1-4f89-a64e-3afe3078f954-kube-api-access-9nfmm\") pod \"kube-state-metrics-59fbdd77d-w2d89\" (UID: \"52adbf5f-ede1-4f89-a64e-3afe3078f954\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" Sep 17 07:22:25.648592 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.648525 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8465f6f8-337f-44a6-9277-bb24929f2158-node-exporter-wtmp\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.648592 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.648562 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8465f6f8-337f-44a6-9277-bb24929f2158-node-exporter-tls\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.648739 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.648594 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8465f6f8-337f-44a6-9277-bb24929f2158-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.648739 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.648630 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8465f6f8-337f-44a6-9277-bb24929f2158-node-exporter-textfile\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.648739 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.648688 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8465f6f8-337f-44a6-9277-bb24929f2158-node-exporter-wtmp\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.648739 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.648694 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8465f6f8-337f-44a6-9277-bb24929f2158-metrics-client-ca\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.648954 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.648761 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8465f6f8-337f-44a6-9277-bb24929f2158-sys\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.648954 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.648786 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8465f6f8-337f-44a6-9277-bb24929f2158-root\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.648954 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.648824 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hns7\" (UniqueName: \"kubernetes.io/projected/8465f6f8-337f-44a6-9277-bb24929f2158-kube-api-access-9hns7\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.648954 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.648857 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8465f6f8-337f-44a6-9277-bb24929f2158-node-exporter-accelerators-collector-config\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.648954 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.648875 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8465f6f8-337f-44a6-9277-bb24929f2158-sys\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.648954 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.648939 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8465f6f8-337f-44a6-9277-bb24929f2158-root\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.649263 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.649223 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8465f6f8-337f-44a6-9277-bb24929f2158-metrics-client-ca\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.649670 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.649609 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8465f6f8-337f-44a6-9277-bb24929f2158-node-exporter-textfile\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.649670 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.649640 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8465f6f8-337f-44a6-9277-bb24929f2158-node-exporter-accelerators-collector-config\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.650901 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.650875 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8465f6f8-337f-44a6-9277-bb24929f2158-node-exporter-tls\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.651039 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.651020 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8465f6f8-337f-44a6-9277-bb24929f2158-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.661564 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.661544 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hns7\" (UniqueName: \"kubernetes.io/projected/8465f6f8-337f-44a6-9277-bb24929f2158-kube-api-access-9hns7\") pod \"node-exporter-hcrn9\" (UID: \"8465f6f8-337f-44a6-9277-bb24929f2158\") " pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.667407 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.667390 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" Sep 17 07:22:25.728986 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.728959 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hcrn9" Sep 17 07:22:25.742042 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:22:25.742009 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8465f6f8_337f_44a6_9277_bb24929f2158.slice/crio-2780a9723d31c146183ba7a372d5b8830ad05f1ed65ce686e45c0e6332112e36 WatchSource:0}: Error finding container 2780a9723d31c146183ba7a372d5b8830ad05f1ed65ce686e45c0e6332112e36: Status 404 returned error can't find the container with id 2780a9723d31c146183ba7a372d5b8830ad05f1ed65ce686e45c0e6332112e36 Sep 17 07:22:25.948686 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.948618 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89"] Sep 17 07:22:25.951643 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:22:25.951612 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52adbf5f_ede1_4f89_a64e_3afe3078f954.slice/crio-bbb227735ef8992adc73661d0696a14108034dd0b97d10217526bcab3d8b89f5 WatchSource:0}: Error finding container bbb227735ef8992adc73661d0696a14108034dd0b97d10217526bcab3d8b89f5: Status 404 returned error can't find the container with id bbb227735ef8992adc73661d0696a14108034dd0b97d10217526bcab3d8b89f5 Sep 17 07:22:25.985525 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.985496 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hcrn9" event={"ID":"8465f6f8-337f-44a6-9277-bb24929f2158","Type":"ContainerStarted","Data":"2780a9723d31c146183ba7a372d5b8830ad05f1ed65ce686e45c0e6332112e36"} Sep 17 07:22:25.989232 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:25.989196 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" event={"ID":"52adbf5f-ede1-4f89-a64e-3afe3078f954","Type":"ContainerStarted","Data":"bbb227735ef8992adc73661d0696a14108034dd0b97d10217526bcab3d8b89f5"} Sep 17 07:22:26.996299 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:26.996259 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hcrn9" event={"ID":"8465f6f8-337f-44a6-9277-bb24929f2158","Type":"ContainerStarted","Data":"201253a1183f943423e3d519a7a0938fb4acbbb2634579ada41c059385c7933d"} Sep 17 07:22:28.001553 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.001517 2563 generic.go:356] "Generic (PLEG): container finished" podID="8465f6f8-337f-44a6-9277-bb24929f2158" containerID="201253a1183f943423e3d519a7a0938fb4acbbb2634579ada41c059385c7933d" exitCode=0 Sep 17 07:22:28.001982 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.001605 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hcrn9" event={"ID":"8465f6f8-337f-44a6-9277-bb24929f2158","Type":"ContainerDied","Data":"201253a1183f943423e3d519a7a0938fb4acbbb2634579ada41c059385c7933d"} Sep 17 07:22:28.003840 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.003817 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" event={"ID":"52adbf5f-ede1-4f89-a64e-3afe3078f954","Type":"ContainerStarted","Data":"904bf17eb62608ad1ebc896a664bc0535bdd25dc612307e271848f7e9b577f0a"} Sep 17 07:22:28.003936 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.003872 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" event={"ID":"52adbf5f-ede1-4f89-a64e-3afe3078f954","Type":"ContainerStarted","Data":"bb5b94e9c864b2254280c75852e42f2d82264ef9a793a720bf3d9078dc9df1b3"} Sep 17 07:22:28.003936 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.003905 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" event={"ID":"52adbf5f-ede1-4f89-a64e-3afe3078f954","Type":"ContainerStarted","Data":"a4cb271b2278c3654f496482aee56943cef5722fcfbde1c1b2f96d3c63f489f2"} Sep 17 07:22:28.053384 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.053297 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-w2d89" podStartSLOduration=1.736319807 podStartE2EDuration="3.053283079s" totalImagesPullingTime="1.316963272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:22:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:22:28.05189544 +0000 UTC m=+155.237628415" watchObservedRunningTime="2026-09-17 07:22:28.053283079 +0000 UTC m=+155.239016064" Sep 17 07:22:28.338491 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.338288 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-d97b757c9-zmgln"] Sep 17 07:22:28.343576 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.343511 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.365777 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.365722 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 17 07:22:28.365777 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.365758 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 17 07:22:28.365989 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.365975 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-ceq0afch1c20u\"" Sep 17 07:22:28.366049 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.366032 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 17 07:22:28.366088 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.366045 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-vr8h9\"" Sep 17 07:22:28.366088 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.366064 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 17 07:22:28.366191 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.366048 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 17 07:22:28.411813 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.411794 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-d97b757c9-zmgln"] Sep 17 07:22:28.474911 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.474880 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/879a10c5-3d92-4906-a0cf-84f104ce2a25-secret-grpc-tls\") pod \"thanos-querier-d97b757c9-zmgln\" (UID: \"879a10c5-3d92-4906-a0cf-84f104ce2a25\") " pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.475058 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.474922 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/879a10c5-3d92-4906-a0cf-84f104ce2a25-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d97b757c9-zmgln\" (UID: \"879a10c5-3d92-4906-a0cf-84f104ce2a25\") " pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.475058 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.474942 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/879a10c5-3d92-4906-a0cf-84f104ce2a25-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d97b757c9-zmgln\" (UID: \"879a10c5-3d92-4906-a0cf-84f104ce2a25\") " pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.475058 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.474965 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4fmp\" (UniqueName: \"kubernetes.io/projected/879a10c5-3d92-4906-a0cf-84f104ce2a25-kube-api-access-x4fmp\") pod \"thanos-querier-d97b757c9-zmgln\" (UID: \"879a10c5-3d92-4906-a0cf-84f104ce2a25\") " pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.475225 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.475055 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/879a10c5-3d92-4906-a0cf-84f104ce2a25-secret-thanos-querier-tls\") pod \"thanos-querier-d97b757c9-zmgln\" (UID: \"879a10c5-3d92-4906-a0cf-84f104ce2a25\") " pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.475225 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.475125 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/879a10c5-3d92-4906-a0cf-84f104ce2a25-metrics-client-ca\") pod \"thanos-querier-d97b757c9-zmgln\" (UID: \"879a10c5-3d92-4906-a0cf-84f104ce2a25\") " pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.475225 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.475180 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/879a10c5-3d92-4906-a0cf-84f104ce2a25-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d97b757c9-zmgln\" (UID: \"879a10c5-3d92-4906-a0cf-84f104ce2a25\") " pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.475225 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.475213 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/879a10c5-3d92-4906-a0cf-84f104ce2a25-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d97b757c9-zmgln\" (UID: \"879a10c5-3d92-4906-a0cf-84f104ce2a25\") " pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.576067 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.576035 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4fmp\" (UniqueName: \"kubernetes.io/projected/879a10c5-3d92-4906-a0cf-84f104ce2a25-kube-api-access-x4fmp\") pod \"thanos-querier-d97b757c9-zmgln\" (UID: \"879a10c5-3d92-4906-a0cf-84f104ce2a25\") " pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.576204 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.576089 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/879a10c5-3d92-4906-a0cf-84f104ce2a25-secret-thanos-querier-tls\") pod \"thanos-querier-d97b757c9-zmgln\" (UID: \"879a10c5-3d92-4906-a0cf-84f104ce2a25\") " pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.576204 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.576147 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/879a10c5-3d92-4906-a0cf-84f104ce2a25-metrics-client-ca\") pod \"thanos-querier-d97b757c9-zmgln\" (UID: \"879a10c5-3d92-4906-a0cf-84f104ce2a25\") " pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.576204 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.576181 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/879a10c5-3d92-4906-a0cf-84f104ce2a25-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d97b757c9-zmgln\" (UID: \"879a10c5-3d92-4906-a0cf-84f104ce2a25\") " pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.576408 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.576210 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/879a10c5-3d92-4906-a0cf-84f104ce2a25-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d97b757c9-zmgln\" (UID: \"879a10c5-3d92-4906-a0cf-84f104ce2a25\") " pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.576408 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.576267 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/879a10c5-3d92-4906-a0cf-84f104ce2a25-secret-grpc-tls\") pod \"thanos-querier-d97b757c9-zmgln\" (UID: \"879a10c5-3d92-4906-a0cf-84f104ce2a25\") " pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.576408 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.576305 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/879a10c5-3d92-4906-a0cf-84f104ce2a25-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d97b757c9-zmgln\" (UID: \"879a10c5-3d92-4906-a0cf-84f104ce2a25\") " pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.576408 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.576356 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/879a10c5-3d92-4906-a0cf-84f104ce2a25-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d97b757c9-zmgln\" (UID: \"879a10c5-3d92-4906-a0cf-84f104ce2a25\") " pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.577080 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.577033 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/879a10c5-3d92-4906-a0cf-84f104ce2a25-metrics-client-ca\") pod \"thanos-querier-d97b757c9-zmgln\" (UID: \"879a10c5-3d92-4906-a0cf-84f104ce2a25\") " pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.579553 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.579521 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/879a10c5-3d92-4906-a0cf-84f104ce2a25-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d97b757c9-zmgln\" (UID: \"879a10c5-3d92-4906-a0cf-84f104ce2a25\") " pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.579721 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.579699 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/879a10c5-3d92-4906-a0cf-84f104ce2a25-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d97b757c9-zmgln\" (UID: \"879a10c5-3d92-4906-a0cf-84f104ce2a25\") " pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.579865 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.579831 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/879a10c5-3d92-4906-a0cf-84f104ce2a25-secret-grpc-tls\") pod \"thanos-querier-d97b757c9-zmgln\" (UID: \"879a10c5-3d92-4906-a0cf-84f104ce2a25\") " pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.580088 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.580068 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/879a10c5-3d92-4906-a0cf-84f104ce2a25-secret-thanos-querier-tls\") pod \"thanos-querier-d97b757c9-zmgln\" (UID: \"879a10c5-3d92-4906-a0cf-84f104ce2a25\") " pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.580211 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.580191 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/879a10c5-3d92-4906-a0cf-84f104ce2a25-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d97b757c9-zmgln\" (UID: \"879a10c5-3d92-4906-a0cf-84f104ce2a25\") " pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.580955 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.580937 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/879a10c5-3d92-4906-a0cf-84f104ce2a25-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d97b757c9-zmgln\" (UID: \"879a10c5-3d92-4906-a0cf-84f104ce2a25\") " pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.587167 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.587146 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4fmp\" (UniqueName: \"kubernetes.io/projected/879a10c5-3d92-4906-a0cf-84f104ce2a25-kube-api-access-x4fmp\") pod \"thanos-querier-d97b757c9-zmgln\" (UID: \"879a10c5-3d92-4906-a0cf-84f104ce2a25\") " pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.652058 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.652007 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:28.940786 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:28.940763 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-d97b757c9-zmgln"] Sep 17 07:22:28.942502 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:22:28.942476 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod879a10c5_3d92_4906_a0cf_84f104ce2a25.slice/crio-913fa751916032bc9c60b38617fc7f3b58f4ce0c14a96fcc94a2f80def7a7c5b WatchSource:0}: Error finding container 913fa751916032bc9c60b38617fc7f3b58f4ce0c14a96fcc94a2f80def7a7c5b: Status 404 returned error can't find the container with id 913fa751916032bc9c60b38617fc7f3b58f4ce0c14a96fcc94a2f80def7a7c5b Sep 17 07:22:29.009548 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.009514 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hcrn9" event={"ID":"8465f6f8-337f-44a6-9277-bb24929f2158","Type":"ContainerStarted","Data":"4a750e3f40f9ef47304c3a06cd97c1b7b07596d91ea3b4962a45bea5de1cd6f3"} Sep 17 07:22:29.009918 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.009584 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hcrn9" event={"ID":"8465f6f8-337f-44a6-9277-bb24929f2158","Type":"ContainerStarted","Data":"b94c8c48d376e3373295d812a6cccd2470dacc5371ce252b98efba5b2b4167b7"} Sep 17 07:22:29.010734 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.010711 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" event={"ID":"879a10c5-3d92-4906-a0cf-84f104ce2a25","Type":"ContainerStarted","Data":"913fa751916032bc9c60b38617fc7f3b58f4ce0c14a96fcc94a2f80def7a7c5b"} Sep 17 07:22:29.036214 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.036171 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hcrn9" podStartSLOduration=2.927781387 podStartE2EDuration="4.036157014s" totalImagesPullingTime="1.029535122s" totalInitContainerRuntime="78.840505ms" isStatefulPod=true podCreationTimestamp="2026-09-17 07:22:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:22:29.034804749 +0000 UTC m=+156.220537720" watchObservedRunningTime="2026-09-17 07:22:29.036157014 +0000 UTC m=+156.221889986" Sep 17 07:22:29.768038 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.768005 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-558649d76f-rfrbj"] Sep 17 07:22:29.772162 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.772136 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:29.776079 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.776039 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 17 07:22:29.776216 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.776096 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 17 07:22:29.776216 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.776162 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 17 07:22:29.777367 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.777348 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-kh688\"" Sep 17 07:22:29.777367 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.777352 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-f11eq6f2n3frk\"" Sep 17 07:22:29.777550 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.777392 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 17 07:22:29.797095 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.797075 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-558649d76f-rfrbj"] Sep 17 07:22:29.888916 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.888880 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a25ca0a8-968b-4ef8-bba5-2a975e3c96cd-client-ca-bundle\") pod \"metrics-server-558649d76f-rfrbj\" (UID: \"a25ca0a8-968b-4ef8-bba5-2a975e3c96cd\") " pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:29.889078 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.888961 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a25ca0a8-968b-4ef8-bba5-2a975e3c96cd-metrics-server-audit-profiles\") pod \"metrics-server-558649d76f-rfrbj\" (UID: \"a25ca0a8-968b-4ef8-bba5-2a975e3c96cd\") " pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:29.889078 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.889045 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lk2r\" (UniqueName: \"kubernetes.io/projected/a25ca0a8-968b-4ef8-bba5-2a975e3c96cd-kube-api-access-8lk2r\") pod \"metrics-server-558649d76f-rfrbj\" (UID: \"a25ca0a8-968b-4ef8-bba5-2a975e3c96cd\") " pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:29.889203 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.889090 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a25ca0a8-968b-4ef8-bba5-2a975e3c96cd-audit-log\") pod \"metrics-server-558649d76f-rfrbj\" (UID: \"a25ca0a8-968b-4ef8-bba5-2a975e3c96cd\") " pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:29.889203 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.889146 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a25ca0a8-968b-4ef8-bba5-2a975e3c96cd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-558649d76f-rfrbj\" (UID: \"a25ca0a8-968b-4ef8-bba5-2a975e3c96cd\") " pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:29.889203 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.889178 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a25ca0a8-968b-4ef8-bba5-2a975e3c96cd-secret-metrics-server-tls\") pod \"metrics-server-558649d76f-rfrbj\" (UID: \"a25ca0a8-968b-4ef8-bba5-2a975e3c96cd\") " pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:29.889365 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.889206 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a25ca0a8-968b-4ef8-bba5-2a975e3c96cd-secret-metrics-server-client-certs\") pod \"metrics-server-558649d76f-rfrbj\" (UID: \"a25ca0a8-968b-4ef8-bba5-2a975e3c96cd\") " pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:29.990342 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.990288 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lk2r\" (UniqueName: \"kubernetes.io/projected/a25ca0a8-968b-4ef8-bba5-2a975e3c96cd-kube-api-access-8lk2r\") pod \"metrics-server-558649d76f-rfrbj\" (UID: \"a25ca0a8-968b-4ef8-bba5-2a975e3c96cd\") " pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:29.990498 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.990351 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a25ca0a8-968b-4ef8-bba5-2a975e3c96cd-audit-log\") pod \"metrics-server-558649d76f-rfrbj\" (UID: \"a25ca0a8-968b-4ef8-bba5-2a975e3c96cd\") " pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:29.990498 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.990401 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a25ca0a8-968b-4ef8-bba5-2a975e3c96cd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-558649d76f-rfrbj\" (UID: \"a25ca0a8-968b-4ef8-bba5-2a975e3c96cd\") " pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:29.990498 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.990422 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a25ca0a8-968b-4ef8-bba5-2a975e3c96cd-secret-metrics-server-tls\") pod \"metrics-server-558649d76f-rfrbj\" (UID: \"a25ca0a8-968b-4ef8-bba5-2a975e3c96cd\") " pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:29.990498 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.990439 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a25ca0a8-968b-4ef8-bba5-2a975e3c96cd-secret-metrics-server-client-certs\") pod \"metrics-server-558649d76f-rfrbj\" (UID: \"a25ca0a8-968b-4ef8-bba5-2a975e3c96cd\") " pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:29.990709 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.990604 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a25ca0a8-968b-4ef8-bba5-2a975e3c96cd-client-ca-bundle\") pod \"metrics-server-558649d76f-rfrbj\" (UID: \"a25ca0a8-968b-4ef8-bba5-2a975e3c96cd\") " pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:29.990709 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.990646 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a25ca0a8-968b-4ef8-bba5-2a975e3c96cd-metrics-server-audit-profiles\") pod \"metrics-server-558649d76f-rfrbj\" (UID: \"a25ca0a8-968b-4ef8-bba5-2a975e3c96cd\") " pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:29.990807 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.990725 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a25ca0a8-968b-4ef8-bba5-2a975e3c96cd-audit-log\") pod \"metrics-server-558649d76f-rfrbj\" (UID: \"a25ca0a8-968b-4ef8-bba5-2a975e3c96cd\") " pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:29.991422 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.991241 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a25ca0a8-968b-4ef8-bba5-2a975e3c96cd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-558649d76f-rfrbj\" (UID: \"a25ca0a8-968b-4ef8-bba5-2a975e3c96cd\") " pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:29.991553 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.991509 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a25ca0a8-968b-4ef8-bba5-2a975e3c96cd-metrics-server-audit-profiles\") pod \"metrics-server-558649d76f-rfrbj\" (UID: \"a25ca0a8-968b-4ef8-bba5-2a975e3c96cd\") " pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:29.993241 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.993220 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a25ca0a8-968b-4ef8-bba5-2a975e3c96cd-secret-metrics-server-client-certs\") pod \"metrics-server-558649d76f-rfrbj\" (UID: \"a25ca0a8-968b-4ef8-bba5-2a975e3c96cd\") " pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:29.993487 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.993463 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a25ca0a8-968b-4ef8-bba5-2a975e3c96cd-client-ca-bundle\") pod \"metrics-server-558649d76f-rfrbj\" (UID: \"a25ca0a8-968b-4ef8-bba5-2a975e3c96cd\") " pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:29.993941 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:29.993901 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a25ca0a8-968b-4ef8-bba5-2a975e3c96cd-secret-metrics-server-tls\") pod \"metrics-server-558649d76f-rfrbj\" (UID: \"a25ca0a8-968b-4ef8-bba5-2a975e3c96cd\") " pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:30.001313 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:30.001293 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lk2r\" (UniqueName: \"kubernetes.io/projected/a25ca0a8-968b-4ef8-bba5-2a975e3c96cd-kube-api-access-8lk2r\") pod \"metrics-server-558649d76f-rfrbj\" (UID: \"a25ca0a8-968b-4ef8-bba5-2a975e3c96cd\") " pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:30.084421 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:30.084145 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:30.203205 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:22:30.203170 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-tr629" podUID="f3305365-b6fc-4a79-97c8-63209d1d71ea" Sep 17 07:22:30.409541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:30.409512 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-558649d76f-rfrbj"] Sep 17 07:22:30.411853 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:22:30.411814 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25ca0a8_968b_4ef8_bba5_2a975e3c96cd.slice/crio-ed4c15d93cbdbc0b64283d22d5cc14eb809bdb4fb60e97de45a6face00ae5cd7 WatchSource:0}: Error finding container ed4c15d93cbdbc0b64283d22d5cc14eb809bdb4fb60e97de45a6face00ae5cd7: Status 404 returned error can't find the container with id ed4c15d93cbdbc0b64283d22d5cc14eb809bdb4fb60e97de45a6face00ae5cd7 Sep 17 07:22:31.020835 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:31.020793 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" event={"ID":"a25ca0a8-968b-4ef8-bba5-2a975e3c96cd","Type":"ContainerStarted","Data":"ed4c15d93cbdbc0b64283d22d5cc14eb809bdb4fb60e97de45a6face00ae5cd7"} Sep 17 07:22:31.020835 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:31.020827 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tr629" Sep 17 07:22:31.321107 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:22:31.321077 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-x7484" podUID="69c30c82-b237-4fc3-a96c-61840e73b7a7" Sep 17 07:22:32.029498 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:32.029416 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" event={"ID":"879a10c5-3d92-4906-a0cf-84f104ce2a25","Type":"ContainerStarted","Data":"6aabbbeea54795276d4d84ab3a50513de1435df55f587896120d6abe46c3b79b"} Sep 17 07:22:32.029677 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:32.029543 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" event={"ID":"879a10c5-3d92-4906-a0cf-84f104ce2a25","Type":"ContainerStarted","Data":"55532a79ae28ca294f94a8411765c4eec6e285dfdb5fb64fc63e863ebd7ce0a0"} Sep 17 07:22:32.029677 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:32.029583 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" event={"ID":"879a10c5-3d92-4906-a0cf-84f104ce2a25","Type":"ContainerStarted","Data":"b27b95f979dad579f8c7f11eaec059280a292f5c68136e86259009dda214ad4c"} Sep 17 07:22:33.036547 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.036510 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" event={"ID":"879a10c5-3d92-4906-a0cf-84f104ce2a25","Type":"ContainerStarted","Data":"6a1df863115198b90ecf5001c0ebd7e09fd2b654f3c7a6ca708e3e518728c202"} Sep 17 07:22:33.036989 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.036574 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" event={"ID":"879a10c5-3d92-4906-a0cf-84f104ce2a25","Type":"ContainerStarted","Data":"3cf3ca0ab8cbf61002c75c13edb857349cb0fcba3cf079e816df6a95bdf27afb"} Sep 17 07:22:33.036989 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.036600 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" event={"ID":"879a10c5-3d92-4906-a0cf-84f104ce2a25","Type":"ContainerStarted","Data":"15647f4d8de98deb0eb6ee800a7e405ee0216d5c6a2fa6ed852857562b3f4285"} Sep 17 07:22:33.036989 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.036683 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:33.038540 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.038509 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" event={"ID":"a25ca0a8-968b-4ef8-bba5-2a975e3c96cd","Type":"ContainerStarted","Data":"206b54a91cfdbd43cb07a2ed90e5f940851c8ef25b55a38620cf95efe665cc95"} Sep 17 07:22:33.066282 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.066239 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" podStartSLOduration=1.678297361 podStartE2EDuration="5.066225268s" totalImagesPullingTime="3.387927907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:22:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:22:33.065125861 +0000 UTC m=+160.250858833" watchObservedRunningTime="2026-09-17 07:22:33.066225268 +0000 UTC m=+160.251958240" Sep 17 07:22:33.088117 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.088071 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" podStartSLOduration=1.989967625 podStartE2EDuration="4.088061589s" totalImagesPullingTime="2.098093964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:22:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:22:33.086475073 +0000 UTC m=+160.272208042" watchObservedRunningTime="2026-09-17 07:22:33.088061589 +0000 UTC m=+160.273794619" Sep 17 07:22:33.459619 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.459593 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f557b8fd4-hxnj8"] Sep 17 07:22:33.462912 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.462891 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:22:33.465997 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.465977 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 17 07:22:33.466251 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.466235 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 17 07:22:33.466342 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.466298 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-fk8mv\"" Sep 17 07:22:33.466401 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.466336 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 17 07:22:33.466401 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.466348 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 17 07:22:33.466401 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.466301 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 17 07:22:33.466882 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.466868 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 17 07:22:33.466958 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.466943 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 17 07:22:33.474758 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.474736 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f557b8fd4-hxnj8"] Sep 17 07:22:33.524430 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.524407 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-console-oauth-config\") pod \"console-5f557b8fd4-hxnj8\" (UID: \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\") " pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:22:33.524518 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.524501 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-service-ca\") pod \"console-5f557b8fd4-hxnj8\" (UID: \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\") " pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:22:33.524555 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.524535 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-oauth-serving-cert\") pod \"console-5f557b8fd4-hxnj8\" (UID: \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\") " pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:22:33.524596 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.524572 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jgcs\" (UniqueName: \"kubernetes.io/projected/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-kube-api-access-8jgcs\") pod \"console-5f557b8fd4-hxnj8\" (UID: \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\") " pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:22:33.524629 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.524612 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-console-config\") pod \"console-5f557b8fd4-hxnj8\" (UID: \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\") " pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:22:33.524665 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.524640 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-console-serving-cert\") pod \"console-5f557b8fd4-hxnj8\" (UID: \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\") " pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:22:33.625036 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.625005 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jgcs\" (UniqueName: \"kubernetes.io/projected/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-kube-api-access-8jgcs\") pod \"console-5f557b8fd4-hxnj8\" (UID: \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\") " pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:22:33.625155 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.625040 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-console-config\") pod \"console-5f557b8fd4-hxnj8\" (UID: \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\") " pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:22:33.625155 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.625063 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-console-serving-cert\") pod \"console-5f557b8fd4-hxnj8\" (UID: \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\") " pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:22:33.625297 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.625274 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-console-oauth-config\") pod \"console-5f557b8fd4-hxnj8\" (UID: \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\") " pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:22:33.625436 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.625419 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-service-ca\") pod \"console-5f557b8fd4-hxnj8\" (UID: \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\") " pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:22:33.625488 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.625454 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-oauth-serving-cert\") pod \"console-5f557b8fd4-hxnj8\" (UID: \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\") " pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:22:33.625838 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.625803 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-console-config\") pod \"console-5f557b8fd4-hxnj8\" (UID: \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\") " pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:22:33.626037 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.626019 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-oauth-serving-cert\") pod \"console-5f557b8fd4-hxnj8\" (UID: \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\") " pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:22:33.626102 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.626031 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-service-ca\") pod \"console-5f557b8fd4-hxnj8\" (UID: \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\") " pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:22:33.627565 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.627545 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-console-serving-cert\") pod \"console-5f557b8fd4-hxnj8\" (UID: \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\") " pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:22:33.627565 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.627553 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-console-oauth-config\") pod \"console-5f557b8fd4-hxnj8\" (UID: \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\") " pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:22:33.635574 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.635556 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jgcs\" (UniqueName: \"kubernetes.io/projected/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-kube-api-access-8jgcs\") pod \"console-5f557b8fd4-hxnj8\" (UID: \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\") " pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:22:33.773035 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:33.772974 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:22:34.063420 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:34.063308 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f557b8fd4-hxnj8"] Sep 17 07:22:34.065963 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:22:34.065931 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode747c79f_7c56_43d6_8d7b_ebe0ae63e616.slice/crio-99d5790299ee409336a4b432a81a0ca8ceb0ba934eb0e4b6ce71b4fc9899f375 WatchSource:0}: Error finding container 99d5790299ee409336a4b432a81a0ca8ceb0ba934eb0e4b6ce71b4fc9899f375: Status 404 returned error can't find the container with id 99d5790299ee409336a4b432a81a0ca8ceb0ba934eb0e4b6ce71b4fc9899f375 Sep 17 07:22:35.038834 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:35.038796 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls\") pod \"dns-default-tr629\" (UID: \"f3305365-b6fc-4a79-97c8-63209d1d71ea\") " pod="openshift-dns/dns-default-tr629" Sep 17 07:22:35.041268 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:35.041248 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3305365-b6fc-4a79-97c8-63209d1d71ea-metrics-tls\") pod \"dns-default-tr629\" (UID: \"f3305365-b6fc-4a79-97c8-63209d1d71ea\") " pod="openshift-dns/dns-default-tr629" Sep 17 07:22:35.048753 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:35.048723 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f557b8fd4-hxnj8" event={"ID":"e747c79f-7c56-43d6-8d7b-ebe0ae63e616","Type":"ContainerStarted","Data":"99d5790299ee409336a4b432a81a0ca8ceb0ba934eb0e4b6ce71b4fc9899f375"} Sep 17 07:22:35.225068 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:35.225046 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tth88\"" Sep 17 07:22:35.231949 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:35.231922 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tr629" Sep 17 07:22:35.506030 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:35.505996 2563 patch_prober.go:28] interesting pod/image-registry-7685554dd6-cwqw2 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 17 07:22:35.506171 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:35.506053 2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" podUID="8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 07:22:35.524809 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:35.524783 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tr629"] Sep 17 07:22:35.527231 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:22:35.527205 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3305365_b6fc_4a79_97c8_63209d1d71ea.slice/crio-fdd057a75176053e10541b9b09fd0e9cd9e51ebdd820f03d84bb7ff0f6fa36a5 WatchSource:0}: Error finding container fdd057a75176053e10541b9b09fd0e9cd9e51ebdd820f03d84bb7ff0f6fa36a5: Status 404 returned error can't find the container with id fdd057a75176053e10541b9b09fd0e9cd9e51ebdd820f03d84bb7ff0f6fa36a5 Sep 17 07:22:36.054920 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:36.054879 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tr629" event={"ID":"f3305365-b6fc-4a79-97c8-63209d1d71ea","Type":"ContainerStarted","Data":"fdd057a75176053e10541b9b09fd0e9cd9e51ebdd820f03d84bb7ff0f6fa36a5"} Sep 17 07:22:37.954967 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:37.954934 2563 patch_prober.go:28] interesting pod/image-registry-7685554dd6-cwqw2 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 17 07:22:37.955427 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:37.954992 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" podUID="8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 07:22:39.048630 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:39.048598 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-d97b757c9-zmgln" Sep 17 07:22:39.068881 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:39.068848 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f557b8fd4-hxnj8" event={"ID":"e747c79f-7c56-43d6-8d7b-ebe0ae63e616","Type":"ContainerStarted","Data":"19b223306c8088cf9a75ad773aed124b15314cf72beccf643554a37b62c1ea9a"} Sep 17 07:22:39.070698 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:39.070676 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tr629" event={"ID":"f3305365-b6fc-4a79-97c8-63209d1d71ea","Type":"ContainerStarted","Data":"a11297b384c059ad51357bb716e7d7adc784977abd8dcc5293f86fe9a0c9337b"} Sep 17 07:22:39.070828 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:39.070722 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tr629" event={"ID":"f3305365-b6fc-4a79-97c8-63209d1d71ea","Type":"ContainerStarted","Data":"62253700e3307a366e10e21fc80cfe507a86a81afc7bff26ac52d30c59cb5b30"} Sep 17 07:22:39.070828 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:39.070751 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tr629" Sep 17 07:22:39.128605 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:39.128554 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f557b8fd4-hxnj8" podStartSLOduration=1.70931381 podStartE2EDuration="6.128534358s" totalImagesPullingTime="4.419220548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:22:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:22:39.125265269 +0000 UTC m=+166.310998242" watchObservedRunningTime="2026-09-17 07:22:39.128534358 +0000 UTC m=+166.314267331" Sep 17 07:22:39.153562 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:39.153520 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tr629" podStartSLOduration=129.195431732 podStartE2EDuration="2m12.153506771s" totalImagesPullingTime="2.958075039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:22:39.152493497 +0000 UTC m=+166.338226469" watchObservedRunningTime="2026-09-17 07:22:39.153506771 +0000 UTC m=+166.339239743" Sep 17 07:22:42.308883 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.308845 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:22:42.451996 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.451970 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8667bf884f-ctv8f"] Sep 17 07:22:42.455379 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.455362 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:42.464072 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.464054 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 17 07:22:42.469455 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.469436 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8667bf884f-ctv8f"] Sep 17 07:22:42.601737 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.601680 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/beaeb8ae-ae96-484e-856e-3d02bcd89eea-console-serving-cert\") pod \"console-8667bf884f-ctv8f\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:42.601737 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.601711 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/beaeb8ae-ae96-484e-856e-3d02bcd89eea-console-config\") pod \"console-8667bf884f-ctv8f\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:42.601737 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.601734 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/beaeb8ae-ae96-484e-856e-3d02bcd89eea-oauth-serving-cert\") pod \"console-8667bf884f-ctv8f\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:42.601894 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.601756 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhxnp\" (UniqueName: \"kubernetes.io/projected/beaeb8ae-ae96-484e-856e-3d02bcd89eea-kube-api-access-nhxnp\") pod \"console-8667bf884f-ctv8f\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:42.601894 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.601866 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/beaeb8ae-ae96-484e-856e-3d02bcd89eea-service-ca\") pod \"console-8667bf884f-ctv8f\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:42.601951 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.601894 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beaeb8ae-ae96-484e-856e-3d02bcd89eea-trusted-ca-bundle\") pod \"console-8667bf884f-ctv8f\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:42.601951 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.601915 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/beaeb8ae-ae96-484e-856e-3d02bcd89eea-console-oauth-config\") pod \"console-8667bf884f-ctv8f\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:42.702617 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.702591 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/beaeb8ae-ae96-484e-856e-3d02bcd89eea-oauth-serving-cert\") pod \"console-8667bf884f-ctv8f\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:42.702697 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.702621 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhxnp\" (UniqueName: \"kubernetes.io/projected/beaeb8ae-ae96-484e-856e-3d02bcd89eea-kube-api-access-nhxnp\") pod \"console-8667bf884f-ctv8f\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:42.702697 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.702671 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/beaeb8ae-ae96-484e-856e-3d02bcd89eea-service-ca\") pod \"console-8667bf884f-ctv8f\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:42.702697 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.702688 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beaeb8ae-ae96-484e-856e-3d02bcd89eea-trusted-ca-bundle\") pod \"console-8667bf884f-ctv8f\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:42.702841 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.702705 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/beaeb8ae-ae96-484e-856e-3d02bcd89eea-console-oauth-config\") pod \"console-8667bf884f-ctv8f\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:42.702841 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.702729 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/beaeb8ae-ae96-484e-856e-3d02bcd89eea-console-serving-cert\") pod \"console-8667bf884f-ctv8f\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:42.702841 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.702744 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/beaeb8ae-ae96-484e-856e-3d02bcd89eea-console-config\") pod \"console-8667bf884f-ctv8f\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:42.703541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.703476 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/beaeb8ae-ae96-484e-856e-3d02bcd89eea-service-ca\") pod \"console-8667bf884f-ctv8f\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:42.703541 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.703481 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/beaeb8ae-ae96-484e-856e-3d02bcd89eea-console-config\") pod \"console-8667bf884f-ctv8f\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:42.703707 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.703572 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beaeb8ae-ae96-484e-856e-3d02bcd89eea-trusted-ca-bundle\") pod \"console-8667bf884f-ctv8f\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:42.703707 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.703684 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/beaeb8ae-ae96-484e-856e-3d02bcd89eea-oauth-serving-cert\") pod \"console-8667bf884f-ctv8f\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:42.705385 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.705361 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/beaeb8ae-ae96-484e-856e-3d02bcd89eea-console-serving-cert\") pod \"console-8667bf884f-ctv8f\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:42.705451 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.705423 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/beaeb8ae-ae96-484e-856e-3d02bcd89eea-console-oauth-config\") pod \"console-8667bf884f-ctv8f\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:42.715303 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.715284 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhxnp\" (UniqueName: \"kubernetes.io/projected/beaeb8ae-ae96-484e-856e-3d02bcd89eea-kube-api-access-nhxnp\") pod \"console-8667bf884f-ctv8f\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:42.763905 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:42.763888 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:43.041181 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:43.041037 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8667bf884f-ctv8f"] Sep 17 07:22:43.043672 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:22:43.043646 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbeaeb8ae_ae96_484e_856e_3d02bcd89eea.slice/crio-f1b561fd13bfe02babd7b81fa04afaf506cfa02f18cba6abcbb3cde47f62fb3d WatchSource:0}: Error finding container f1b561fd13bfe02babd7b81fa04afaf506cfa02f18cba6abcbb3cde47f62fb3d: Status 404 returned error can't find the container with id f1b561fd13bfe02babd7b81fa04afaf506cfa02f18cba6abcbb3cde47f62fb3d Sep 17 07:22:43.087558 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:43.087533 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8667bf884f-ctv8f" event={"ID":"beaeb8ae-ae96-484e-856e-3d02bcd89eea","Type":"ContainerStarted","Data":"f1b561fd13bfe02babd7b81fa04afaf506cfa02f18cba6abcbb3cde47f62fb3d"} Sep 17 07:22:43.774145 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:43.774117 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:22:43.774570 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:43.774177 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:22:43.778977 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:43.778953 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:22:44.092881 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:44.092793 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8667bf884f-ctv8f" event={"ID":"beaeb8ae-ae96-484e-856e-3d02bcd89eea","Type":"ContainerStarted","Data":"2f39788206f21e7ef0bd30e029c130bb4585abc9811bba668f08d1c7789b821e"} Sep 17 07:22:44.096665 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:44.096634 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:22:44.118339 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:44.118283 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8667bf884f-ctv8f" podStartSLOduration=2.118267294 podStartE2EDuration="2.118267294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:22:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:22:44.116576132 +0000 UTC m=+171.302309108" watchObservedRunningTime="2026-09-17 07:22:44.118267294 +0000 UTC m=+171.304000266" Sep 17 07:22:45.505333 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:45.505283 2563 patch_prober.go:28] interesting pod/image-registry-7685554dd6-cwqw2 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 17 07:22:45.505705 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:45.505361 2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" podUID="8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 07:22:47.954850 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:47.954825 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7685554dd6-cwqw2" Sep 17 07:22:49.077088 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:49.077061 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tr629" Sep 17 07:22:50.084758 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:50.084724 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:50.085180 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:50.084806 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:22:52.764258 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:52.764220 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:52.764655 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:52.764284 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:52.769171 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:52.769148 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:53.134492 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:53.134408 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:22:53.228368 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:22:53.228341 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f557b8fd4-hxnj8"] Sep 17 07:23:05.178490 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:05.178457 2563 generic.go:356] "Generic (PLEG): container finished" podID="b4e5f925-07d2-41f5-b94b-f275630921ed" containerID="f555c742315b8665f30ee7ea17102b0d8b7dc6574e72c5ff635dfd1c4ba08de9" exitCode=0 Sep 17 07:23:05.178817 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:05.178521 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gxzj" event={"ID":"b4e5f925-07d2-41f5-b94b-f275630921ed","Type":"ContainerDied","Data":"f555c742315b8665f30ee7ea17102b0d8b7dc6574e72c5ff635dfd1c4ba08de9"} Sep 17 07:23:05.178817 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:05.178805 2563 scope.go:117] "RemoveContainer" containerID="f555c742315b8665f30ee7ea17102b0d8b7dc6574e72c5ff635dfd1c4ba08de9" Sep 17 07:23:06.184476 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:06.184437 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-4gxzj" event={"ID":"b4e5f925-07d2-41f5-b94b-f275630921ed","Type":"ContainerStarted","Data":"0db5194ee8e0cfee77f844c58a2c4e539c560e3bcbee2e4f0e751e18ae6fa0f1"} Sep 17 07:23:10.089951 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:10.089921 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:23:10.093664 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:10.093634 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-558649d76f-rfrbj" Sep 17 07:23:15.220341 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:15.220291 2563 generic.go:356] "Generic (PLEG): container finished" podID="59c7192a-8c88-4409-ba0a-811cccbc2112" containerID="6577e0426941d17790ba2170603f28ab717183eebecff7add3a1b57add291ec1" exitCode=0 Sep 17 07:23:15.220680 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:15.220365 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-fwkfn" event={"ID":"59c7192a-8c88-4409-ba0a-811cccbc2112","Type":"ContainerDied","Data":"6577e0426941d17790ba2170603f28ab717183eebecff7add3a1b57add291ec1"} Sep 17 07:23:15.220733 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:15.220700 2563 scope.go:117] "RemoveContainer" containerID="6577e0426941d17790ba2170603f28ab717183eebecff7add3a1b57add291ec1" Sep 17 07:23:16.225940 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:16.225904 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-fwkfn" event={"ID":"59c7192a-8c88-4409-ba0a-811cccbc2112","Type":"ContainerStarted","Data":"dbc81d58bbee8be03e930ec64277242be510d393d0213650db306949e0f3201a"} Sep 17 07:23:18.248803 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:18.248754 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f557b8fd4-hxnj8" podUID="e747c79f-7c56-43d6-8d7b-ebe0ae63e616" containerName="console" containerID="cri-o://19b223306c8088cf9a75ad773aed124b15314cf72beccf643554a37b62c1ea9a" gracePeriod=15 Sep 17 07:23:18.490557 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:18.490531 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:23:18.592930 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:18.592864 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jgcs\" (UniqueName: \"kubernetes.io/projected/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-kube-api-access-8jgcs\") pod \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\" (UID: \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\") " Sep 17 07:23:18.592930 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:18.592893 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-console-serving-cert\") pod \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\" (UID: \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\") " Sep 17 07:23:18.592930 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:18.592924 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-oauth-serving-cert\") pod \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\" (UID: \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\") " Sep 17 07:23:18.593168 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:18.592949 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-console-oauth-config\") pod \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\" (UID: \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\") " Sep 17 07:23:18.593168 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:18.592976 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-service-ca\") pod \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\" (UID: \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\") " Sep 17 07:23:18.593168 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:18.593028 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-console-config\") pod \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\" (UID: \"e747c79f-7c56-43d6-8d7b-ebe0ae63e616\") " Sep 17 07:23:18.593387 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:18.593357 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e747c79f-7c56-43d6-8d7b-ebe0ae63e616" (UID: "e747c79f-7c56-43d6-8d7b-ebe0ae63e616"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:23:18.593496 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:18.593465 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-service-ca" (OuterVolumeSpecName: "service-ca") pod "e747c79f-7c56-43d6-8d7b-ebe0ae63e616" (UID: "e747c79f-7c56-43d6-8d7b-ebe0ae63e616"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:23:18.593582 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:18.593528 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-console-config" (OuterVolumeSpecName: "console-config") pod "e747c79f-7c56-43d6-8d7b-ebe0ae63e616" (UID: "e747c79f-7c56-43d6-8d7b-ebe0ae63e616"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:23:18.595235 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:18.595205 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e747c79f-7c56-43d6-8d7b-ebe0ae63e616" (UID: "e747c79f-7c56-43d6-8d7b-ebe0ae63e616"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:23:18.595353 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:18.595278 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e747c79f-7c56-43d6-8d7b-ebe0ae63e616" (UID: "e747c79f-7c56-43d6-8d7b-ebe0ae63e616"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:23:18.595353 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:18.595339 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-kube-api-access-8jgcs" (OuterVolumeSpecName: "kube-api-access-8jgcs") pod "e747c79f-7c56-43d6-8d7b-ebe0ae63e616" (UID: "e747c79f-7c56-43d6-8d7b-ebe0ae63e616"). InnerVolumeSpecName "kube-api-access-8jgcs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:23:18.694475 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:18.694446 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-console-config\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:23:18.694475 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:18.694474 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jgcs\" (UniqueName: \"kubernetes.io/projected/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-kube-api-access-8jgcs\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:23:18.694607 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:18.694488 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-console-serving-cert\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:23:18.694607 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:18.694500 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-oauth-serving-cert\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:23:18.694607 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:18.694512 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-console-oauth-config\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:23:18.694607 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:18.694524 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e747c79f-7c56-43d6-8d7b-ebe0ae63e616-service-ca\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:23:19.242030 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:19.241993 2563 generic.go:356] "Generic (PLEG): container finished" podID="e747c79f-7c56-43d6-8d7b-ebe0ae63e616" containerID="19b223306c8088cf9a75ad773aed124b15314cf72beccf643554a37b62c1ea9a" exitCode=0 Sep 17 07:23:19.242241 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:19.242050 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f557b8fd4-hxnj8" Sep 17 07:23:19.242241 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:19.242087 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f557b8fd4-hxnj8" event={"ID":"e747c79f-7c56-43d6-8d7b-ebe0ae63e616","Type":"ContainerDied","Data":"19b223306c8088cf9a75ad773aed124b15314cf72beccf643554a37b62c1ea9a"} Sep 17 07:23:19.242241 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:19.242147 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f557b8fd4-hxnj8" event={"ID":"e747c79f-7c56-43d6-8d7b-ebe0ae63e616","Type":"ContainerDied","Data":"99d5790299ee409336a4b432a81a0ca8ceb0ba934eb0e4b6ce71b4fc9899f375"} Sep 17 07:23:19.242241 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:19.242169 2563 scope.go:117] "RemoveContainer" containerID="19b223306c8088cf9a75ad773aed124b15314cf72beccf643554a37b62c1ea9a" Sep 17 07:23:19.251720 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:19.251642 2563 scope.go:117] "RemoveContainer" containerID="19b223306c8088cf9a75ad773aed124b15314cf72beccf643554a37b62c1ea9a" Sep 17 07:23:19.251961 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:23:19.251941 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19b223306c8088cf9a75ad773aed124b15314cf72beccf643554a37b62c1ea9a\": container with ID starting with 19b223306c8088cf9a75ad773aed124b15314cf72beccf643554a37b62c1ea9a not found: ID does not exist" containerID="19b223306c8088cf9a75ad773aed124b15314cf72beccf643554a37b62c1ea9a" Sep 17 07:23:19.252007 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:19.251970 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19b223306c8088cf9a75ad773aed124b15314cf72beccf643554a37b62c1ea9a"} err="failed to get container status \"19b223306c8088cf9a75ad773aed124b15314cf72beccf643554a37b62c1ea9a\": rpc error: code = NotFound desc = could not find container \"19b223306c8088cf9a75ad773aed124b15314cf72beccf643554a37b62c1ea9a\": container with ID starting with 19b223306c8088cf9a75ad773aed124b15314cf72beccf643554a37b62c1ea9a not found: ID does not exist" Sep 17 07:23:19.265373 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:19.265349 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f557b8fd4-hxnj8"] Sep 17 07:23:19.269441 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:19.269420 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f557b8fd4-hxnj8"] Sep 17 07:23:19.313834 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:19.313801 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e747c79f-7c56-43d6-8d7b-ebe0ae63e616" path="/var/lib/kubelet/pods/e747c79f-7c56-43d6-8d7b-ebe0ae63e616/volumes" Sep 17 07:23:50.056131 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.056092 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-766fc79c5-qqd9l"] Sep 17 07:23:50.057164 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.057135 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e747c79f-7c56-43d6-8d7b-ebe0ae63e616" containerName="console" Sep 17 07:23:50.057164 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.057162 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e747c79f-7c56-43d6-8d7b-ebe0ae63e616" containerName="console" Sep 17 07:23:50.057313 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.057266 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e747c79f-7c56-43d6-8d7b-ebe0ae63e616" containerName="console" Sep 17 07:23:50.060584 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.060568 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.064629 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.064163 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 17 07:23:50.064629 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.064501 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 17 07:23:50.064830 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.064808 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 17 07:23:50.064830 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.064826 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 17 07:23:50.064950 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.064835 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 17 07:23:50.064950 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.064890 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-97f7g\"" Sep 17 07:23:50.070979 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.070958 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 17 07:23:50.094442 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.094422 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-766fc79c5-qqd9l"] Sep 17 07:23:50.242192 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.242157 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08d633b1-b5be-41fc-a04d-ccaaa3e8fad6-metrics-client-ca\") pod \"telemeter-client-766fc79c5-qqd9l\" (UID: \"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6\") " pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.242395 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.242199 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2fwg\" (UniqueName: \"kubernetes.io/projected/08d633b1-b5be-41fc-a04d-ccaaa3e8fad6-kube-api-access-v2fwg\") pod \"telemeter-client-766fc79c5-qqd9l\" (UID: \"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6\") " pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.242395 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.242229 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08d633b1-b5be-41fc-a04d-ccaaa3e8fad6-serving-certs-ca-bundle\") pod \"telemeter-client-766fc79c5-qqd9l\" (UID: \"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6\") " pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.242395 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.242284 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/08d633b1-b5be-41fc-a04d-ccaaa3e8fad6-telemeter-client-tls\") pod \"telemeter-client-766fc79c5-qqd9l\" (UID: \"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6\") " pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.242561 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.242405 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/08d633b1-b5be-41fc-a04d-ccaaa3e8fad6-secret-telemeter-client\") pod \"telemeter-client-766fc79c5-qqd9l\" (UID: \"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6\") " pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.242561 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.242450 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/08d633b1-b5be-41fc-a04d-ccaaa3e8fad6-federate-client-tls\") pod \"telemeter-client-766fc79c5-qqd9l\" (UID: \"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6\") " pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.242561 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.242474 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08d633b1-b5be-41fc-a04d-ccaaa3e8fad6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-766fc79c5-qqd9l\" (UID: \"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6\") " pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.242561 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.242523 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/08d633b1-b5be-41fc-a04d-ccaaa3e8fad6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-766fc79c5-qqd9l\" (UID: \"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6\") " pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.343538 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.343454 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08d633b1-b5be-41fc-a04d-ccaaa3e8fad6-serving-certs-ca-bundle\") pod \"telemeter-client-766fc79c5-qqd9l\" (UID: \"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6\") " pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.343538 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.343498 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/08d633b1-b5be-41fc-a04d-ccaaa3e8fad6-telemeter-client-tls\") pod \"telemeter-client-766fc79c5-qqd9l\" (UID: \"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6\") " pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.343812 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.343548 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/08d633b1-b5be-41fc-a04d-ccaaa3e8fad6-secret-telemeter-client\") pod \"telemeter-client-766fc79c5-qqd9l\" (UID: \"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6\") " pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.343812 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.343576 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/08d633b1-b5be-41fc-a04d-ccaaa3e8fad6-federate-client-tls\") pod \"telemeter-client-766fc79c5-qqd9l\" (UID: \"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6\") " pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.343812 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.343590 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08d633b1-b5be-41fc-a04d-ccaaa3e8fad6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-766fc79c5-qqd9l\" (UID: \"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6\") " pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.343812 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.343609 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/08d633b1-b5be-41fc-a04d-ccaaa3e8fad6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-766fc79c5-qqd9l\" (UID: \"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6\") " pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.343812 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.343657 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08d633b1-b5be-41fc-a04d-ccaaa3e8fad6-metrics-client-ca\") pod \"telemeter-client-766fc79c5-qqd9l\" (UID: \"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6\") " pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.343812 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.343683 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2fwg\" (UniqueName: \"kubernetes.io/projected/08d633b1-b5be-41fc-a04d-ccaaa3e8fad6-kube-api-access-v2fwg\") pod \"telemeter-client-766fc79c5-qqd9l\" (UID: \"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6\") " pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.344279 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.344252 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08d633b1-b5be-41fc-a04d-ccaaa3e8fad6-serving-certs-ca-bundle\") pod \"telemeter-client-766fc79c5-qqd9l\" (UID: \"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6\") " pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.344519 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.344494 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08d633b1-b5be-41fc-a04d-ccaaa3e8fad6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-766fc79c5-qqd9l\" (UID: \"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6\") " pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.344634 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.344569 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08d633b1-b5be-41fc-a04d-ccaaa3e8fad6-metrics-client-ca\") pod \"telemeter-client-766fc79c5-qqd9l\" (UID: \"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6\") " pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.346727 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.346707 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/08d633b1-b5be-41fc-a04d-ccaaa3e8fad6-federate-client-tls\") pod \"telemeter-client-766fc79c5-qqd9l\" (UID: \"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6\") " pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.346812 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.346727 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/08d633b1-b5be-41fc-a04d-ccaaa3e8fad6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-766fc79c5-qqd9l\" (UID: \"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6\") " pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.346937 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.346918 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/08d633b1-b5be-41fc-a04d-ccaaa3e8fad6-secret-telemeter-client\") pod \"telemeter-client-766fc79c5-qqd9l\" (UID: \"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6\") " pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.346972 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.346918 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/08d633b1-b5be-41fc-a04d-ccaaa3e8fad6-telemeter-client-tls\") pod \"telemeter-client-766fc79c5-qqd9l\" (UID: \"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6\") " pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.358993 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.358965 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2fwg\" (UniqueName: \"kubernetes.io/projected/08d633b1-b5be-41fc-a04d-ccaaa3e8fad6-kube-api-access-v2fwg\") pod \"telemeter-client-766fc79c5-qqd9l\" (UID: \"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6\") " pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.371247 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.371227 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" Sep 17 07:23:50.664727 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:50.664648 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-766fc79c5-qqd9l"] Sep 17 07:23:50.666658 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:23:50.666628 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08d633b1_b5be_41fc_a04d_ccaaa3e8fad6.slice/crio-d5012a1e28fe1b83ecfc18311efa13e4b116a44ce37ab1191b2830df6704bfe8 WatchSource:0}: Error finding container d5012a1e28fe1b83ecfc18311efa13e4b116a44ce37ab1191b2830df6704bfe8: Status 404 returned error can't find the container with id d5012a1e28fe1b83ecfc18311efa13e4b116a44ce37ab1191b2830df6704bfe8 Sep 17 07:23:51.375966 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:51.375928 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" event={"ID":"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6","Type":"ContainerStarted","Data":"d5012a1e28fe1b83ecfc18311efa13e4b116a44ce37ab1191b2830df6704bfe8"} Sep 17 07:23:53.385756 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:53.385721 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" event={"ID":"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6","Type":"ContainerStarted","Data":"6b32993e77e223949e525c5aafdf5b17534fe6d050ff791a201923509013a931"} Sep 17 07:23:54.391555 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.391523 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" event={"ID":"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6","Type":"ContainerStarted","Data":"e56e29420d4fd2bdc1728ad7325aee3dcbda16d663d40346046ff4995d25400f"} Sep 17 07:23:54.391911 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.391586 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" event={"ID":"08d633b1-b5be-41fc-a04d-ccaaa3e8fad6","Type":"ContainerStarted","Data":"442e44bfd2440131d72d999b3f893862411f04c6eeace7bcc8cb7c7efc9dda29"} Sep 17 07:23:54.417272 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.417227 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-766fc79c5-qqd9l" podStartSLOduration=0.888722911 podStartE2EDuration="4.417212402s" totalImagesPullingTime="3.528489491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:23:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:23:54.415673763 +0000 UTC m=+241.601406734" watchObservedRunningTime="2026-09-17 07:23:54.417212402 +0000 UTC m=+241.602945412" Sep 17 07:23:54.509995 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.509938 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-74f6fdf9f9-nrsfz"] Sep 17 07:23:54.513065 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.513050 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:23:54.524401 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.524379 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74f6fdf9f9-nrsfz"] Sep 17 07:23:54.581290 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.581268 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb3eb372-ac0c-40e2-b940-1f00e251f56b-trusted-ca-bundle\") pod \"console-74f6fdf9f9-nrsfz\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:23:54.581404 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.581299 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb3eb372-ac0c-40e2-b940-1f00e251f56b-service-ca\") pod \"console-74f6fdf9f9-nrsfz\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:23:54.581404 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.581317 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb3eb372-ac0c-40e2-b940-1f00e251f56b-oauth-serving-cert\") pod \"console-74f6fdf9f9-nrsfz\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:23:54.581404 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.581380 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb3eb372-ac0c-40e2-b940-1f00e251f56b-console-config\") pod \"console-74f6fdf9f9-nrsfz\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:23:54.581515 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.581427 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb3eb372-ac0c-40e2-b940-1f00e251f56b-console-oauth-config\") pod \"console-74f6fdf9f9-nrsfz\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:23:54.581515 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.581471 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzrqp\" (UniqueName: \"kubernetes.io/projected/cb3eb372-ac0c-40e2-b940-1f00e251f56b-kube-api-access-hzrqp\") pod \"console-74f6fdf9f9-nrsfz\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:23:54.581515 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.581487 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb3eb372-ac0c-40e2-b940-1f00e251f56b-console-serving-cert\") pod \"console-74f6fdf9f9-nrsfz\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:23:54.686356 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.682928 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb3eb372-ac0c-40e2-b940-1f00e251f56b-console-oauth-config\") pod \"console-74f6fdf9f9-nrsfz\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:23:54.686356 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.683015 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzrqp\" (UniqueName: \"kubernetes.io/projected/cb3eb372-ac0c-40e2-b940-1f00e251f56b-kube-api-access-hzrqp\") pod \"console-74f6fdf9f9-nrsfz\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:23:54.686356 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.683051 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb3eb372-ac0c-40e2-b940-1f00e251f56b-console-serving-cert\") pod \"console-74f6fdf9f9-nrsfz\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:23:54.686356 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.683100 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb3eb372-ac0c-40e2-b940-1f00e251f56b-trusted-ca-bundle\") pod \"console-74f6fdf9f9-nrsfz\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:23:54.686356 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.683136 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb3eb372-ac0c-40e2-b940-1f00e251f56b-service-ca\") pod \"console-74f6fdf9f9-nrsfz\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:23:54.686356 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.683167 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb3eb372-ac0c-40e2-b940-1f00e251f56b-oauth-serving-cert\") pod \"console-74f6fdf9f9-nrsfz\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:23:54.686356 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.683197 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb3eb372-ac0c-40e2-b940-1f00e251f56b-console-config\") pod \"console-74f6fdf9f9-nrsfz\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:23:54.686356 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.683923 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb3eb372-ac0c-40e2-b940-1f00e251f56b-console-config\") pod \"console-74f6fdf9f9-nrsfz\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:23:54.687284 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.686896 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb3eb372-ac0c-40e2-b940-1f00e251f56b-console-oauth-config\") pod \"console-74f6fdf9f9-nrsfz\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:23:54.687837 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.687812 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb3eb372-ac0c-40e2-b940-1f00e251f56b-service-ca\") pod \"console-74f6fdf9f9-nrsfz\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:23:54.687963 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.687922 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb3eb372-ac0c-40e2-b940-1f00e251f56b-oauth-serving-cert\") pod \"console-74f6fdf9f9-nrsfz\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:23:54.688023 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.687929 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb3eb372-ac0c-40e2-b940-1f00e251f56b-trusted-ca-bundle\") pod \"console-74f6fdf9f9-nrsfz\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:23:54.689770 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.689744 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb3eb372-ac0c-40e2-b940-1f00e251f56b-console-serving-cert\") pod \"console-74f6fdf9f9-nrsfz\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:23:54.696092 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.696069 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzrqp\" (UniqueName: \"kubernetes.io/projected/cb3eb372-ac0c-40e2-b940-1f00e251f56b-kube-api-access-hzrqp\") pod \"console-74f6fdf9f9-nrsfz\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:23:54.822299 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:54.822221 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:23:55.099502 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:55.098367 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74f6fdf9f9-nrsfz"] Sep 17 07:23:55.101073 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:23:55.101045 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb3eb372_ac0c_40e2_b940_1f00e251f56b.slice/crio-3c5ec09f8847cef4d1ebe6155bf642781046421764d271c16e0f01b2638fcee4 WatchSource:0}: Error finding container 3c5ec09f8847cef4d1ebe6155bf642781046421764d271c16e0f01b2638fcee4: Status 404 returned error can't find the container with id 3c5ec09f8847cef4d1ebe6155bf642781046421764d271c16e0f01b2638fcee4 Sep 17 07:23:55.397075 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:55.396991 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74f6fdf9f9-nrsfz" event={"ID":"cb3eb372-ac0c-40e2-b940-1f00e251f56b","Type":"ContainerStarted","Data":"15f099780686e54cec0b3397e2b49276ff35d7c02cc6ff6de986bdd0e5528be1"} Sep 17 07:23:55.397075 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:55.397042 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74f6fdf9f9-nrsfz" event={"ID":"cb3eb372-ac0c-40e2-b940-1f00e251f56b","Type":"ContainerStarted","Data":"3c5ec09f8847cef4d1ebe6155bf642781046421764d271c16e0f01b2638fcee4"} Sep 17 07:23:55.422024 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:23:55.421968 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-74f6fdf9f9-nrsfz" podStartSLOduration=1.421951437 podStartE2EDuration="1.421951437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:23:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:23:55.420523158 +0000 UTC m=+242.606256152" watchObservedRunningTime="2026-09-17 07:23:55.421951437 +0000 UTC m=+242.607684408" Sep 17 07:24:04.822869 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:04.822834 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:24:04.823254 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:04.822909 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:24:04.827874 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:04.827848 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:24:05.168388 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:05.168290 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs\") pod \"network-metrics-daemon-x7484\" (UID: \"69c30c82-b237-4fc3-a96c-61840e73b7a7\") " pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:24:05.170812 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:05.170790 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69c30c82-b237-4fc3-a96c-61840e73b7a7-metrics-certs\") pod \"network-metrics-daemon-x7484\" (UID: \"69c30c82-b237-4fc3-a96c-61840e73b7a7\") " pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:24:05.412954 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:05.412925 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lkdnc\"" Sep 17 07:24:05.420388 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:05.420297 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x7484" Sep 17 07:24:05.442001 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:05.441979 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:24:05.491535 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:05.491510 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8667bf884f-ctv8f"] Sep 17 07:24:05.714343 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:05.714296 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-x7484"] Sep 17 07:24:05.717698 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:24:05.717671 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69c30c82_b237_4fc3_a96c_61840e73b7a7.slice/crio-51dab1b7698dabf62b24e0c08204bf5c2f5f26dec3227ca63b149fd791a115dd WatchSource:0}: Error finding container 51dab1b7698dabf62b24e0c08204bf5c2f5f26dec3227ca63b149fd791a115dd: Status 404 returned error can't find the container with id 51dab1b7698dabf62b24e0c08204bf5c2f5f26dec3227ca63b149fd791a115dd Sep 17 07:24:06.442204 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:06.442162 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x7484" event={"ID":"69c30c82-b237-4fc3-a96c-61840e73b7a7","Type":"ContainerStarted","Data":"51dab1b7698dabf62b24e0c08204bf5c2f5f26dec3227ca63b149fd791a115dd"} Sep 17 07:24:07.447948 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:07.447910 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x7484" event={"ID":"69c30c82-b237-4fc3-a96c-61840e73b7a7","Type":"ContainerStarted","Data":"4688723f814d2b5337c766f7158e281c406ed5e2f25f7b314fdebc85dd00e547"} Sep 17 07:24:07.448298 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:07.447965 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x7484" event={"ID":"69c30c82-b237-4fc3-a96c-61840e73b7a7","Type":"ContainerStarted","Data":"16abfbaf4a1db699ebbb3a75ce8bb768b8e66dc609d677df1bb7b78fa953c93a"} Sep 17 07:24:07.468340 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:07.468274 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-x7484" podStartSLOduration=253.581076277 podStartE2EDuration="4m14.468258017s" totalImagesPullingTime="887.18174ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:24:07.467467952 +0000 UTC m=+254.653200926" watchObservedRunningTime="2026-09-17 07:24:07.468258017 +0000 UTC m=+254.653990989" Sep 17 07:24:30.512592 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:30.512503 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8667bf884f-ctv8f" podUID="beaeb8ae-ae96-484e-856e-3d02bcd89eea" containerName="console" containerID="cri-o://2f39788206f21e7ef0bd30e029c130bb4585abc9811bba668f08d1c7789b821e" gracePeriod=15 Sep 17 07:24:30.762032 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:30.762011 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:24:30.873363 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:30.873279 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beaeb8ae-ae96-484e-856e-3d02bcd89eea-trusted-ca-bundle\") pod \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " Sep 17 07:24:30.873363 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:30.873314 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhxnp\" (UniqueName: \"kubernetes.io/projected/beaeb8ae-ae96-484e-856e-3d02bcd89eea-kube-api-access-nhxnp\") pod \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " Sep 17 07:24:30.873363 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:30.873354 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/beaeb8ae-ae96-484e-856e-3d02bcd89eea-service-ca\") pod \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " Sep 17 07:24:30.874073 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:30.873377 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/beaeb8ae-ae96-484e-856e-3d02bcd89eea-console-config\") pod \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " Sep 17 07:24:30.874073 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:30.873419 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/beaeb8ae-ae96-484e-856e-3d02bcd89eea-console-serving-cert\") pod \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " Sep 17 07:24:30.874073 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:30.873455 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/beaeb8ae-ae96-484e-856e-3d02bcd89eea-oauth-serving-cert\") pod \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " Sep 17 07:24:30.874073 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:30.873498 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/beaeb8ae-ae96-484e-856e-3d02bcd89eea-console-oauth-config\") pod \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\" (UID: \"beaeb8ae-ae96-484e-856e-3d02bcd89eea\") " Sep 17 07:24:30.874073 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:30.873770 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/beaeb8ae-ae96-484e-856e-3d02bcd89eea-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "beaeb8ae-ae96-484e-856e-3d02bcd89eea" (UID: "beaeb8ae-ae96-484e-856e-3d02bcd89eea"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:24:30.874073 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:30.873940 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/beaeb8ae-ae96-484e-856e-3d02bcd89eea-service-ca" (OuterVolumeSpecName: "service-ca") pod "beaeb8ae-ae96-484e-856e-3d02bcd89eea" (UID: "beaeb8ae-ae96-484e-856e-3d02bcd89eea"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:24:30.874073 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:30.873986 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/beaeb8ae-ae96-484e-856e-3d02bcd89eea-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "beaeb8ae-ae96-484e-856e-3d02bcd89eea" (UID: "beaeb8ae-ae96-484e-856e-3d02bcd89eea"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:24:30.874625 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:30.874603 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/beaeb8ae-ae96-484e-856e-3d02bcd89eea-console-config" (OuterVolumeSpecName: "console-config") pod "beaeb8ae-ae96-484e-856e-3d02bcd89eea" (UID: "beaeb8ae-ae96-484e-856e-3d02bcd89eea"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:24:30.876689 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:30.876664 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/beaeb8ae-ae96-484e-856e-3d02bcd89eea-kube-api-access-nhxnp" (OuterVolumeSpecName: "kube-api-access-nhxnp") pod "beaeb8ae-ae96-484e-856e-3d02bcd89eea" (UID: "beaeb8ae-ae96-484e-856e-3d02bcd89eea"). InnerVolumeSpecName "kube-api-access-nhxnp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:24:30.876689 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:30.876668 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/beaeb8ae-ae96-484e-856e-3d02bcd89eea-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "beaeb8ae-ae96-484e-856e-3d02bcd89eea" (UID: "beaeb8ae-ae96-484e-856e-3d02bcd89eea"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:24:30.876826 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:30.876751 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/beaeb8ae-ae96-484e-856e-3d02bcd89eea-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "beaeb8ae-ae96-484e-856e-3d02bcd89eea" (UID: "beaeb8ae-ae96-484e-856e-3d02bcd89eea"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:24:30.974804 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:30.974777 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/beaeb8ae-ae96-484e-856e-3d02bcd89eea-console-serving-cert\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:24:30.974804 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:30.974798 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/beaeb8ae-ae96-484e-856e-3d02bcd89eea-oauth-serving-cert\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:24:30.974917 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:30.974811 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/beaeb8ae-ae96-484e-856e-3d02bcd89eea-console-oauth-config\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:24:30.974917 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:30.974825 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beaeb8ae-ae96-484e-856e-3d02bcd89eea-trusted-ca-bundle\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:24:30.974917 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:30.974833 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhxnp\" (UniqueName: \"kubernetes.io/projected/beaeb8ae-ae96-484e-856e-3d02bcd89eea-kube-api-access-nhxnp\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:24:30.974917 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:30.974841 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/beaeb8ae-ae96-484e-856e-3d02bcd89eea-service-ca\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:24:30.974917 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:30.974849 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/beaeb8ae-ae96-484e-856e-3d02bcd89eea-console-config\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:24:31.545624 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:31.545578 2563 generic.go:356] "Generic (PLEG): container finished" podID="beaeb8ae-ae96-484e-856e-3d02bcd89eea" containerID="2f39788206f21e7ef0bd30e029c130bb4585abc9811bba668f08d1c7789b821e" exitCode=0 Sep 17 07:24:31.546042 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:31.545639 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8667bf884f-ctv8f" Sep 17 07:24:31.546042 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:31.545664 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8667bf884f-ctv8f" event={"ID":"beaeb8ae-ae96-484e-856e-3d02bcd89eea","Type":"ContainerDied","Data":"2f39788206f21e7ef0bd30e029c130bb4585abc9811bba668f08d1c7789b821e"} Sep 17 07:24:31.546042 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:31.545730 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8667bf884f-ctv8f" event={"ID":"beaeb8ae-ae96-484e-856e-3d02bcd89eea","Type":"ContainerDied","Data":"f1b561fd13bfe02babd7b81fa04afaf506cfa02f18cba6abcbb3cde47f62fb3d"} Sep 17 07:24:31.546042 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:31.545750 2563 scope.go:117] "RemoveContainer" containerID="2f39788206f21e7ef0bd30e029c130bb4585abc9811bba668f08d1c7789b821e" Sep 17 07:24:31.554357 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:31.554312 2563 scope.go:117] "RemoveContainer" containerID="2f39788206f21e7ef0bd30e029c130bb4585abc9811bba668f08d1c7789b821e" Sep 17 07:24:31.554667 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:24:31.554641 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f39788206f21e7ef0bd30e029c130bb4585abc9811bba668f08d1c7789b821e\": container with ID starting with 2f39788206f21e7ef0bd30e029c130bb4585abc9811bba668f08d1c7789b821e not found: ID does not exist" containerID="2f39788206f21e7ef0bd30e029c130bb4585abc9811bba668f08d1c7789b821e" Sep 17 07:24:31.554748 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:31.554672 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f39788206f21e7ef0bd30e029c130bb4585abc9811bba668f08d1c7789b821e"} err="failed to get container status \"2f39788206f21e7ef0bd30e029c130bb4585abc9811bba668f08d1c7789b821e\": rpc error: code = NotFound desc = could not find container \"2f39788206f21e7ef0bd30e029c130bb4585abc9811bba668f08d1c7789b821e\": container with ID starting with 2f39788206f21e7ef0bd30e029c130bb4585abc9811bba668f08d1c7789b821e not found: ID does not exist" Sep 17 07:24:31.562402 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:31.562378 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8667bf884f-ctv8f"] Sep 17 07:24:31.564544 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:31.564520 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8667bf884f-ctv8f"] Sep 17 07:24:33.313977 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:33.313939 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="beaeb8ae-ae96-484e-856e-3d02bcd89eea" path="/var/lib/kubelet/pods/beaeb8ae-ae96-484e-856e-3d02bcd89eea/volumes" Sep 17 07:24:53.221228 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:24:53.221197 2563 kubelet.go:1648] "Image garbage collection succeeded" Sep 17 07:25:20.227811 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.227781 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79cbf7459b-r8f2k"] Sep 17 07:25:20.230210 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.228077 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="beaeb8ae-ae96-484e-856e-3d02bcd89eea" containerName="console" Sep 17 07:25:20.230210 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.228088 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="beaeb8ae-ae96-484e-856e-3d02bcd89eea" containerName="console" Sep 17 07:25:20.230210 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.228154 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="beaeb8ae-ae96-484e-856e-3d02bcd89eea" containerName="console" Sep 17 07:25:20.231087 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.231072 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:20.254846 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.254813 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79cbf7459b-r8f2k"] Sep 17 07:25:20.347609 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.347568 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/83175f0a-edaf-4002-937b-a06c19647b7a-console-config\") pod \"console-79cbf7459b-r8f2k\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:20.347609 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.347606 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83175f0a-edaf-4002-937b-a06c19647b7a-trusted-ca-bundle\") pod \"console-79cbf7459b-r8f2k\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:20.347877 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.347637 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgwkn\" (UniqueName: \"kubernetes.io/projected/83175f0a-edaf-4002-937b-a06c19647b7a-kube-api-access-zgwkn\") pod \"console-79cbf7459b-r8f2k\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:20.347877 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.347757 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/83175f0a-edaf-4002-937b-a06c19647b7a-console-oauth-config\") pod \"console-79cbf7459b-r8f2k\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:20.347877 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.347815 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/83175f0a-edaf-4002-937b-a06c19647b7a-oauth-serving-cert\") pod \"console-79cbf7459b-r8f2k\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:20.347877 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.347846 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/83175f0a-edaf-4002-937b-a06c19647b7a-service-ca\") pod \"console-79cbf7459b-r8f2k\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:20.348048 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.347908 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/83175f0a-edaf-4002-937b-a06c19647b7a-console-serving-cert\") pod \"console-79cbf7459b-r8f2k\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:20.449096 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.449055 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/83175f0a-edaf-4002-937b-a06c19647b7a-console-oauth-config\") pod \"console-79cbf7459b-r8f2k\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:20.449096 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.449100 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/83175f0a-edaf-4002-937b-a06c19647b7a-oauth-serving-cert\") pod \"console-79cbf7459b-r8f2k\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:20.449351 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.449119 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/83175f0a-edaf-4002-937b-a06c19647b7a-service-ca\") pod \"console-79cbf7459b-r8f2k\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:20.449351 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.449144 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/83175f0a-edaf-4002-937b-a06c19647b7a-console-serving-cert\") pod \"console-79cbf7459b-r8f2k\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:20.449351 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.449169 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/83175f0a-edaf-4002-937b-a06c19647b7a-console-config\") pod \"console-79cbf7459b-r8f2k\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:20.449501 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.449383 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83175f0a-edaf-4002-937b-a06c19647b7a-trusted-ca-bundle\") pod \"console-79cbf7459b-r8f2k\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:20.449501 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.449431 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgwkn\" (UniqueName: \"kubernetes.io/projected/83175f0a-edaf-4002-937b-a06c19647b7a-kube-api-access-zgwkn\") pod \"console-79cbf7459b-r8f2k\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:20.450048 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.449969 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/83175f0a-edaf-4002-937b-a06c19647b7a-service-ca\") pod \"console-79cbf7459b-r8f2k\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:20.450048 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.449973 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/83175f0a-edaf-4002-937b-a06c19647b7a-console-config\") pod \"console-79cbf7459b-r8f2k\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:20.450283 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.450130 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/83175f0a-edaf-4002-937b-a06c19647b7a-oauth-serving-cert\") pod \"console-79cbf7459b-r8f2k\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:20.450283 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.450193 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83175f0a-edaf-4002-937b-a06c19647b7a-trusted-ca-bundle\") pod \"console-79cbf7459b-r8f2k\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:20.451808 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.451778 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/83175f0a-edaf-4002-937b-a06c19647b7a-console-serving-cert\") pod \"console-79cbf7459b-r8f2k\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:20.451808 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.451801 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/83175f0a-edaf-4002-937b-a06c19647b7a-console-oauth-config\") pod \"console-79cbf7459b-r8f2k\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:20.459530 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.459510 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgwkn\" (UniqueName: \"kubernetes.io/projected/83175f0a-edaf-4002-937b-a06c19647b7a-kube-api-access-zgwkn\") pod \"console-79cbf7459b-r8f2k\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:20.540804 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.540715 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:20.828740 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:20.828686 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79cbf7459b-r8f2k"] Sep 17 07:25:20.831492 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:25:20.831463 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83175f0a_edaf_4002_937b_a06c19647b7a.slice/crio-5ac204c5a31b16ff74d34340e036e4931e1e08304c8ba646882b6e48fb93eaa2 WatchSource:0}: Error finding container 5ac204c5a31b16ff74d34340e036e4931e1e08304c8ba646882b6e48fb93eaa2: Status 404 returned error can't find the container with id 5ac204c5a31b16ff74d34340e036e4931e1e08304c8ba646882b6e48fb93eaa2 Sep 17 07:25:21.754487 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:21.754449 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79cbf7459b-r8f2k" event={"ID":"83175f0a-edaf-4002-937b-a06c19647b7a","Type":"ContainerStarted","Data":"dd559a76d939d3dba57a77cba356cfbdba8bd11ae2631931dacf18eaee5fec3c"} Sep 17 07:25:21.754898 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:21.754500 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79cbf7459b-r8f2k" event={"ID":"83175f0a-edaf-4002-937b-a06c19647b7a","Type":"ContainerStarted","Data":"5ac204c5a31b16ff74d34340e036e4931e1e08304c8ba646882b6e48fb93eaa2"} Sep 17 07:25:21.778024 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:21.777973 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79cbf7459b-r8f2k" podStartSLOduration=1.7779570470000001 podStartE2EDuration="1.777957047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:25:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:25:21.774696266 +0000 UTC m=+328.960429237" watchObservedRunningTime="2026-09-17 07:25:21.777957047 +0000 UTC m=+328.963690019" Sep 17 07:25:30.541502 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:30.541463 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:30.541891 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:30.541525 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:30.546183 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:30.546160 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:30.795445 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:30.795353 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:25:30.846358 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:30.846293 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74f6fdf9f9-nrsfz"] Sep 17 07:25:55.866833 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:55.866725 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-74f6fdf9f9-nrsfz" podUID="cb3eb372-ac0c-40e2-b940-1f00e251f56b" containerName="console" containerID="cri-o://15f099780686e54cec0b3397e2b49276ff35d7c02cc6ff6de986bdd0e5528be1" gracePeriod=15 Sep 17 07:25:56.104166 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.104145 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:25:56.165376 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.165293 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb3eb372-ac0c-40e2-b940-1f00e251f56b-console-serving-cert\") pod \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " Sep 17 07:25:56.165376 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.165356 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb3eb372-ac0c-40e2-b940-1f00e251f56b-service-ca\") pod \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " Sep 17 07:25:56.165564 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.165392 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb3eb372-ac0c-40e2-b940-1f00e251f56b-console-oauth-config\") pod \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " Sep 17 07:25:56.165564 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.165413 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb3eb372-ac0c-40e2-b940-1f00e251f56b-oauth-serving-cert\") pod \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " Sep 17 07:25:56.165564 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.165544 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb3eb372-ac0c-40e2-b940-1f00e251f56b-console-config\") pod \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " Sep 17 07:25:56.165724 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.165575 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzrqp\" (UniqueName: \"kubernetes.io/projected/cb3eb372-ac0c-40e2-b940-1f00e251f56b-kube-api-access-hzrqp\") pod \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " Sep 17 07:25:56.165724 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.165656 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb3eb372-ac0c-40e2-b940-1f00e251f56b-trusted-ca-bundle\") pod \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\" (UID: \"cb3eb372-ac0c-40e2-b940-1f00e251f56b\") " Sep 17 07:25:56.165877 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.165854 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb3eb372-ac0c-40e2-b940-1f00e251f56b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cb3eb372-ac0c-40e2-b940-1f00e251f56b" (UID: "cb3eb372-ac0c-40e2-b940-1f00e251f56b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:25:56.165926 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.165868 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb3eb372-ac0c-40e2-b940-1f00e251f56b-service-ca" (OuterVolumeSpecName: "service-ca") pod "cb3eb372-ac0c-40e2-b940-1f00e251f56b" (UID: "cb3eb372-ac0c-40e2-b940-1f00e251f56b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:25:56.165926 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.165892 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb3eb372-ac0c-40e2-b940-1f00e251f56b-console-config" (OuterVolumeSpecName: "console-config") pod "cb3eb372-ac0c-40e2-b940-1f00e251f56b" (UID: "cb3eb372-ac0c-40e2-b940-1f00e251f56b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:25:56.166018 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.165957 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb3eb372-ac0c-40e2-b940-1f00e251f56b-service-ca\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:25:56.166085 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.166023 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb3eb372-ac0c-40e2-b940-1f00e251f56b-oauth-serving-cert\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:25:56.166162 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.166150 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb3eb372-ac0c-40e2-b940-1f00e251f56b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cb3eb372-ac0c-40e2-b940-1f00e251f56b" (UID: "cb3eb372-ac0c-40e2-b940-1f00e251f56b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:25:56.167644 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.167618 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb3eb372-ac0c-40e2-b940-1f00e251f56b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cb3eb372-ac0c-40e2-b940-1f00e251f56b" (UID: "cb3eb372-ac0c-40e2-b940-1f00e251f56b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:25:56.167758 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.167656 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb3eb372-ac0c-40e2-b940-1f00e251f56b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cb3eb372-ac0c-40e2-b940-1f00e251f56b" (UID: "cb3eb372-ac0c-40e2-b940-1f00e251f56b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:25:56.167960 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.167924 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb3eb372-ac0c-40e2-b940-1f00e251f56b-kube-api-access-hzrqp" (OuterVolumeSpecName: "kube-api-access-hzrqp") pod "cb3eb372-ac0c-40e2-b940-1f00e251f56b" (UID: "cb3eb372-ac0c-40e2-b940-1f00e251f56b"). InnerVolumeSpecName "kube-api-access-hzrqp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:25:56.267082 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.267043 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb3eb372-ac0c-40e2-b940-1f00e251f56b-console-oauth-config\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:25:56.267082 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.267077 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb3eb372-ac0c-40e2-b940-1f00e251f56b-console-config\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:25:56.267082 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.267087 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzrqp\" (UniqueName: \"kubernetes.io/projected/cb3eb372-ac0c-40e2-b940-1f00e251f56b-kube-api-access-hzrqp\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:25:56.267082 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.267095 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb3eb372-ac0c-40e2-b940-1f00e251f56b-trusted-ca-bundle\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:25:56.267316 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.267103 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb3eb372-ac0c-40e2-b940-1f00e251f56b-console-serving-cert\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:25:56.895644 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.895614 2563 generic.go:356] "Generic (PLEG): container finished" podID="cb3eb372-ac0c-40e2-b940-1f00e251f56b" containerID="15f099780686e54cec0b3397e2b49276ff35d7c02cc6ff6de986bdd0e5528be1" exitCode=0 Sep 17 07:25:56.896078 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.895666 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74f6fdf9f9-nrsfz" Sep 17 07:25:56.896078 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.895679 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74f6fdf9f9-nrsfz" event={"ID":"cb3eb372-ac0c-40e2-b940-1f00e251f56b","Type":"ContainerDied","Data":"15f099780686e54cec0b3397e2b49276ff35d7c02cc6ff6de986bdd0e5528be1"} Sep 17 07:25:56.896078 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.895719 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74f6fdf9f9-nrsfz" event={"ID":"cb3eb372-ac0c-40e2-b940-1f00e251f56b","Type":"ContainerDied","Data":"3c5ec09f8847cef4d1ebe6155bf642781046421764d271c16e0f01b2638fcee4"} Sep 17 07:25:56.896078 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.895738 2563 scope.go:117] "RemoveContainer" containerID="15f099780686e54cec0b3397e2b49276ff35d7c02cc6ff6de986bdd0e5528be1" Sep 17 07:25:56.904934 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.904905 2563 scope.go:117] "RemoveContainer" containerID="15f099780686e54cec0b3397e2b49276ff35d7c02cc6ff6de986bdd0e5528be1" Sep 17 07:25:56.905213 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:25:56.905194 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15f099780686e54cec0b3397e2b49276ff35d7c02cc6ff6de986bdd0e5528be1\": container with ID starting with 15f099780686e54cec0b3397e2b49276ff35d7c02cc6ff6de986bdd0e5528be1 not found: ID does not exist" containerID="15f099780686e54cec0b3397e2b49276ff35d7c02cc6ff6de986bdd0e5528be1" Sep 17 07:25:56.905288 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.905221 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15f099780686e54cec0b3397e2b49276ff35d7c02cc6ff6de986bdd0e5528be1"} err="failed to get container status \"15f099780686e54cec0b3397e2b49276ff35d7c02cc6ff6de986bdd0e5528be1\": rpc error: code = NotFound desc = could not find container \"15f099780686e54cec0b3397e2b49276ff35d7c02cc6ff6de986bdd0e5528be1\": container with ID starting with 15f099780686e54cec0b3397e2b49276ff35d7c02cc6ff6de986bdd0e5528be1 not found: ID does not exist" Sep 17 07:25:56.918441 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.918415 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74f6fdf9f9-nrsfz"] Sep 17 07:25:56.924839 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:56.924816 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-74f6fdf9f9-nrsfz"] Sep 17 07:25:57.314079 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:25:57.314049 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb3eb372-ac0c-40e2-b940-1f00e251f56b" path="/var/lib/kubelet/pods/cb3eb372-ac0c-40e2-b940-1f00e251f56b/volumes" Sep 17 07:27:34.334565 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.334488 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57c46cc9c7-dfmx4"] Sep 17 07:27:34.335000 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.334795 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb3eb372-ac0c-40e2-b940-1f00e251f56b" containerName="console" Sep 17 07:27:34.335000 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.334806 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb3eb372-ac0c-40e2-b940-1f00e251f56b" containerName="console" Sep 17 07:27:34.335000 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.334870 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb3eb372-ac0c-40e2-b940-1f00e251f56b" containerName="console" Sep 17 07:27:34.337779 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.337758 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:34.366125 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.366096 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57c46cc9c7-dfmx4"] Sep 17 07:27:34.444752 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.444715 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7791fe6-9290-4cb6-b8ce-6280bfd843d6-console-config\") pod \"console-57c46cc9c7-dfmx4\" (UID: \"a7791fe6-9290-4cb6-b8ce-6280bfd843d6\") " pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:34.444752 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.444756 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7791fe6-9290-4cb6-b8ce-6280bfd843d6-oauth-serving-cert\") pod \"console-57c46cc9c7-dfmx4\" (UID: \"a7791fe6-9290-4cb6-b8ce-6280bfd843d6\") " pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:34.445022 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.444785 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7791fe6-9290-4cb6-b8ce-6280bfd843d6-console-serving-cert\") pod \"console-57c46cc9c7-dfmx4\" (UID: \"a7791fe6-9290-4cb6-b8ce-6280bfd843d6\") " pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:34.445022 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.444805 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7791fe6-9290-4cb6-b8ce-6280bfd843d6-trusted-ca-bundle\") pod \"console-57c46cc9c7-dfmx4\" (UID: \"a7791fe6-9290-4cb6-b8ce-6280bfd843d6\") " pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:34.445022 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.444823 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksjwv\" (UniqueName: \"kubernetes.io/projected/a7791fe6-9290-4cb6-b8ce-6280bfd843d6-kube-api-access-ksjwv\") pod \"console-57c46cc9c7-dfmx4\" (UID: \"a7791fe6-9290-4cb6-b8ce-6280bfd843d6\") " pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:34.445022 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.444889 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7791fe6-9290-4cb6-b8ce-6280bfd843d6-console-oauth-config\") pod \"console-57c46cc9c7-dfmx4\" (UID: \"a7791fe6-9290-4cb6-b8ce-6280bfd843d6\") " pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:34.445022 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.444934 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7791fe6-9290-4cb6-b8ce-6280bfd843d6-service-ca\") pod \"console-57c46cc9c7-dfmx4\" (UID: \"a7791fe6-9290-4cb6-b8ce-6280bfd843d6\") " pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:34.546234 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.546190 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7791fe6-9290-4cb6-b8ce-6280bfd843d6-trusted-ca-bundle\") pod \"console-57c46cc9c7-dfmx4\" (UID: \"a7791fe6-9290-4cb6-b8ce-6280bfd843d6\") " pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:34.546234 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.546238 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksjwv\" (UniqueName: \"kubernetes.io/projected/a7791fe6-9290-4cb6-b8ce-6280bfd843d6-kube-api-access-ksjwv\") pod \"console-57c46cc9c7-dfmx4\" (UID: \"a7791fe6-9290-4cb6-b8ce-6280bfd843d6\") " pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:34.546539 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.546291 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7791fe6-9290-4cb6-b8ce-6280bfd843d6-console-oauth-config\") pod \"console-57c46cc9c7-dfmx4\" (UID: \"a7791fe6-9290-4cb6-b8ce-6280bfd843d6\") " pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:34.546539 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.546362 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7791fe6-9290-4cb6-b8ce-6280bfd843d6-service-ca\") pod \"console-57c46cc9c7-dfmx4\" (UID: \"a7791fe6-9290-4cb6-b8ce-6280bfd843d6\") " pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:34.546539 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.546445 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7791fe6-9290-4cb6-b8ce-6280bfd843d6-console-config\") pod \"console-57c46cc9c7-dfmx4\" (UID: \"a7791fe6-9290-4cb6-b8ce-6280bfd843d6\") " pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:34.546539 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.546474 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7791fe6-9290-4cb6-b8ce-6280bfd843d6-oauth-serving-cert\") pod \"console-57c46cc9c7-dfmx4\" (UID: \"a7791fe6-9290-4cb6-b8ce-6280bfd843d6\") " pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:34.546539 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.546516 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7791fe6-9290-4cb6-b8ce-6280bfd843d6-console-serving-cert\") pod \"console-57c46cc9c7-dfmx4\" (UID: \"a7791fe6-9290-4cb6-b8ce-6280bfd843d6\") " pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:34.547177 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.547149 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7791fe6-9290-4cb6-b8ce-6280bfd843d6-service-ca\") pod \"console-57c46cc9c7-dfmx4\" (UID: \"a7791fe6-9290-4cb6-b8ce-6280bfd843d6\") " pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:34.547397 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.547298 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7791fe6-9290-4cb6-b8ce-6280bfd843d6-console-config\") pod \"console-57c46cc9c7-dfmx4\" (UID: \"a7791fe6-9290-4cb6-b8ce-6280bfd843d6\") " pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:34.547397 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.547304 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7791fe6-9290-4cb6-b8ce-6280bfd843d6-oauth-serving-cert\") pod \"console-57c46cc9c7-dfmx4\" (UID: \"a7791fe6-9290-4cb6-b8ce-6280bfd843d6\") " pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:34.547534 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.547374 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7791fe6-9290-4cb6-b8ce-6280bfd843d6-trusted-ca-bundle\") pod \"console-57c46cc9c7-dfmx4\" (UID: \"a7791fe6-9290-4cb6-b8ce-6280bfd843d6\") " pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:34.549092 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.549072 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7791fe6-9290-4cb6-b8ce-6280bfd843d6-console-serving-cert\") pod \"console-57c46cc9c7-dfmx4\" (UID: \"a7791fe6-9290-4cb6-b8ce-6280bfd843d6\") " pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:34.549177 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.549099 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7791fe6-9290-4cb6-b8ce-6280bfd843d6-console-oauth-config\") pod \"console-57c46cc9c7-dfmx4\" (UID: \"a7791fe6-9290-4cb6-b8ce-6280bfd843d6\") " pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:34.556590 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.556564 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksjwv\" (UniqueName: \"kubernetes.io/projected/a7791fe6-9290-4cb6-b8ce-6280bfd843d6-kube-api-access-ksjwv\") pod \"console-57c46cc9c7-dfmx4\" (UID: \"a7791fe6-9290-4cb6-b8ce-6280bfd843d6\") " pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:34.648005 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.647909 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:34.924533 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:34.924510 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57c46cc9c7-dfmx4"] Sep 17 07:27:34.927121 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:27:34.927086 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7791fe6_9290_4cb6_b8ce_6280bfd843d6.slice/crio-988651cff2aa1e3628ef9fce6ef2e34d253aaef8ac8f97e8ff47a96f85739068 WatchSource:0}: Error finding container 988651cff2aa1e3628ef9fce6ef2e34d253aaef8ac8f97e8ff47a96f85739068: Status 404 returned error can't find the container with id 988651cff2aa1e3628ef9fce6ef2e34d253aaef8ac8f97e8ff47a96f85739068 Sep 17 07:27:35.283846 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:35.283807 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57c46cc9c7-dfmx4" event={"ID":"a7791fe6-9290-4cb6-b8ce-6280bfd843d6","Type":"ContainerStarted","Data":"9dfb3b240ebaf58f64d6f64db0640ed2d0e982d213816d288f26066f2747cfd5"} Sep 17 07:27:35.284017 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:35.283868 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57c46cc9c7-dfmx4" event={"ID":"a7791fe6-9290-4cb6-b8ce-6280bfd843d6","Type":"ContainerStarted","Data":"988651cff2aa1e3628ef9fce6ef2e34d253aaef8ac8f97e8ff47a96f85739068"} Sep 17 07:27:35.303175 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:35.303119 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57c46cc9c7-dfmx4" podStartSLOduration=1.303097648 podStartE2EDuration="1.303097648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:27:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:27:35.30258414 +0000 UTC m=+462.488317124" watchObservedRunningTime="2026-09-17 07:27:35.303097648 +0000 UTC m=+462.488830619" Sep 17 07:27:39.596437 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:39.596396 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-8699f9b4c7-dpjp8"] Sep 17 07:27:39.599836 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:39.599819 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-8699f9b4c7-dpjp8" Sep 17 07:27:39.605267 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:39.605240 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Sep 17 07:27:39.605410 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:39.605241 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-sgnpj\"" Sep 17 07:27:39.605586 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:39.605569 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Sep 17 07:27:39.625642 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:39.625612 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-8699f9b4c7-dpjp8"] Sep 17 07:27:39.691911 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:39.691869 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqm7f\" (UniqueName: \"kubernetes.io/projected/96e0c094-1439-4724-bfa0-f0c3c79f0a17-kube-api-access-cqm7f\") pod \"feast-operator-controller-manager-8699f9b4c7-dpjp8\" (UID: \"96e0c094-1439-4724-bfa0-f0c3c79f0a17\") " pod="feast-operator-system/feast-operator-controller-manager-8699f9b4c7-dpjp8" Sep 17 07:27:39.793132 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:39.793095 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqm7f\" (UniqueName: \"kubernetes.io/projected/96e0c094-1439-4724-bfa0-f0c3c79f0a17-kube-api-access-cqm7f\") pod \"feast-operator-controller-manager-8699f9b4c7-dpjp8\" (UID: \"96e0c094-1439-4724-bfa0-f0c3c79f0a17\") " pod="feast-operator-system/feast-operator-controller-manager-8699f9b4c7-dpjp8" Sep 17 07:27:39.802834 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:39.802805 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqm7f\" (UniqueName: \"kubernetes.io/projected/96e0c094-1439-4724-bfa0-f0c3c79f0a17-kube-api-access-cqm7f\") pod \"feast-operator-controller-manager-8699f9b4c7-dpjp8\" (UID: \"96e0c094-1439-4724-bfa0-f0c3c79f0a17\") " pod="feast-operator-system/feast-operator-controller-manager-8699f9b4c7-dpjp8" Sep 17 07:27:39.910699 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:39.910597 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-8699f9b4c7-dpjp8" Sep 17 07:27:40.189781 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:40.189698 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-8699f9b4c7-dpjp8"] Sep 17 07:27:40.193993 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:27:40.193959 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96e0c094_1439_4724_bfa0_f0c3c79f0a17.slice/crio-5f74e8dd62e36d4d811e230a3605d7d55a8ec797f6e130b489bd033d181d110a WatchSource:0}: Error finding container 5f74e8dd62e36d4d811e230a3605d7d55a8ec797f6e130b489bd033d181d110a: Status 404 returned error can't find the container with id 5f74e8dd62e36d4d811e230a3605d7d55a8ec797f6e130b489bd033d181d110a Sep 17 07:27:40.196130 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:40.196110 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 07:27:40.305223 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:40.305189 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-8699f9b4c7-dpjp8" event={"ID":"96e0c094-1439-4724-bfa0-f0c3c79f0a17","Type":"ContainerStarted","Data":"5f74e8dd62e36d4d811e230a3605d7d55a8ec797f6e130b489bd033d181d110a"} Sep 17 07:27:43.321388 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:43.321351 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-8699f9b4c7-dpjp8" event={"ID":"96e0c094-1439-4724-bfa0-f0c3c79f0a17","Type":"ContainerStarted","Data":"3231cae819af892886f0f65f247092ad361229da6fb8d7a95099bb85a75d7257"} Sep 17 07:27:43.321775 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:43.321474 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-8699f9b4c7-dpjp8" Sep 17 07:27:43.344411 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:43.344366 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-8699f9b4c7-dpjp8" podStartSLOduration=1.951949955 podStartE2EDuration="4.344349986s" totalImagesPullingTime="2.392400031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:27:43.34198374 +0000 UTC m=+470.527716710" watchObservedRunningTime="2026-09-17 07:27:43.344349986 +0000 UTC m=+470.530083011" Sep 17 07:27:44.648277 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:44.648177 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:44.648277 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:44.648252 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:44.653202 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:44.653176 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:45.335831 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:45.335795 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57c46cc9c7-dfmx4" Sep 17 07:27:45.391724 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:45.391682 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79cbf7459b-r8f2k"] Sep 17 07:27:54.327549 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:27:54.327521 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-8699f9b4c7-dpjp8" Sep 17 07:28:10.413040 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:10.412998 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-79cbf7459b-r8f2k" podUID="83175f0a-edaf-4002-937b-a06c19647b7a" containerName="console" containerID="cri-o://dd559a76d939d3dba57a77cba356cfbdba8bd11ae2631931dacf18eaee5fec3c" gracePeriod=15 Sep 17 07:28:10.654902 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:10.654880 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:28:10.740345 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:10.740288 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83175f0a-edaf-4002-937b-a06c19647b7a-trusted-ca-bundle\") pod \"83175f0a-edaf-4002-937b-a06c19647b7a\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " Sep 17 07:28:10.740483 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:10.740349 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/83175f0a-edaf-4002-937b-a06c19647b7a-console-oauth-config\") pod \"83175f0a-edaf-4002-937b-a06c19647b7a\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " Sep 17 07:28:10.740483 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:10.740405 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/83175f0a-edaf-4002-937b-a06c19647b7a-console-serving-cert\") pod \"83175f0a-edaf-4002-937b-a06c19647b7a\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " Sep 17 07:28:10.740483 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:10.740431 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/83175f0a-edaf-4002-937b-a06c19647b7a-oauth-serving-cert\") pod \"83175f0a-edaf-4002-937b-a06c19647b7a\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " Sep 17 07:28:10.740483 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:10.740458 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/83175f0a-edaf-4002-937b-a06c19647b7a-service-ca\") pod \"83175f0a-edaf-4002-937b-a06c19647b7a\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " Sep 17 07:28:10.740685 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:10.740487 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/83175f0a-edaf-4002-937b-a06c19647b7a-console-config\") pod \"83175f0a-edaf-4002-937b-a06c19647b7a\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " Sep 17 07:28:10.740685 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:10.740546 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgwkn\" (UniqueName: \"kubernetes.io/projected/83175f0a-edaf-4002-937b-a06c19647b7a-kube-api-access-zgwkn\") pod \"83175f0a-edaf-4002-937b-a06c19647b7a\" (UID: \"83175f0a-edaf-4002-937b-a06c19647b7a\") " Sep 17 07:28:10.740830 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:10.740712 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83175f0a-edaf-4002-937b-a06c19647b7a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "83175f0a-edaf-4002-937b-a06c19647b7a" (UID: "83175f0a-edaf-4002-937b-a06c19647b7a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:28:10.740900 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:10.740829 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83175f0a-edaf-4002-937b-a06c19647b7a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "83175f0a-edaf-4002-937b-a06c19647b7a" (UID: "83175f0a-edaf-4002-937b-a06c19647b7a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:28:10.740900 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:10.740859 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83175f0a-edaf-4002-937b-a06c19647b7a-service-ca" (OuterVolumeSpecName: "service-ca") pod "83175f0a-edaf-4002-937b-a06c19647b7a" (UID: "83175f0a-edaf-4002-937b-a06c19647b7a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:28:10.741023 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:10.741003 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83175f0a-edaf-4002-937b-a06c19647b7a-console-config" (OuterVolumeSpecName: "console-config") pod "83175f0a-edaf-4002-937b-a06c19647b7a" (UID: "83175f0a-edaf-4002-937b-a06c19647b7a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:28:10.742595 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:10.742570 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83175f0a-edaf-4002-937b-a06c19647b7a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "83175f0a-edaf-4002-937b-a06c19647b7a" (UID: "83175f0a-edaf-4002-937b-a06c19647b7a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:28:10.742724 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:10.742639 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83175f0a-edaf-4002-937b-a06c19647b7a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "83175f0a-edaf-4002-937b-a06c19647b7a" (UID: "83175f0a-edaf-4002-937b-a06c19647b7a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:28:10.742787 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:10.742765 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83175f0a-edaf-4002-937b-a06c19647b7a-kube-api-access-zgwkn" (OuterVolumeSpecName: "kube-api-access-zgwkn") pod "83175f0a-edaf-4002-937b-a06c19647b7a" (UID: "83175f0a-edaf-4002-937b-a06c19647b7a"). InnerVolumeSpecName "kube-api-access-zgwkn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:28:10.842050 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:10.842018 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83175f0a-edaf-4002-937b-a06c19647b7a-trusted-ca-bundle\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:28:10.842050 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:10.842047 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/83175f0a-edaf-4002-937b-a06c19647b7a-console-oauth-config\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:28:10.842050 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:10.842056 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/83175f0a-edaf-4002-937b-a06c19647b7a-console-serving-cert\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:28:10.842214 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:10.842065 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/83175f0a-edaf-4002-937b-a06c19647b7a-oauth-serving-cert\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:28:10.842214 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:10.842073 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/83175f0a-edaf-4002-937b-a06c19647b7a-service-ca\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:28:10.842214 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:10.842081 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/83175f0a-edaf-4002-937b-a06c19647b7a-console-config\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:28:10.842214 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:10.842088 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgwkn\" (UniqueName: \"kubernetes.io/projected/83175f0a-edaf-4002-937b-a06c19647b7a-kube-api-access-zgwkn\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:28:11.431152 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:11.431120 2563 generic.go:356] "Generic (PLEG): container finished" podID="83175f0a-edaf-4002-937b-a06c19647b7a" containerID="dd559a76d939d3dba57a77cba356cfbdba8bd11ae2631931dacf18eaee5fec3c" exitCode=0 Sep 17 07:28:11.431575 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:11.431162 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79cbf7459b-r8f2k" Sep 17 07:28:11.431575 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:11.431197 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79cbf7459b-r8f2k" event={"ID":"83175f0a-edaf-4002-937b-a06c19647b7a","Type":"ContainerDied","Data":"dd559a76d939d3dba57a77cba356cfbdba8bd11ae2631931dacf18eaee5fec3c"} Sep 17 07:28:11.431575 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:11.431237 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79cbf7459b-r8f2k" event={"ID":"83175f0a-edaf-4002-937b-a06c19647b7a","Type":"ContainerDied","Data":"5ac204c5a31b16ff74d34340e036e4931e1e08304c8ba646882b6e48fb93eaa2"} Sep 17 07:28:11.431575 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:11.431254 2563 scope.go:117] "RemoveContainer" containerID="dd559a76d939d3dba57a77cba356cfbdba8bd11ae2631931dacf18eaee5fec3c" Sep 17 07:28:11.439667 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:11.439652 2563 scope.go:117] "RemoveContainer" containerID="dd559a76d939d3dba57a77cba356cfbdba8bd11ae2631931dacf18eaee5fec3c" Sep 17 07:28:11.439910 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:28:11.439895 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd559a76d939d3dba57a77cba356cfbdba8bd11ae2631931dacf18eaee5fec3c\": container with ID starting with dd559a76d939d3dba57a77cba356cfbdba8bd11ae2631931dacf18eaee5fec3c not found: ID does not exist" containerID="dd559a76d939d3dba57a77cba356cfbdba8bd11ae2631931dacf18eaee5fec3c" Sep 17 07:28:11.439965 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:11.439918 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd559a76d939d3dba57a77cba356cfbdba8bd11ae2631931dacf18eaee5fec3c"} err="failed to get container status \"dd559a76d939d3dba57a77cba356cfbdba8bd11ae2631931dacf18eaee5fec3c\": rpc error: code = NotFound desc = could not find container \"dd559a76d939d3dba57a77cba356cfbdba8bd11ae2631931dacf18eaee5fec3c\": container with ID starting with dd559a76d939d3dba57a77cba356cfbdba8bd11ae2631931dacf18eaee5fec3c not found: ID does not exist" Sep 17 07:28:11.454166 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:11.454146 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79cbf7459b-r8f2k"] Sep 17 07:28:11.458196 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:11.458176 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-79cbf7459b-r8f2k"] Sep 17 07:28:13.314291 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:13.314259 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83175f0a-edaf-4002-937b-a06c19647b7a" path="/var/lib/kubelet/pods/83175f0a-edaf-4002-937b-a06c19647b7a/volumes" Sep 17 07:28:18.499442 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.499398 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc"] Sep 17 07:28:18.499850 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.499834 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83175f0a-edaf-4002-937b-a06c19647b7a" containerName="console" Sep 17 07:28:18.499850 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.499851 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="83175f0a-edaf-4002-937b-a06c19647b7a" containerName="console" Sep 17 07:28:18.499934 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.499925 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="83175f0a-edaf-4002-937b-a06c19647b7a" containerName="console" Sep 17 07:28:18.503285 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.503270 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:28:18.508221 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.508160 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Sep 17 07:28:18.508221 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.508195 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 17 07:28:18.508453 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.508314 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Sep 17 07:28:18.508519 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.508486 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-v9gsc\"" Sep 17 07:28:18.508592 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.508542 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 17 07:28:18.508680 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.508662 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 17 07:28:18.509615 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.509590 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Sep 17 07:28:18.509722 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.509673 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Sep 17 07:28:18.537749 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.537726 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc"] Sep 17 07:28:18.598937 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.598917 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-ui-tls\") pod \"feast-simple-feast-setup-b6bf84db7-6msnc\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:28:18.599031 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.598964 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-offline-tls\") pod \"feast-simple-feast-setup-b6bf84db7-6msnc\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:28:18.599031 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.598981 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ef115454-197f-4272-b3d7-148e48cabcfd-feast-data\") pod \"feast-simple-feast-setup-b6bf84db7-6msnc\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:28:18.599031 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.598999 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcs9m\" (UniqueName: \"kubernetes.io/projected/ef115454-197f-4272-b3d7-148e48cabcfd-kube-api-access-wcs9m\") pod \"feast-simple-feast-setup-b6bf84db7-6msnc\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:28:18.599031 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.599023 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-registry-tls\") pod \"feast-simple-feast-setup-b6bf84db7-6msnc\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:28:18.599170 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.599038 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-online-tls\") pod \"feast-simple-feast-setup-b6bf84db7-6msnc\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:28:18.699891 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.699865 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-registry-tls\") pod \"feast-simple-feast-setup-b6bf84db7-6msnc\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:28:18.700046 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.699895 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-online-tls\") pod \"feast-simple-feast-setup-b6bf84db7-6msnc\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:28:18.700046 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.699934 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-ui-tls\") pod \"feast-simple-feast-setup-b6bf84db7-6msnc\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:28:18.700046 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.699972 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-offline-tls\") pod \"feast-simple-feast-setup-b6bf84db7-6msnc\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:28:18.700046 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.699985 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ef115454-197f-4272-b3d7-148e48cabcfd-feast-data\") pod \"feast-simple-feast-setup-b6bf84db7-6msnc\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:28:18.700046 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.700003 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcs9m\" (UniqueName: \"kubernetes.io/projected/ef115454-197f-4272-b3d7-148e48cabcfd-kube-api-access-wcs9m\") pod \"feast-simple-feast-setup-b6bf84db7-6msnc\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:28:18.700294 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:28:18.700120 2563 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Sep 17 07:28:18.700294 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:28:18.700209 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-ui-tls podName:ef115454-197f-4272-b3d7-148e48cabcfd nodeName:}" failed. No retries permitted until 2026-09-17 07:28:19.200189123 +0000 UTC m=+506.385922081 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-ui-tls") pod "feast-simple-feast-setup-b6bf84db7-6msnc" (UID: "ef115454-197f-4272-b3d7-148e48cabcfd") : secret "feast-simple-feast-setup-ui-tls" not found Sep 17 07:28:18.700460 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.700414 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ef115454-197f-4272-b3d7-148e48cabcfd-feast-data\") pod \"feast-simple-feast-setup-b6bf84db7-6msnc\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:28:18.702580 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.702562 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-online-tls\") pod \"feast-simple-feast-setup-b6bf84db7-6msnc\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:28:18.702709 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.702689 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-registry-tls\") pod \"feast-simple-feast-setup-b6bf84db7-6msnc\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:28:18.702751 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.702717 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-offline-tls\") pod \"feast-simple-feast-setup-b6bf84db7-6msnc\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:28:18.716100 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:18.716075 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcs9m\" (UniqueName: \"kubernetes.io/projected/ef115454-197f-4272-b3d7-148e48cabcfd-kube-api-access-wcs9m\") pod \"feast-simple-feast-setup-b6bf84db7-6msnc\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:28:19.204571 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:19.204532 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-ui-tls\") pod \"feast-simple-feast-setup-b6bf84db7-6msnc\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:28:19.207104 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:19.207080 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-ui-tls\") pod \"feast-simple-feast-setup-b6bf84db7-6msnc\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:28:19.429513 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:19.429470 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:28:19.772043 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:19.771980 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc"] Sep 17 07:28:19.775132 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:28:19.775108 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef115454_197f_4272_b3d7_148e48cabcfd.slice/crio-f80beca34258697fc69f59621fe4c8bfa945e7063a870238f26a390dab9466bd WatchSource:0}: Error finding container f80beca34258697fc69f59621fe4c8bfa945e7063a870238f26a390dab9466bd: Status 404 returned error can't find the container with id f80beca34258697fc69f59621fe4c8bfa945e7063a870238f26a390dab9466bd Sep 17 07:28:20.470256 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:20.470220 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" event={"ID":"ef115454-197f-4272-b3d7-148e48cabcfd","Type":"ContainerStarted","Data":"f80beca34258697fc69f59621fe4c8bfa945e7063a870238f26a390dab9466bd"} Sep 17 07:28:52.609182 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:52.609140 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" event={"ID":"ef115454-197f-4272-b3d7-148e48cabcfd","Type":"ContainerStarted","Data":"8b0e0ada1719bc8342373a50e6e6205139d6d93151cafd167307bf6c0f401c4b"} Sep 17 07:28:56.626649 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:56.626566 2563 generic.go:356] "Generic (PLEG): container finished" podID="ef115454-197f-4272-b3d7-148e48cabcfd" containerID="8b0e0ada1719bc8342373a50e6e6205139d6d93151cafd167307bf6c0f401c4b" exitCode=0 Sep 17 07:28:56.626649 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:56.626639 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" event={"ID":"ef115454-197f-4272-b3d7-148e48cabcfd","Type":"ContainerDied","Data":"8b0e0ada1719bc8342373a50e6e6205139d6d93151cafd167307bf6c0f401c4b"} Sep 17 07:28:57.632799 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:28:57.632764 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" event={"ID":"ef115454-197f-4272-b3d7-148e48cabcfd","Type":"ContainerStarted","Data":"5019bd6b0b700308b49ca5a7754f65cb89837849e63e6dcb360dc41cb44377b3"} Sep 17 07:29:02.656674 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:02.656595 2563 generic.go:356] "Generic (PLEG): container finished" podID="ef115454-197f-4272-b3d7-148e48cabcfd" containerID="5019bd6b0b700308b49ca5a7754f65cb89837849e63e6dcb360dc41cb44377b3" exitCode=0 Sep 17 07:29:02.656674 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:02.656654 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" event={"ID":"ef115454-197f-4272-b3d7-148e48cabcfd","Type":"ContainerDied","Data":"5019bd6b0b700308b49ca5a7754f65cb89837849e63e6dcb360dc41cb44377b3"} Sep 17 07:29:02.666219 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:02.666200 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Sep 17 07:29:03.667803 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:03.667757 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" event={"ID":"ef115454-197f-4272-b3d7-148e48cabcfd","Type":"ContainerStarted","Data":"ef55f33e57842eb4c39a67f4948efe57319bc3641e1a5071800ad6480c19811a"} Sep 17 07:29:03.668272 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:03.667830 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" event={"ID":"ef115454-197f-4272-b3d7-148e48cabcfd","Type":"ContainerStarted","Data":"cff43efdd83e0c7a2bd384e15519fa482d96116da4c01fb1750b2d0e6bc7fe0e"} Sep 17 07:29:03.668272 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:03.667851 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" event={"ID":"ef115454-197f-4272-b3d7-148e48cabcfd","Type":"ContainerStarted","Data":"b0149255b08e757ec4be12425417fe316c5d207d475eda4e258f1458822dadb0"} Sep 17 07:29:03.668272 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:03.667869 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" event={"ID":"ef115454-197f-4272-b3d7-148e48cabcfd","Type":"ContainerStarted","Data":"15b5040bc50ad797be935596a114b15b503e6572f17c704f836b2faa048819b1"} Sep 17 07:29:03.700776 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:03.700703 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" podStartSLOduration=13.913139373 podStartE2EDuration="45.70068182s" totalImagesPullingTime="31.787542447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:28:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:29:03.696476491 +0000 UTC m=+550.882209458" watchObservedRunningTime="2026-09-17 07:29:03.70068182 +0000 UTC m=+550.886414793" Sep 17 07:29:04.430549 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:04.430513 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:29:04.430822 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:04.430799 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:29:04.430822 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:04.430827 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:29:04.431004 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:04.430845 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:29:04.433023 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:04.432983 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.28:6571: connect: connection refused" Sep 17 07:29:04.433155 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:04.433018 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.28:8816: connect: connection refused" Sep 17 07:29:04.433155 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:04.433038 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.28:8443: connect: connection refused" Sep 17 07:29:04.433338 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:04.433288 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused" Sep 17 07:29:07.429869 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:07.429819 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.28:6571: connect: connection refused" Sep 17 07:29:07.430544 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:07.429819 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.28:8443: connect: connection refused" Sep 17 07:29:07.430653 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:07.429835 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.28:8816: connect: connection refused" Sep 17 07:29:07.430653 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:07.430068 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused" Sep 17 07:29:09.429961 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:09.429919 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:29:09.430455 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:09.429989 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:29:09.430455 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:09.430010 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:29:09.430663 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:09.430641 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:29:10.430345 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:10.430288 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.28:8443: connect: connection refused" Sep 17 07:29:10.431204 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:10.431182 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:29:10.431428 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:10.431400 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:29:10.436162 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:10.436144 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:29:10.710087 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:10.710066 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:29:10.711496 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:10.711477 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:29:10.712529 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:10.712516 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:29:13.431007 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:13.430978 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:29:13.722494 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:29:13.722467 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:30:16.570345 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:16.570299 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc"] Sep 17 07:30:16.570930 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:16.570574 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="registry" containerID="cri-o://15b5040bc50ad797be935596a114b15b503e6572f17c704f836b2faa048819b1" gracePeriod=30 Sep 17 07:30:16.570930 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:16.570637 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="ui" containerID="cri-o://ef55f33e57842eb4c39a67f4948efe57319bc3641e1a5071800ad6480c19811a" gracePeriod=30 Sep 17 07:30:16.570930 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:16.570655 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="offline" containerID="cri-o://cff43efdd83e0c7a2bd384e15519fa482d96116da4c01fb1750b2d0e6bc7fe0e" gracePeriod=30 Sep 17 07:30:16.570930 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:16.570720 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="online" containerID="cri-o://b0149255b08e757ec4be12425417fe316c5d207d475eda4e258f1458822dadb0" gracePeriod=30 Sep 17 07:30:17.125466 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.125424 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl"] Sep 17 07:30:17.130181 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.130158 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:17.136296 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.136274 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-d6468\"" Sep 17 07:30:17.157753 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.157718 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl"] Sep 17 07:30:17.162802 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.162770 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfkbz\" (UniqueName: \"kubernetes.io/projected/62e673eb-a4f3-452a-95e3-5f533820c351-kube-api-access-lfkbz\") pod \"feast-simple-feast-setup-b6bf84db7-rdkbl\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:17.162929 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.162826 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-offline-tls\") pod \"feast-simple-feast-setup-b6bf84db7-rdkbl\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:17.162929 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.162865 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/62e673eb-a4f3-452a-95e3-5f533820c351-feast-data\") pod \"feast-simple-feast-setup-b6bf84db7-rdkbl\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:17.163041 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.162931 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-online-tls\") pod \"feast-simple-feast-setup-b6bf84db7-rdkbl\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:17.163041 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.162981 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-registry-tls\") pod \"feast-simple-feast-setup-b6bf84db7-rdkbl\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:17.163041 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.163014 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-ui-tls\") pod \"feast-simple-feast-setup-b6bf84db7-rdkbl\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:17.264371 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.264317 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-registry-tls\") pod \"feast-simple-feast-setup-b6bf84db7-rdkbl\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:17.264544 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.264404 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-ui-tls\") pod \"feast-simple-feast-setup-b6bf84db7-rdkbl\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:17.264544 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:30:17.264436 2563 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Sep 17 07:30:17.264544 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.264464 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfkbz\" (UniqueName: \"kubernetes.io/projected/62e673eb-a4f3-452a-95e3-5f533820c351-kube-api-access-lfkbz\") pod \"feast-simple-feast-setup-b6bf84db7-rdkbl\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:17.264544 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:30:17.264498 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-registry-tls podName:62e673eb-a4f3-452a-95e3-5f533820c351 nodeName:}" failed. No retries permitted until 2026-09-17 07:30:17.764480299 +0000 UTC m=+624.950213268 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-registry-tls") pod "feast-simple-feast-setup-b6bf84db7-rdkbl" (UID: "62e673eb-a4f3-452a-95e3-5f533820c351") : secret "feast-simple-feast-setup-registry-tls" not found Sep 17 07:30:17.264544 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.264516 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-offline-tls\") pod \"feast-simple-feast-setup-b6bf84db7-rdkbl\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:17.265104 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.264562 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/62e673eb-a4f3-452a-95e3-5f533820c351-feast-data\") pod \"feast-simple-feast-setup-b6bf84db7-rdkbl\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:17.265104 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.264608 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-online-tls\") pod \"feast-simple-feast-setup-b6bf84db7-rdkbl\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:17.265104 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:30:17.264958 2563 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Sep 17 07:30:17.265104 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:30:17.265018 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-offline-tls podName:62e673eb-a4f3-452a-95e3-5f533820c351 nodeName:}" failed. No retries permitted until 2026-09-17 07:30:17.76500176 +0000 UTC m=+624.950734710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-offline-tls") pod "feast-simple-feast-setup-b6bf84db7-rdkbl" (UID: "62e673eb-a4f3-452a-95e3-5f533820c351") : secret "feast-simple-feast-setup-offline-tls" not found Sep 17 07:30:17.265429 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.265406 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/62e673eb-a4f3-452a-95e3-5f533820c351-feast-data\") pod \"feast-simple-feast-setup-b6bf84db7-rdkbl\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:17.267930 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.267905 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-online-tls\") pod \"feast-simple-feast-setup-b6bf84db7-rdkbl\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:17.268051 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.267905 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-ui-tls\") pod \"feast-simple-feast-setup-b6bf84db7-rdkbl\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:17.287528 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.287503 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfkbz\" (UniqueName: \"kubernetes.io/projected/62e673eb-a4f3-452a-95e3-5f533820c351-kube-api-access-lfkbz\") pod \"feast-simple-feast-setup-b6bf84db7-rdkbl\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:17.767722 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.767695 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-offline-tls\") pod \"feast-simple-feast-setup-b6bf84db7-rdkbl\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:17.768071 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.767807 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-registry-tls\") pod \"feast-simple-feast-setup-b6bf84db7-rdkbl\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:17.770243 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.770219 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-registry-tls\") pod \"feast-simple-feast-setup-b6bf84db7-rdkbl\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:17.770396 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.770304 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-offline-tls\") pod \"feast-simple-feast-setup-b6bf84db7-rdkbl\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:17.988169 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.988138 2563 generic.go:356] "Generic (PLEG): container finished" podID="ef115454-197f-4272-b3d7-148e48cabcfd" containerID="ef55f33e57842eb4c39a67f4948efe57319bc3641e1a5071800ad6480c19811a" exitCode=0 Sep 17 07:30:17.988169 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.988162 2563 generic.go:356] "Generic (PLEG): container finished" podID="ef115454-197f-4272-b3d7-148e48cabcfd" containerID="b0149255b08e757ec4be12425417fe316c5d207d475eda4e258f1458822dadb0" exitCode=0 Sep 17 07:30:17.988363 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.988182 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" event={"ID":"ef115454-197f-4272-b3d7-148e48cabcfd","Type":"ContainerDied","Data":"ef55f33e57842eb4c39a67f4948efe57319bc3641e1a5071800ad6480c19811a"} Sep 17 07:30:17.988363 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:17.988211 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" event={"ID":"ef115454-197f-4272-b3d7-148e48cabcfd","Type":"ContainerDied","Data":"b0149255b08e757ec4be12425417fe316c5d207d475eda4e258f1458822dadb0"} Sep 17 07:30:18.043409 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:18.043315 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:18.325392 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:18.325370 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl"] Sep 17 07:30:18.327293 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:30:18.327267 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod62e673eb_a4f3_452a_95e3_5f533820c351.slice/crio-a88b498860c5999e5f0be579b6e8b6e4d283de3eef10b9fb7a6f7822ee6612b9 WatchSource:0}: Error finding container a88b498860c5999e5f0be579b6e8b6e4d283de3eef10b9fb7a6f7822ee6612b9: Status 404 returned error can't find the container with id a88b498860c5999e5f0be579b6e8b6e4d283de3eef10b9fb7a6f7822ee6612b9 Sep 17 07:30:18.994162 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:18.994129 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" event={"ID":"62e673eb-a4f3-452a-95e3-5f533820c351","Type":"ContainerStarted","Data":"a8ad2687e8ccf22f4f9be646a36d1486b8b12a07361cb6d52a165105d7efbaeb"} Sep 17 07:30:18.994620 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:18.994178 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" event={"ID":"62e673eb-a4f3-452a-95e3-5f533820c351","Type":"ContainerStarted","Data":"a88b498860c5999e5f0be579b6e8b6e4d283de3eef10b9fb7a6f7822ee6612b9"} Sep 17 07:30:20.710471 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:20.710412 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused" Sep 17 07:30:23.722274 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:23.722232 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.28:8443: connect: connection refused" Sep 17 07:30:24.018102 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:24.018012 2563 generic.go:356] "Generic (PLEG): container finished" podID="62e673eb-a4f3-452a-95e3-5f533820c351" containerID="a8ad2687e8ccf22f4f9be646a36d1486b8b12a07361cb6d52a165105d7efbaeb" exitCode=0 Sep 17 07:30:24.018244 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:24.018091 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" event={"ID":"62e673eb-a4f3-452a-95e3-5f533820c351","Type":"ContainerDied","Data":"a8ad2687e8ccf22f4f9be646a36d1486b8b12a07361cb6d52a165105d7efbaeb"} Sep 17 07:30:25.024136 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:25.024101 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" event={"ID":"62e673eb-a4f3-452a-95e3-5f533820c351","Type":"ContainerStarted","Data":"c4f120466c3b52b666ff87a121164e2bacfc1a09cb126f681c62b494059432ef"} Sep 17 07:30:30.045479 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:30.045442 2563 generic.go:356] "Generic (PLEG): container finished" podID="62e673eb-a4f3-452a-95e3-5f533820c351" containerID="c4f120466c3b52b666ff87a121164e2bacfc1a09cb126f681c62b494059432ef" exitCode=0 Sep 17 07:30:30.045843 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:30.045518 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" event={"ID":"62e673eb-a4f3-452a-95e3-5f533820c351","Type":"ContainerDied","Data":"c4f120466c3b52b666ff87a121164e2bacfc1a09cb126f681c62b494059432ef"} Sep 17 07:30:30.710009 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:30.709964 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused" Sep 17 07:30:31.056217 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:31.056124 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" event={"ID":"62e673eb-a4f3-452a-95e3-5f533820c351","Type":"ContainerStarted","Data":"f8f0c08d7d1e28683ece928fd2482ba29fe7c8a4f7b9083a6b5ae8e1fcd0e9f0"} Sep 17 07:30:31.056217 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:31.056189 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" event={"ID":"62e673eb-a4f3-452a-95e3-5f533820c351","Type":"ContainerStarted","Data":"6ba19a099547d1ab4fb802459df3b3a6dfeb5df8ef428b05fab2d177dc5777a6"} Sep 17 07:30:31.056217 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:31.056210 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" event={"ID":"62e673eb-a4f3-452a-95e3-5f533820c351","Type":"ContainerStarted","Data":"26d007bc3753b1697fca04d47a4cd881964bb231464d1b944b002ded010e90b0"} Sep 17 07:30:31.056805 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:31.056228 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" event={"ID":"62e673eb-a4f3-452a-95e3-5f533820c351","Type":"ContainerStarted","Data":"58be9e70a901b28d3e3a13b3afcc2bab794034c37cfe74667b0ad587d4ef8b18"} Sep 17 07:30:31.085559 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:31.085490 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" podStartSLOduration=14.085468075 podStartE2EDuration="14.085468075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:30:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:30:31.084386932 +0000 UTC m=+638.270119901" watchObservedRunningTime="2026-09-17 07:30:31.085468075 +0000 UTC m=+638.271201048" Sep 17 07:30:33.044651 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:33.044517 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:33.044651 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:33.044579 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:33.044651 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:33.044601 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:33.044651 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:33.044622 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:33.046862 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:33.046822 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.29:6571: connect: connection refused" Sep 17 07:30:33.047013 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:33.046848 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Sep 17 07:30:33.047013 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:33.046831 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused" Sep 17 07:30:33.047156 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:33.047109 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.29:8816: connect: connection refused" Sep 17 07:30:33.722650 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:33.722605 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.28:8443: connect: connection refused" Sep 17 07:30:36.044416 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:36.044373 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.29:6571: connect: connection refused" Sep 17 07:30:36.044894 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:36.044408 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused" Sep 17 07:30:36.044894 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:36.044415 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Sep 17 07:30:36.044894 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:36.044859 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.29:8816: connect: connection refused" Sep 17 07:30:38.044146 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:38.044105 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:38.044601 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:38.044158 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:38.044601 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:38.044173 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:38.044601 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:38.044183 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:39.044496 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:39.044467 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:39.044851 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:39.044628 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:39.045001 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:39.044979 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:39.045498 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:39.045472 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:39.045669 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:39.045543 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:39.049954 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:39.049912 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:39.102053 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:39.102026 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:39.110758 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:39.110737 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:30:40.710005 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:40.709967 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused" Sep 17 07:30:40.710441 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:40.710143 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:30:43.722622 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:43.722583 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.28:8443: connect: connection refused" Sep 17 07:30:43.723013 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:43.722709 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:30:47.138273 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:47.138237 2563 generic.go:356] "Generic (PLEG): container finished" podID="ef115454-197f-4272-b3d7-148e48cabcfd" containerID="cff43efdd83e0c7a2bd384e15519fa482d96116da4c01fb1750b2d0e6bc7fe0e" exitCode=137 Sep 17 07:30:47.138273 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:47.138269 2563 generic.go:356] "Generic (PLEG): container finished" podID="ef115454-197f-4272-b3d7-148e48cabcfd" containerID="15b5040bc50ad797be935596a114b15b503e6572f17c704f836b2faa048819b1" exitCode=137 Sep 17 07:30:47.138687 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:47.138306 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" event={"ID":"ef115454-197f-4272-b3d7-148e48cabcfd","Type":"ContainerDied","Data":"cff43efdd83e0c7a2bd384e15519fa482d96116da4c01fb1750b2d0e6bc7fe0e"} Sep 17 07:30:47.138687 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:47.138383 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" event={"ID":"ef115454-197f-4272-b3d7-148e48cabcfd","Type":"ContainerDied","Data":"15b5040bc50ad797be935596a114b15b503e6572f17c704f836b2faa048819b1"} Sep 17 07:30:47.227284 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:47.227257 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:30:47.319750 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:47.319725 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-offline-tls\") pod \"ef115454-197f-4272-b3d7-148e48cabcfd\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " Sep 17 07:30:47.319895 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:47.319760 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wcs9m\" (UniqueName: \"kubernetes.io/projected/ef115454-197f-4272-b3d7-148e48cabcfd-kube-api-access-wcs9m\") pod \"ef115454-197f-4272-b3d7-148e48cabcfd\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " Sep 17 07:30:47.319895 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:47.319826 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ef115454-197f-4272-b3d7-148e48cabcfd-feast-data\") pod \"ef115454-197f-4272-b3d7-148e48cabcfd\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " Sep 17 07:30:47.319895 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:47.319844 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-online-tls\") pod \"ef115454-197f-4272-b3d7-148e48cabcfd\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " Sep 17 07:30:47.319895 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:47.319882 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-registry-tls\") pod \"ef115454-197f-4272-b3d7-148e48cabcfd\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " Sep 17 07:30:47.320102 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:47.319924 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-ui-tls\") pod \"ef115454-197f-4272-b3d7-148e48cabcfd\" (UID: \"ef115454-197f-4272-b3d7-148e48cabcfd\") " Sep 17 07:30:47.320639 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:47.320422 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef115454-197f-4272-b3d7-148e48cabcfd-feast-data" (OuterVolumeSpecName: "feast-data") pod "ef115454-197f-4272-b3d7-148e48cabcfd" (UID: "ef115454-197f-4272-b3d7-148e48cabcfd"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:30:47.322103 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:47.322078 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "ef115454-197f-4272-b3d7-148e48cabcfd" (UID: "ef115454-197f-4272-b3d7-148e48cabcfd"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:30:47.322209 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:47.322113 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ef115454-197f-4272-b3d7-148e48cabcfd" (UID: "ef115454-197f-4272-b3d7-148e48cabcfd"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:30:47.322209 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:47.322175 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-online-tls" (OuterVolumeSpecName: "online-tls") pod "ef115454-197f-4272-b3d7-148e48cabcfd" (UID: "ef115454-197f-4272-b3d7-148e48cabcfd"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:30:47.322562 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:47.322536 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef115454-197f-4272-b3d7-148e48cabcfd-kube-api-access-wcs9m" (OuterVolumeSpecName: "kube-api-access-wcs9m") pod "ef115454-197f-4272-b3d7-148e48cabcfd" (UID: "ef115454-197f-4272-b3d7-148e48cabcfd"). InnerVolumeSpecName "kube-api-access-wcs9m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:30:47.322651 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:47.322585 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "ef115454-197f-4272-b3d7-148e48cabcfd" (UID: "ef115454-197f-4272-b3d7-148e48cabcfd"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:30:47.420831 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:47.420775 2563 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-offline-tls\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:30:47.420831 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:47.420797 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcs9m\" (UniqueName: \"kubernetes.io/projected/ef115454-197f-4272-b3d7-148e48cabcfd-kube-api-access-wcs9m\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:30:47.420831 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:47.420807 2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ef115454-197f-4272-b3d7-148e48cabcfd-feast-data\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:30:47.420831 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:47.420833 2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-online-tls\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:30:47.421017 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:47.420841 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-registry-tls\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:30:47.421017 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:47.420849 2563 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ef115454-197f-4272-b3d7-148e48cabcfd-ui-tls\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:30:48.146365 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:48.146318 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" Sep 17 07:30:48.146365 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:48.146358 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc" event={"ID":"ef115454-197f-4272-b3d7-148e48cabcfd","Type":"ContainerDied","Data":"f80beca34258697fc69f59621fe4c8bfa945e7063a870238f26a390dab9466bd"} Sep 17 07:30:48.146902 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:48.146414 2563 scope.go:117] "RemoveContainer" containerID="ef55f33e57842eb4c39a67f4948efe57319bc3641e1a5071800ad6480c19811a" Sep 17 07:30:48.155900 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:48.155885 2563 scope.go:117] "RemoveContainer" containerID="cff43efdd83e0c7a2bd384e15519fa482d96116da4c01fb1750b2d0e6bc7fe0e" Sep 17 07:30:48.163553 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:48.163533 2563 scope.go:117] "RemoveContainer" containerID="b0149255b08e757ec4be12425417fe316c5d207d475eda4e258f1458822dadb0" Sep 17 07:30:48.171089 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:48.171070 2563 scope.go:117] "RemoveContainer" containerID="15b5040bc50ad797be935596a114b15b503e6572f17c704f836b2faa048819b1" Sep 17 07:30:48.174148 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:48.174126 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc"] Sep 17 07:30:48.178524 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:48.178505 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-b6bf84db7-6msnc"] Sep 17 07:30:48.179759 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:48.179734 2563 scope.go:117] "RemoveContainer" containerID="5019bd6b0b700308b49ca5a7754f65cb89837849e63e6dcb360dc41cb44377b3" Sep 17 07:30:48.186790 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:48.186774 2563 scope.go:117] "RemoveContainer" containerID="8b0e0ada1719bc8342373a50e6e6205139d6d93151cafd167307bf6c0f401c4b" Sep 17 07:30:49.314439 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:30:49.314406 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" path="/var/lib/kubelet/pods/ef115454-197f-4272-b3d7-148e48cabcfd/volumes" Sep 17 07:31:42.105275 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.105238 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8"] Sep 17 07:31:42.106021 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.105743 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="feast-init" Sep 17 07:31:42.106021 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.105759 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="feast-init" Sep 17 07:31:42.106021 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.105780 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="ui" Sep 17 07:31:42.106021 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.105788 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="ui" Sep 17 07:31:42.106021 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.105807 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="feast-apply" Sep 17 07:31:42.106021 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.105816 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="feast-apply" Sep 17 07:31:42.106021 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.105826 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="registry" Sep 17 07:31:42.106021 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.105836 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="registry" Sep 17 07:31:42.106021 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.105851 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="online" Sep 17 07:31:42.106021 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.105859 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="online" Sep 17 07:31:42.106021 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.105868 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="offline" Sep 17 07:31:42.106021 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.105876 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="offline" Sep 17 07:31:42.106021 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.105942 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="registry" Sep 17 07:31:42.106021 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.105953 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="online" Sep 17 07:31:42.106021 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.105967 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="offline" Sep 17 07:31:42.106021 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.105975 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef115454-197f-4272-b3d7-148e48cabcfd" containerName="ui" Sep 17 07:31:42.109994 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.109973 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:31:42.113782 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.113759 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Sep 17 07:31:42.113875 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.113845 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Sep 17 07:31:42.113914 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.113876 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Sep 17 07:31:42.114203 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.114178 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-982w8\"" Sep 17 07:31:42.114303 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.114184 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Sep 17 07:31:42.114392 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.114200 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Sep 17 07:31:42.114392 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.114228 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Sep 17 07:31:42.114684 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.114666 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Sep 17 07:31:42.127780 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.127753 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8"] Sep 17 07:31:42.264699 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.264670 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rftk2\" (UniqueName: \"kubernetes.io/projected/365c344c-05a4-475c-90ff-45acc03e85e1-kube-api-access-rftk2\") pod \"feast-simple-feast-remote-setup-6648699555-2tld8\" (UID: \"365c344c-05a4-475c-90ff-45acc03e85e1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:31:42.264834 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.264709 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/365c344c-05a4-475c-90ff-45acc03e85e1-ui-tls\") pod \"feast-simple-feast-remote-setup-6648699555-2tld8\" (UID: \"365c344c-05a4-475c-90ff-45acc03e85e1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:31:42.264834 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.264730 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/365c344c-05a4-475c-90ff-45acc03e85e1-online-tls\") pod \"feast-simple-feast-remote-setup-6648699555-2tld8\" (UID: \"365c344c-05a4-475c-90ff-45acc03e85e1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:31:42.264834 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.264796 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/365c344c-05a4-475c-90ff-45acc03e85e1-offline-tls\") pod \"feast-simple-feast-remote-setup-6648699555-2tld8\" (UID: \"365c344c-05a4-475c-90ff-45acc03e85e1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:31:42.264834 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.264819 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/365c344c-05a4-475c-90ff-45acc03e85e1-feast-data\") pod \"feast-simple-feast-remote-setup-6648699555-2tld8\" (UID: \"365c344c-05a4-475c-90ff-45acc03e85e1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:31:42.264980 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.264854 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/365c344c-05a4-475c-90ff-45acc03e85e1-registry-tls\") pod \"feast-simple-feast-remote-setup-6648699555-2tld8\" (UID: \"365c344c-05a4-475c-90ff-45acc03e85e1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:31:42.365267 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.365199 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/365c344c-05a4-475c-90ff-45acc03e85e1-offline-tls\") pod \"feast-simple-feast-remote-setup-6648699555-2tld8\" (UID: \"365c344c-05a4-475c-90ff-45acc03e85e1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:31:42.365267 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.365236 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/365c344c-05a4-475c-90ff-45acc03e85e1-feast-data\") pod \"feast-simple-feast-remote-setup-6648699555-2tld8\" (UID: \"365c344c-05a4-475c-90ff-45acc03e85e1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:31:42.365476 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.365272 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/365c344c-05a4-475c-90ff-45acc03e85e1-registry-tls\") pod \"feast-simple-feast-remote-setup-6648699555-2tld8\" (UID: \"365c344c-05a4-475c-90ff-45acc03e85e1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:31:42.365476 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.365312 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rftk2\" (UniqueName: \"kubernetes.io/projected/365c344c-05a4-475c-90ff-45acc03e85e1-kube-api-access-rftk2\") pod \"feast-simple-feast-remote-setup-6648699555-2tld8\" (UID: \"365c344c-05a4-475c-90ff-45acc03e85e1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:31:42.365476 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.365382 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/365c344c-05a4-475c-90ff-45acc03e85e1-ui-tls\") pod \"feast-simple-feast-remote-setup-6648699555-2tld8\" (UID: \"365c344c-05a4-475c-90ff-45acc03e85e1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:31:42.365476 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.365413 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/365c344c-05a4-475c-90ff-45acc03e85e1-online-tls\") pod \"feast-simple-feast-remote-setup-6648699555-2tld8\" (UID: \"365c344c-05a4-475c-90ff-45acc03e85e1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:31:42.365760 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.365728 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/365c344c-05a4-475c-90ff-45acc03e85e1-feast-data\") pod \"feast-simple-feast-remote-setup-6648699555-2tld8\" (UID: \"365c344c-05a4-475c-90ff-45acc03e85e1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:31:42.366313 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.366286 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/365c344c-05a4-475c-90ff-45acc03e85e1-registry-tls\") pod \"feast-simple-feast-remote-setup-6648699555-2tld8\" (UID: \"365c344c-05a4-475c-90ff-45acc03e85e1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:31:42.367942 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.367917 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/365c344c-05a4-475c-90ff-45acc03e85e1-offline-tls\") pod \"feast-simple-feast-remote-setup-6648699555-2tld8\" (UID: \"365c344c-05a4-475c-90ff-45acc03e85e1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:31:42.368025 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.368001 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/365c344c-05a4-475c-90ff-45acc03e85e1-online-tls\") pod \"feast-simple-feast-remote-setup-6648699555-2tld8\" (UID: \"365c344c-05a4-475c-90ff-45acc03e85e1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:31:42.368200 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.368177 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/365c344c-05a4-475c-90ff-45acc03e85e1-ui-tls\") pod \"feast-simple-feast-remote-setup-6648699555-2tld8\" (UID: \"365c344c-05a4-475c-90ff-45acc03e85e1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:31:42.377981 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.377963 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rftk2\" (UniqueName: \"kubernetes.io/projected/365c344c-05a4-475c-90ff-45acc03e85e1-kube-api-access-rftk2\") pod \"feast-simple-feast-remote-setup-6648699555-2tld8\" (UID: \"365c344c-05a4-475c-90ff-45acc03e85e1\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:31:42.419841 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.419814 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:31:42.706842 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:42.706814 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8"] Sep 17 07:31:42.708575 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:31:42.708544 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod365c344c_05a4_475c_90ff_45acc03e85e1.slice/crio-76da82a4f315389fd32d65f8dad9a1a2dc0480e920f631f8076c18e78eda31d0 WatchSource:0}: Error finding container 76da82a4f315389fd32d65f8dad9a1a2dc0480e920f631f8076c18e78eda31d0: Status 404 returned error can't find the container with id 76da82a4f315389fd32d65f8dad9a1a2dc0480e920f631f8076c18e78eda31d0 Sep 17 07:31:43.375021 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:43.374987 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" event={"ID":"365c344c-05a4-475c-90ff-45acc03e85e1","Type":"ContainerStarted","Data":"2a14a223c90e920d1cda7e45945a501779f7bf0276ed103558c1bd21b2ac0582"} Sep 17 07:31:43.375507 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:43.375037 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" event={"ID":"365c344c-05a4-475c-90ff-45acc03e85e1","Type":"ContainerStarted","Data":"76da82a4f315389fd32d65f8dad9a1a2dc0480e920f631f8076c18e78eda31d0"} Sep 17 07:31:48.397001 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:48.396967 2563 generic.go:356] "Generic (PLEG): container finished" podID="365c344c-05a4-475c-90ff-45acc03e85e1" containerID="2a14a223c90e920d1cda7e45945a501779f7bf0276ed103558c1bd21b2ac0582" exitCode=0 Sep 17 07:31:48.397564 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:48.397044 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" event={"ID":"365c344c-05a4-475c-90ff-45acc03e85e1","Type":"ContainerDied","Data":"2a14a223c90e920d1cda7e45945a501779f7bf0276ed103558c1bd21b2ac0582"} Sep 17 07:31:49.408783 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:31:49.408744 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" event={"ID":"365c344c-05a4-475c-90ff-45acc03e85e1","Type":"ContainerStarted","Data":"4b3cf5a8a44746a17bb3946a5e557637cd1b8d029e835cbaf54691366d68f6ef"} Sep 17 07:32:14.516564 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:14.516532 2563 generic.go:356] "Generic (PLEG): container finished" podID="365c344c-05a4-475c-90ff-45acc03e85e1" containerID="4b3cf5a8a44746a17bb3946a5e557637cd1b8d029e835cbaf54691366d68f6ef" exitCode=0 Sep 17 07:32:14.516912 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:14.516577 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" event={"ID":"365c344c-05a4-475c-90ff-45acc03e85e1","Type":"ContainerDied","Data":"4b3cf5a8a44746a17bb3946a5e557637cd1b8d029e835cbaf54691366d68f6ef"} Sep 17 07:32:14.521830 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:14.521803 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Sep 17 07:32:15.525479 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:15.525438 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" event={"ID":"365c344c-05a4-475c-90ff-45acc03e85e1","Type":"ContainerStarted","Data":"3c4f5f225bc1fd796fa3e4d9a5f0650c6c1d642127d36f642f0b8079be5136b0"} Sep 17 07:32:15.525947 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:15.525497 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" event={"ID":"365c344c-05a4-475c-90ff-45acc03e85e1","Type":"ContainerStarted","Data":"15b92a9be5c4624cdda5f42ec54ac072001659347de77d7f81bf1ffc6569e142"} Sep 17 07:32:15.525947 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:15.525517 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" event={"ID":"365c344c-05a4-475c-90ff-45acc03e85e1","Type":"ContainerStarted","Data":"9272fcd2a17ce023f9f2426425bab3af9d20664be9d1bd818b9e299125106a0f"} Sep 17 07:32:15.559173 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:15.559115 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" podStartSLOduration=33.559099334 podStartE2EDuration="33.559099334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:31:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:32:15.556195224 +0000 UTC m=+742.741928208" watchObservedRunningTime="2026-09-17 07:32:15.559099334 +0000 UTC m=+742.744832305" Sep 17 07:32:18.420537 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:18.420493 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:32:18.420537 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:18.420547 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:32:18.421038 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:18.420599 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:32:18.422769 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:18.422727 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.30:8443: connect: connection refused" Sep 17 07:32:18.422901 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:18.422755 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.30:8816: connect: connection refused" Sep 17 07:32:18.422901 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:18.422801 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused" Sep 17 07:32:21.420672 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:21.420625 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.30:8816: connect: connection refused" Sep 17 07:32:21.421215 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:21.420625 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.30:8443: connect: connection refused" Sep 17 07:32:21.421215 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:21.420724 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused" Sep 17 07:32:22.420340 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:22.420279 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:32:22.420340 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:22.420349 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:32:22.420555 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:22.420362 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:32:24.421507 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:24.421471 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:32:24.421960 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:24.421621 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:32:24.422306 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:24.422285 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:32:24.422489 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:24.422470 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:32:24.449875 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:24.449848 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:32:24.595580 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:32:24.595551 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:33:26.400380 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:26.400271 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8"] Sep 17 07:33:26.400900 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:26.400581 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="online" containerID="cri-o://9272fcd2a17ce023f9f2426425bab3af9d20664be9d1bd818b9e299125106a0f" gracePeriod=30 Sep 17 07:33:26.400900 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:26.400659 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="ui" containerID="cri-o://3c4f5f225bc1fd796fa3e4d9a5f0650c6c1d642127d36f642f0b8079be5136b0" gracePeriod=30 Sep 17 07:33:26.400900 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:26.400675 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="offline" containerID="cri-o://15b92a9be5c4624cdda5f42ec54ac072001659347de77d7f81bf1ffc6569e142" gracePeriod=30 Sep 17 07:33:26.510476 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:26.510443 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl"] Sep 17 07:33:26.510798 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:26.510748 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="registry" containerID="cri-o://58be9e70a901b28d3e3a13b3afcc2bab794034c37cfe74667b0ad587d4ef8b18" gracePeriod=30 Sep 17 07:33:26.511027 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:26.510802 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="ui" containerID="cri-o://f8f0c08d7d1e28683ece928fd2482ba29fe7c8a4f7b9083a6b5ae8e1fcd0e9f0" gracePeriod=30 Sep 17 07:33:26.511124 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:26.510824 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="offline" containerID="cri-o://6ba19a099547d1ab4fb802459df3b3a6dfeb5df8ef428b05fab2d177dc5777a6" gracePeriod=30 Sep 17 07:33:26.511230 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:26.510851 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="online" containerID="cri-o://26d007bc3753b1697fca04d47a4cd881964bb231464d1b944b002ded010e90b0" gracePeriod=30 Sep 17 07:33:27.845766 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:27.845728 2563 generic.go:356] "Generic (PLEG): container finished" podID="62e673eb-a4f3-452a-95e3-5f533820c351" containerID="f8f0c08d7d1e28683ece928fd2482ba29fe7c8a4f7b9083a6b5ae8e1fcd0e9f0" exitCode=0 Sep 17 07:33:27.845766 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:27.845760 2563 generic.go:356] "Generic (PLEG): container finished" podID="62e673eb-a4f3-452a-95e3-5f533820c351" containerID="26d007bc3753b1697fca04d47a4cd881964bb231464d1b944b002ded010e90b0" exitCode=0 Sep 17 07:33:27.846214 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:27.845800 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" event={"ID":"62e673eb-a4f3-452a-95e3-5f533820c351","Type":"ContainerDied","Data":"f8f0c08d7d1e28683ece928fd2482ba29fe7c8a4f7b9083a6b5ae8e1fcd0e9f0"} Sep 17 07:33:27.846214 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:27.845853 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" event={"ID":"62e673eb-a4f3-452a-95e3-5f533820c351","Type":"ContainerDied","Data":"26d007bc3753b1697fca04d47a4cd881964bb231464d1b944b002ded010e90b0"} Sep 17 07:33:27.851551 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:27.851525 2563 generic.go:356] "Generic (PLEG): container finished" podID="365c344c-05a4-475c-90ff-45acc03e85e1" containerID="3c4f5f225bc1fd796fa3e4d9a5f0650c6c1d642127d36f642f0b8079be5136b0" exitCode=0 Sep 17 07:33:27.851551 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:27.851546 2563 generic.go:356] "Generic (PLEG): container finished" podID="365c344c-05a4-475c-90ff-45acc03e85e1" containerID="9272fcd2a17ce023f9f2426425bab3af9d20664be9d1bd818b9e299125106a0f" exitCode=0 Sep 17 07:33:27.851709 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:27.851567 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" event={"ID":"365c344c-05a4-475c-90ff-45acc03e85e1","Type":"ContainerDied","Data":"3c4f5f225bc1fd796fa3e4d9a5f0650c6c1d642127d36f642f0b8079be5136b0"} Sep 17 07:33:27.851709 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:27.851594 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" event={"ID":"365c344c-05a4-475c-90ff-45acc03e85e1","Type":"ContainerDied","Data":"9272fcd2a17ce023f9f2426425bab3af9d20664be9d1bd818b9e299125106a0f"} Sep 17 07:33:29.045336 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:29.045273 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused" Sep 17 07:33:29.102036 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:29.102003 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Sep 17 07:33:34.422179 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:34.422131 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.30:8443: connect: connection refused" Sep 17 07:33:34.571468 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:34.571421 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused" Sep 17 07:33:39.045680 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:39.045640 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused" Sep 17 07:33:39.102110 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:39.102078 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Sep 17 07:33:44.422461 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:44.422416 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.30:8443: connect: connection refused" Sep 17 07:33:44.571533 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:44.571489 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused" Sep 17 07:33:49.045157 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:49.045118 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused" Sep 17 07:33:49.045595 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:49.045256 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:33:49.102481 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:49.102449 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Sep 17 07:33:49.102645 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:49.102557 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:33:54.423047 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:54.422991 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.30:8443: connect: connection refused" Sep 17 07:33:54.423540 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:54.423147 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:33:54.571517 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:54.571483 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="online" probeResult="failure" output="Get \"https://10.133.0.30:6567/health\": dial tcp 10.133.0.30:6567: connect: connection refused" Sep 17 07:33:54.571671 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:54.571584 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:33:56.980861 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:56.980834 2563 generic.go:356] "Generic (PLEG): container finished" podID="62e673eb-a4f3-452a-95e3-5f533820c351" containerID="6ba19a099547d1ab4fb802459df3b3a6dfeb5df8ef428b05fab2d177dc5777a6" exitCode=137 Sep 17 07:33:56.980861 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:56.980857 2563 generic.go:356] "Generic (PLEG): container finished" podID="62e673eb-a4f3-452a-95e3-5f533820c351" containerID="58be9e70a901b28d3e3a13b3afcc2bab794034c37cfe74667b0ad587d4ef8b18" exitCode=137 Sep 17 07:33:56.981312 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:56.980924 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" event={"ID":"62e673eb-a4f3-452a-95e3-5f533820c351","Type":"ContainerDied","Data":"6ba19a099547d1ab4fb802459df3b3a6dfeb5df8ef428b05fab2d177dc5777a6"} Sep 17 07:33:56.981312 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:56.980972 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" event={"ID":"62e673eb-a4f3-452a-95e3-5f533820c351","Type":"ContainerDied","Data":"58be9e70a901b28d3e3a13b3afcc2bab794034c37cfe74667b0ad587d4ef8b18"} Sep 17 07:33:56.984416 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:56.984390 2563 generic.go:356] "Generic (PLEG): container finished" podID="365c344c-05a4-475c-90ff-45acc03e85e1" containerID="15b92a9be5c4624cdda5f42ec54ac072001659347de77d7f81bf1ffc6569e142" exitCode=137 Sep 17 07:33:56.984528 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:56.984476 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" event={"ID":"365c344c-05a4-475c-90ff-45acc03e85e1","Type":"ContainerDied","Data":"15b92a9be5c4624cdda5f42ec54ac072001659347de77d7f81bf1ffc6569e142"} Sep 17 07:33:57.053079 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.053058 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:33:57.148177 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.148153 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/365c344c-05a4-475c-90ff-45acc03e85e1-registry-tls\") pod \"365c344c-05a4-475c-90ff-45acc03e85e1\" (UID: \"365c344c-05a4-475c-90ff-45acc03e85e1\") " Sep 17 07:33:57.148367 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.148190 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/365c344c-05a4-475c-90ff-45acc03e85e1-online-tls\") pod \"365c344c-05a4-475c-90ff-45acc03e85e1\" (UID: \"365c344c-05a4-475c-90ff-45acc03e85e1\") " Sep 17 07:33:57.148367 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.148246 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/365c344c-05a4-475c-90ff-45acc03e85e1-ui-tls\") pod \"365c344c-05a4-475c-90ff-45acc03e85e1\" (UID: \"365c344c-05a4-475c-90ff-45acc03e85e1\") " Sep 17 07:33:57.148367 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.148275 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/365c344c-05a4-475c-90ff-45acc03e85e1-offline-tls\") pod \"365c344c-05a4-475c-90ff-45acc03e85e1\" (UID: \"365c344c-05a4-475c-90ff-45acc03e85e1\") " Sep 17 07:33:57.148367 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.148297 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/365c344c-05a4-475c-90ff-45acc03e85e1-feast-data\") pod \"365c344c-05a4-475c-90ff-45acc03e85e1\" (UID: \"365c344c-05a4-475c-90ff-45acc03e85e1\") " Sep 17 07:33:57.148367 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.148363 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rftk2\" (UniqueName: \"kubernetes.io/projected/365c344c-05a4-475c-90ff-45acc03e85e1-kube-api-access-rftk2\") pod \"365c344c-05a4-475c-90ff-45acc03e85e1\" (UID: \"365c344c-05a4-475c-90ff-45acc03e85e1\") " Sep 17 07:33:57.148635 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.148555 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/365c344c-05a4-475c-90ff-45acc03e85e1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "365c344c-05a4-475c-90ff-45acc03e85e1" (UID: "365c344c-05a4-475c-90ff-45acc03e85e1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 07:33:57.148692 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.148640 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/365c344c-05a4-475c-90ff-45acc03e85e1-registry-tls\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:33:57.148850 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.148812 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/365c344c-05a4-475c-90ff-45acc03e85e1-feast-data" (OuterVolumeSpecName: "feast-data") pod "365c344c-05a4-475c-90ff-45acc03e85e1" (UID: "365c344c-05a4-475c-90ff-45acc03e85e1"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:33:57.151048 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.151003 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/365c344c-05a4-475c-90ff-45acc03e85e1-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "365c344c-05a4-475c-90ff-45acc03e85e1" (UID: "365c344c-05a4-475c-90ff-45acc03e85e1"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:33:57.151048 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.151024 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/365c344c-05a4-475c-90ff-45acc03e85e1-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "365c344c-05a4-475c-90ff-45acc03e85e1" (UID: "365c344c-05a4-475c-90ff-45acc03e85e1"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:33:57.151179 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.151066 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/365c344c-05a4-475c-90ff-45acc03e85e1-kube-api-access-rftk2" (OuterVolumeSpecName: "kube-api-access-rftk2") pod "365c344c-05a4-475c-90ff-45acc03e85e1" (UID: "365c344c-05a4-475c-90ff-45acc03e85e1"). InnerVolumeSpecName "kube-api-access-rftk2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:33:57.151179 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.151073 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/365c344c-05a4-475c-90ff-45acc03e85e1-online-tls" (OuterVolumeSpecName: "online-tls") pod "365c344c-05a4-475c-90ff-45acc03e85e1" (UID: "365c344c-05a4-475c-90ff-45acc03e85e1"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:33:57.156049 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.156032 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:33:57.249830 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.249806 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-online-tls\") pod \"62e673eb-a4f3-452a-95e3-5f533820c351\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " Sep 17 07:33:57.249961 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.249849 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-offline-tls\") pod \"62e673eb-a4f3-452a-95e3-5f533820c351\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " Sep 17 07:33:57.249961 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.249863 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-registry-tls\") pod \"62e673eb-a4f3-452a-95e3-5f533820c351\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " Sep 17 07:33:57.249961 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.249911 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-ui-tls\") pod \"62e673eb-a4f3-452a-95e3-5f533820c351\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " Sep 17 07:33:57.250122 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.249984 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfkbz\" (UniqueName: \"kubernetes.io/projected/62e673eb-a4f3-452a-95e3-5f533820c351-kube-api-access-lfkbz\") pod \"62e673eb-a4f3-452a-95e3-5f533820c351\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " Sep 17 07:33:57.250122 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.250030 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/62e673eb-a4f3-452a-95e3-5f533820c351-feast-data\") pod \"62e673eb-a4f3-452a-95e3-5f533820c351\" (UID: \"62e673eb-a4f3-452a-95e3-5f533820c351\") " Sep 17 07:33:57.250305 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.250269 2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/365c344c-05a4-475c-90ff-45acc03e85e1-online-tls\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:33:57.250305 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.250286 2563 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/365c344c-05a4-475c-90ff-45acc03e85e1-ui-tls\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:33:57.250305 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.250298 2563 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/365c344c-05a4-475c-90ff-45acc03e85e1-offline-tls\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:33:57.250524 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.250308 2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/365c344c-05a4-475c-90ff-45acc03e85e1-feast-data\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:33:57.250524 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.250350 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rftk2\" (UniqueName: \"kubernetes.io/projected/365c344c-05a4-475c-90ff-45acc03e85e1-kube-api-access-rftk2\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:33:57.250849 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.250822 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62e673eb-a4f3-452a-95e3-5f533820c351-feast-data" (OuterVolumeSpecName: "feast-data") pod "62e673eb-a4f3-452a-95e3-5f533820c351" (UID: "62e673eb-a4f3-452a-95e3-5f533820c351"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:33:57.252153 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.252127 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "62e673eb-a4f3-452a-95e3-5f533820c351" (UID: "62e673eb-a4f3-452a-95e3-5f533820c351"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:33:57.252153 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.252149 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "62e673eb-a4f3-452a-95e3-5f533820c351" (UID: "62e673eb-a4f3-452a-95e3-5f533820c351"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:33:57.252289 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.252170 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "62e673eb-a4f3-452a-95e3-5f533820c351" (UID: "62e673eb-a4f3-452a-95e3-5f533820c351"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:33:57.252611 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.252588 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-online-tls" (OuterVolumeSpecName: "online-tls") pod "62e673eb-a4f3-452a-95e3-5f533820c351" (UID: "62e673eb-a4f3-452a-95e3-5f533820c351"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:33:57.252702 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.252610 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62e673eb-a4f3-452a-95e3-5f533820c351-kube-api-access-lfkbz" (OuterVolumeSpecName: "kube-api-access-lfkbz") pod "62e673eb-a4f3-452a-95e3-5f533820c351" (UID: "62e673eb-a4f3-452a-95e3-5f533820c351"). InnerVolumeSpecName "kube-api-access-lfkbz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:33:57.351468 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.351446 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfkbz\" (UniqueName: \"kubernetes.io/projected/62e673eb-a4f3-452a-95e3-5f533820c351-kube-api-access-lfkbz\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:33:57.351468 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.351467 2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/62e673eb-a4f3-452a-95e3-5f533820c351-feast-data\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:33:57.351584 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.351477 2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-online-tls\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:33:57.351584 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.351484 2563 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-offline-tls\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:33:57.351584 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.351491 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-registry-tls\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:33:57.351584 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.351498 2563 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/62e673eb-a4f3-452a-95e3-5f533820c351-ui-tls\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:33:57.992257 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.992223 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" Sep 17 07:33:57.992257 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.992248 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl" event={"ID":"62e673eb-a4f3-452a-95e3-5f533820c351","Type":"ContainerDied","Data":"a88b498860c5999e5f0be579b6e8b6e4d283de3eef10b9fb7a6f7822ee6612b9"} Sep 17 07:33:57.992829 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.992307 2563 scope.go:117] "RemoveContainer" containerID="f8f0c08d7d1e28683ece928fd2482ba29fe7c8a4f7b9083a6b5ae8e1fcd0e9f0" Sep 17 07:33:57.995617 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.995593 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" event={"ID":"365c344c-05a4-475c-90ff-45acc03e85e1","Type":"ContainerDied","Data":"76da82a4f315389fd32d65f8dad9a1a2dc0480e920f631f8076c18e78eda31d0"} Sep 17 07:33:57.995727 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:57.995643 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8" Sep 17 07:33:58.002125 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:58.002106 2563 scope.go:117] "RemoveContainer" containerID="6ba19a099547d1ab4fb802459df3b3a6dfeb5df8ef428b05fab2d177dc5777a6" Sep 17 07:33:58.010224 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:58.010209 2563 scope.go:117] "RemoveContainer" containerID="26d007bc3753b1697fca04d47a4cd881964bb231464d1b944b002ded010e90b0" Sep 17 07:33:58.018651 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:58.018632 2563 scope.go:117] "RemoveContainer" containerID="58be9e70a901b28d3e3a13b3afcc2bab794034c37cfe74667b0ad587d4ef8b18" Sep 17 07:33:58.019789 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:58.019771 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8"] Sep 17 07:33:58.024754 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:58.024731 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-6648699555-2tld8"] Sep 17 07:33:58.029076 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:58.029039 2563 scope.go:117] "RemoveContainer" containerID="c4f120466c3b52b666ff87a121164e2bacfc1a09cb126f681c62b494059432ef" Sep 17 07:33:58.036894 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:58.036877 2563 scope.go:117] "RemoveContainer" containerID="a8ad2687e8ccf22f4f9be646a36d1486b8b12a07361cb6d52a165105d7efbaeb" Sep 17 07:33:58.044689 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:58.044670 2563 scope.go:117] "RemoveContainer" containerID="3c4f5f225bc1fd796fa3e4d9a5f0650c6c1d642127d36f642f0b8079be5136b0" Sep 17 07:33:58.044762 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:58.044721 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl"] Sep 17 07:33:58.050193 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:58.050172 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-b6bf84db7-rdkbl"] Sep 17 07:33:58.052419 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:58.052405 2563 scope.go:117] "RemoveContainer" containerID="15b92a9be5c4624cdda5f42ec54ac072001659347de77d7f81bf1ffc6569e142" Sep 17 07:33:58.059688 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:58.059671 2563 scope.go:117] "RemoveContainer" containerID="9272fcd2a17ce023f9f2426425bab3af9d20664be9d1bd818b9e299125106a0f" Sep 17 07:33:58.066852 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:58.066838 2563 scope.go:117] "RemoveContainer" containerID="4b3cf5a8a44746a17bb3946a5e557637cd1b8d029e835cbaf54691366d68f6ef" Sep 17 07:33:58.073819 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:58.073803 2563 scope.go:117] "RemoveContainer" containerID="2a14a223c90e920d1cda7e45945a501779f7bf0276ed103558c1bd21b2ac0582" Sep 17 07:33:59.314133 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:59.314102 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" path="/var/lib/kubelet/pods/365c344c-05a4-475c-90ff-45acc03e85e1/volumes" Sep 17 07:33:59.314650 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:33:59.314635 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" path="/var/lib/kubelet/pods/62e673eb-a4f3-452a-95e3-5f533820c351/volumes" Sep 17 07:34:09.835997 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.835969 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-w9btr"] Sep 17 07:34:09.836402 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836265 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="feast-apply" Sep 17 07:34:09.836402 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836276 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="feast-apply" Sep 17 07:34:09.836402 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836288 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="feast-apply" Sep 17 07:34:09.836402 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836293 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="feast-apply" Sep 17 07:34:09.836402 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836305 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="feast-init" Sep 17 07:34:09.836402 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836310 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="feast-init" Sep 17 07:34:09.836402 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836316 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="online" Sep 17 07:34:09.836402 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836346 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="online" Sep 17 07:34:09.836402 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836354 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="feast-init" Sep 17 07:34:09.836402 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836360 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="feast-init" Sep 17 07:34:09.836402 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836366 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="offline" Sep 17 07:34:09.836402 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836371 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="offline" Sep 17 07:34:09.836402 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836379 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="ui" Sep 17 07:34:09.836402 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836386 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="ui" Sep 17 07:34:09.836402 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836394 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="offline" Sep 17 07:34:09.836402 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836399 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="offline" Sep 17 07:34:09.836402 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836407 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="registry" Sep 17 07:34:09.836402 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836412 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="registry" Sep 17 07:34:09.836958 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836419 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="ui" Sep 17 07:34:09.836958 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836424 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="ui" Sep 17 07:34:09.836958 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836432 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="online" Sep 17 07:34:09.836958 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836437 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="online" Sep 17 07:34:09.836958 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836495 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="online" Sep 17 07:34:09.836958 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836502 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="registry" Sep 17 07:34:09.836958 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836508 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="offline" Sep 17 07:34:09.836958 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836514 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="online" Sep 17 07:34:09.836958 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836520 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="62e673eb-a4f3-452a-95e3-5f533820c351" containerName="ui" Sep 17 07:34:09.836958 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836526 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="offline" Sep 17 07:34:09.836958 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.836534 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="365c344c-05a4-475c-90ff-45acc03e85e1" containerName="ui" Sep 17 07:34:09.841143 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.841127 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-w9btr" Sep 17 07:34:09.844476 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.844453 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Sep 17 07:34:09.844590 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.844492 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Sep 17 07:34:09.844590 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.844567 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Sep 17 07:34:09.846090 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.846073 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-m22nf\"" Sep 17 07:34:09.865984 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.865963 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-w9btr"] Sep 17 07:34:09.901396 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.901372 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-69bbd798b6-f6g8p"] Sep 17 07:34:09.904613 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.904594 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-f6g8p" Sep 17 07:34:09.919200 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.919178 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-f6g8p"] Sep 17 07:34:09.947365 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.947310 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/7411ed1c-7274-4d6f-988f-8b22a4fc5065-postgresdata\") pod \"postgres-864846d4f5-w9btr\" (UID: \"7411ed1c-7274-4d6f-988f-8b22a4fc5065\") " pod="test-ns-feast/postgres-864846d4f5-w9btr" Sep 17 07:34:09.947488 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:09.947388 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9sfs\" (UniqueName: \"kubernetes.io/projected/7411ed1c-7274-4d6f-988f-8b22a4fc5065-kube-api-access-b9sfs\") pod \"postgres-864846d4f5-w9btr\" (UID: \"7411ed1c-7274-4d6f-988f-8b22a4fc5065\") " pod="test-ns-feast/postgres-864846d4f5-w9btr" Sep 17 07:34:10.047963 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:10.047939 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9sfs\" (UniqueName: \"kubernetes.io/projected/7411ed1c-7274-4d6f-988f-8b22a4fc5065-kube-api-access-b9sfs\") pod \"postgres-864846d4f5-w9btr\" (UID: \"7411ed1c-7274-4d6f-988f-8b22a4fc5065\") " pod="test-ns-feast/postgres-864846d4f5-w9btr" Sep 17 07:34:10.048115 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:10.048002 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh745\" (UniqueName: \"kubernetes.io/projected/c19cfb66-3f5a-4f63-bc92-bb1133e511c1-kube-api-access-mh745\") pod \"redis-69bbd798b6-f6g8p\" (UID: \"c19cfb66-3f5a-4f63-bc92-bb1133e511c1\") " pod="test-ns-feast/redis-69bbd798b6-f6g8p" Sep 17 07:34:10.048115 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:10.048028 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/7411ed1c-7274-4d6f-988f-8b22a4fc5065-postgresdata\") pod \"postgres-864846d4f5-w9btr\" (UID: \"7411ed1c-7274-4d6f-988f-8b22a4fc5065\") " pod="test-ns-feast/postgres-864846d4f5-w9btr" Sep 17 07:34:10.048383 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:10.048366 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/7411ed1c-7274-4d6f-988f-8b22a4fc5065-postgresdata\") pod \"postgres-864846d4f5-w9btr\" (UID: \"7411ed1c-7274-4d6f-988f-8b22a4fc5065\") " pod="test-ns-feast/postgres-864846d4f5-w9btr" Sep 17 07:34:10.060567 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:10.060548 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9sfs\" (UniqueName: \"kubernetes.io/projected/7411ed1c-7274-4d6f-988f-8b22a4fc5065-kube-api-access-b9sfs\") pod \"postgres-864846d4f5-w9btr\" (UID: \"7411ed1c-7274-4d6f-988f-8b22a4fc5065\") " pod="test-ns-feast/postgres-864846d4f5-w9btr" Sep 17 07:34:10.149220 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:10.149160 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh745\" (UniqueName: \"kubernetes.io/projected/c19cfb66-3f5a-4f63-bc92-bb1133e511c1-kube-api-access-mh745\") pod \"redis-69bbd798b6-f6g8p\" (UID: \"c19cfb66-3f5a-4f63-bc92-bb1133e511c1\") " pod="test-ns-feast/redis-69bbd798b6-f6g8p" Sep 17 07:34:10.158909 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:10.158882 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh745\" (UniqueName: \"kubernetes.io/projected/c19cfb66-3f5a-4f63-bc92-bb1133e511c1-kube-api-access-mh745\") pod \"redis-69bbd798b6-f6g8p\" (UID: \"c19cfb66-3f5a-4f63-bc92-bb1133e511c1\") " pod="test-ns-feast/redis-69bbd798b6-f6g8p" Sep 17 07:34:10.163704 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:10.163689 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-w9btr" Sep 17 07:34:10.213199 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:10.213140 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-f6g8p" Sep 17 07:34:10.444181 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:10.444160 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-w9btr"] Sep 17 07:34:10.446677 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:34:10.446645 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7411ed1c_7274_4d6f_988f_8b22a4fc5065.slice/crio-b692c39894e5cf7b710926528663917bc01c60b3197432ca30fc7cbbcf8bc79d WatchSource:0}: Error finding container b692c39894e5cf7b710926528663917bc01c60b3197432ca30fc7cbbcf8bc79d: Status 404 returned error can't find the container with id b692c39894e5cf7b710926528663917bc01c60b3197432ca30fc7cbbcf8bc79d Sep 17 07:34:10.448571 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:10.448556 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 07:34:10.499164 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:10.499138 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-f6g8p"] Sep 17 07:34:10.501972 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:34:10.501942 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc19cfb66_3f5a_4f63_bc92_bb1133e511c1.slice/crio-3577773fd5e09eb41d88c263deaea3eed95ce9af290f4dd3dadf4911bcc9ef59 WatchSource:0}: Error finding container 3577773fd5e09eb41d88c263deaea3eed95ce9af290f4dd3dadf4911bcc9ef59: Status 404 returned error can't find the container with id 3577773fd5e09eb41d88c263deaea3eed95ce9af290f4dd3dadf4911bcc9ef59 Sep 17 07:34:11.048120 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:11.048082 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-f6g8p" event={"ID":"c19cfb66-3f5a-4f63-bc92-bb1133e511c1","Type":"ContainerStarted","Data":"3577773fd5e09eb41d88c263deaea3eed95ce9af290f4dd3dadf4911bcc9ef59"} Sep 17 07:34:11.049371 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:11.049339 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-w9btr" event={"ID":"7411ed1c-7274-4d6f-988f-8b22a4fc5065","Type":"ContainerStarted","Data":"b692c39894e5cf7b710926528663917bc01c60b3197432ca30fc7cbbcf8bc79d"} Sep 17 07:34:13.061580 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:13.061549 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-f6g8p" event={"ID":"c19cfb66-3f5a-4f63-bc92-bb1133e511c1","Type":"ContainerStarted","Data":"8c3475b86edce062a822ee5d8b35aa54ccdd70a73880665b125be8b417e1bb0a"} Sep 17 07:34:13.082482 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:13.082432 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/redis-69bbd798b6-f6g8p" podStartSLOduration=2.451613905 podStartE2EDuration="4.082411237s" totalImagesPullingTime="1.630797332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:34:13.080284765 +0000 UTC m=+860.266017738" watchObservedRunningTime="2026-09-17 07:34:13.082411237 +0000 UTC m=+860.268144209" Sep 17 07:34:16.077821 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:16.077791 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-w9btr" event={"ID":"7411ed1c-7274-4d6f-988f-8b22a4fc5065","Type":"ContainerStarted","Data":"515884a95ff6b62aa0d5f05e25532866dd173398dac69a2706ae9badadf96802"} Sep 17 07:34:16.100258 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:16.100199 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-w9btr" podStartSLOduration=2.265379927 podStartE2EDuration="7.100178845s" totalImagesPullingTime="4.834798918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:34:16.097727754 +0000 UTC m=+863.283460726" watchObservedRunningTime="2026-09-17 07:34:16.100178845 +0000 UTC m=+863.285911818" Sep 17 07:34:20.785433 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:20.785399 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d"] Sep 17 07:34:20.788942 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:20.788921 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:20.797584 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:20.797562 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\"" Sep 17 07:34:20.797900 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:20.797884 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-hzmrz\"" Sep 17 07:34:20.798204 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:20.798188 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\"" Sep 17 07:34:20.798645 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:20.798631 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Sep 17 07:34:20.799112 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:20.799099 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\"" Sep 17 07:34:20.840217 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:20.840195 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d"] Sep 17 07:34:20.937708 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:20.937686 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhxk2\" (UniqueName: \"kubernetes.io/projected/d198d556-3626-4394-938d-2ca4266cec0c-kube-api-access-bhxk2\") pod \"feast-credit-scoring-64475b79d6-xwk7d\" (UID: \"d198d556-3626-4394-938d-2ca4266cec0c\") " pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:20.937832 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:20.937726 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d198d556-3626-4394-938d-2ca4266cec0c-online-tls\") pod \"feast-credit-scoring-64475b79d6-xwk7d\" (UID: \"d198d556-3626-4394-938d-2ca4266cec0c\") " pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:20.937832 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:20.937751 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d198d556-3626-4394-938d-2ca4266cec0c-feast-data\") pod \"feast-credit-scoring-64475b79d6-xwk7d\" (UID: \"d198d556-3626-4394-938d-2ca4266cec0c\") " pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:20.937941 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:20.937870 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d198d556-3626-4394-938d-2ca4266cec0c-registry-tls\") pod \"feast-credit-scoring-64475b79d6-xwk7d\" (UID: \"d198d556-3626-4394-938d-2ca4266cec0c\") " pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:20.937999 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:20.937941 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d198d556-3626-4394-938d-2ca4266cec0c-offline-tls\") pod \"feast-credit-scoring-64475b79d6-xwk7d\" (UID: \"d198d556-3626-4394-938d-2ca4266cec0c\") " pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:21.038534 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:21.038460 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d198d556-3626-4394-938d-2ca4266cec0c-offline-tls\") pod \"feast-credit-scoring-64475b79d6-xwk7d\" (UID: \"d198d556-3626-4394-938d-2ca4266cec0c\") " pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:21.038534 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:21.038516 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhxk2\" (UniqueName: \"kubernetes.io/projected/d198d556-3626-4394-938d-2ca4266cec0c-kube-api-access-bhxk2\") pod \"feast-credit-scoring-64475b79d6-xwk7d\" (UID: \"d198d556-3626-4394-938d-2ca4266cec0c\") " pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:21.038718 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:21.038561 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d198d556-3626-4394-938d-2ca4266cec0c-online-tls\") pod \"feast-credit-scoring-64475b79d6-xwk7d\" (UID: \"d198d556-3626-4394-938d-2ca4266cec0c\") " pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:21.038718 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:21.038590 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d198d556-3626-4394-938d-2ca4266cec0c-feast-data\") pod \"feast-credit-scoring-64475b79d6-xwk7d\" (UID: \"d198d556-3626-4394-938d-2ca4266cec0c\") " pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:21.038718 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:21.038648 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d198d556-3626-4394-938d-2ca4266cec0c-registry-tls\") pod \"feast-credit-scoring-64475b79d6-xwk7d\" (UID: \"d198d556-3626-4394-938d-2ca4266cec0c\") " pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:21.038931 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:34:21.038764 2563 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-registry-tls: secret "feast-credit-scoring-registry-tls" not found Sep 17 07:34:21.038931 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:34:21.038835 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d198d556-3626-4394-938d-2ca4266cec0c-registry-tls podName:d198d556-3626-4394-938d-2ca4266cec0c nodeName:}" failed. No retries permitted until 2026-09-17 07:34:21.538814379 +0000 UTC m=+868.724547333 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/d198d556-3626-4394-938d-2ca4266cec0c-registry-tls") pod "feast-credit-scoring-64475b79d6-xwk7d" (UID: "d198d556-3626-4394-938d-2ca4266cec0c") : secret "feast-credit-scoring-registry-tls" not found Sep 17 07:34:21.039043 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:21.039019 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d198d556-3626-4394-938d-2ca4266cec0c-feast-data\") pod \"feast-credit-scoring-64475b79d6-xwk7d\" (UID: \"d198d556-3626-4394-938d-2ca4266cec0c\") " pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:21.041599 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:21.041573 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d198d556-3626-4394-938d-2ca4266cec0c-offline-tls\") pod \"feast-credit-scoring-64475b79d6-xwk7d\" (UID: \"d198d556-3626-4394-938d-2ca4266cec0c\") " pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:21.041702 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:21.041689 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d198d556-3626-4394-938d-2ca4266cec0c-online-tls\") pod \"feast-credit-scoring-64475b79d6-xwk7d\" (UID: \"d198d556-3626-4394-938d-2ca4266cec0c\") " pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:21.048828 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:21.048808 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhxk2\" (UniqueName: \"kubernetes.io/projected/d198d556-3626-4394-938d-2ca4266cec0c-kube-api-access-bhxk2\") pod \"feast-credit-scoring-64475b79d6-xwk7d\" (UID: \"d198d556-3626-4394-938d-2ca4266cec0c\") " pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:21.541745 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:21.541713 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d198d556-3626-4394-938d-2ca4266cec0c-registry-tls\") pod \"feast-credit-scoring-64475b79d6-xwk7d\" (UID: \"d198d556-3626-4394-938d-2ca4266cec0c\") " pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:21.544243 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:21.544219 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d198d556-3626-4394-938d-2ca4266cec0c-registry-tls\") pod \"feast-credit-scoring-64475b79d6-xwk7d\" (UID: \"d198d556-3626-4394-938d-2ca4266cec0c\") " pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:21.698062 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:21.698036 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:21.994023 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:21.993995 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d"] Sep 17 07:34:21.995399 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:34:21.995365 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd198d556_3626_4394_938d_2ca4266cec0c.slice/crio-ba67f13e6fd2b3d42e49a49ae09c6a49d8350e2e92621ce6b7e8c54990e784e8 WatchSource:0}: Error finding container ba67f13e6fd2b3d42e49a49ae09c6a49d8350e2e92621ce6b7e8c54990e784e8: Status 404 returned error can't find the container with id ba67f13e6fd2b3d42e49a49ae09c6a49d8350e2e92621ce6b7e8c54990e784e8 Sep 17 07:34:22.103490 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:22.103457 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" event={"ID":"d198d556-3626-4394-938d-2ca4266cec0c","Type":"ContainerStarted","Data":"d4d349d58ddcf35dc8107b9d3f1a64edd6357b76eea3b380286014155b1f21e4"} Sep 17 07:34:22.103602 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:22.103513 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" event={"ID":"d198d556-3626-4394-938d-2ca4266cec0c","Type":"ContainerStarted","Data":"ba67f13e6fd2b3d42e49a49ae09c6a49d8350e2e92621ce6b7e8c54990e784e8"} Sep 17 07:34:24.113926 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:24.113851 2563 generic.go:356] "Generic (PLEG): container finished" podID="d198d556-3626-4394-938d-2ca4266cec0c" containerID="d4d349d58ddcf35dc8107b9d3f1a64edd6357b76eea3b380286014155b1f21e4" exitCode=0 Sep 17 07:34:24.114253 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:24.113924 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" event={"ID":"d198d556-3626-4394-938d-2ca4266cec0c","Type":"ContainerDied","Data":"d4d349d58ddcf35dc8107b9d3f1a64edd6357b76eea3b380286014155b1f21e4"} Sep 17 07:34:25.120265 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:25.120231 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" event={"ID":"d198d556-3626-4394-938d-2ca4266cec0c","Type":"ContainerStarted","Data":"072cb8658328198f8cbcffd250607a34d5f68d9460b697214d06fcf06a925974"} Sep 17 07:34:31.150368 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:31.150273 2563 generic.go:356] "Generic (PLEG): container finished" podID="d198d556-3626-4394-938d-2ca4266cec0c" containerID="072cb8658328198f8cbcffd250607a34d5f68d9460b697214d06fcf06a925974" exitCode=0 Sep 17 07:34:31.150368 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:31.150311 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" event={"ID":"d198d556-3626-4394-938d-2ca4266cec0c","Type":"ContainerDied","Data":"072cb8658328198f8cbcffd250607a34d5f68d9460b697214d06fcf06a925974"} Sep 17 07:34:32.159447 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:32.159403 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" event={"ID":"d198d556-3626-4394-938d-2ca4266cec0c","Type":"ContainerStarted","Data":"b948c0efcf7420d104bd9b370269bf0c687a9e6d2c5bc6801f5e04a24a404aa8"} Sep 17 07:34:32.159932 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:32.159484 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" event={"ID":"d198d556-3626-4394-938d-2ca4266cec0c","Type":"ContainerStarted","Data":"94280fe506c05d1926f832e3f99924c51e37f1731efffaa8b1bec56deb5ff95c"} Sep 17 07:34:32.159932 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:32.159513 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" event={"ID":"d198d556-3626-4394-938d-2ca4266cec0c","Type":"ContainerStarted","Data":"8525f3ea8a7ae5d0bdf200e925928e7e691b7a19e312d4d7f07c17bd7dfe022c"} Sep 17 07:34:32.191609 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:32.191562 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" podStartSLOduration=12.191545496 podStartE2EDuration="12.191545496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:34:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:34:32.18872969 +0000 UTC m=+879.374462675" watchObservedRunningTime="2026-09-17 07:34:32.191545496 +0000 UTC m=+879.377278505" Sep 17 07:34:33.698254 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:33.698079 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:33.698254 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:33.698143 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:33.698254 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:33.698168 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:33.699976 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:33.699944 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.33:8816: connect: connection refused" Sep 17 07:34:33.700089 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:33.699984 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused" Sep 17 07:34:33.700189 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:33.699968 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.33:6571: connect: connection refused" Sep 17 07:34:36.698992 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:36.698939 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.33:6571: connect: connection refused" Sep 17 07:34:36.699494 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:36.698948 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.33:8816: connect: connection refused" Sep 17 07:34:36.699494 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:36.698964 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused" Sep 17 07:34:39.699221 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:39.699176 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.33:6571: connect: connection refused" Sep 17 07:34:39.700267 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:39.700242 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:39.700901 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:39.700878 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:39.701500 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:39.701479 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:39.705685 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:39.705667 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:40.207076 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:40.207041 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:40.210465 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:40.210443 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:41.699004 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:41.698969 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:42.698654 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:42.698615 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.33:6571: connect: connection refused" Sep 17 07:34:45.699271 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:45.699242 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:34:45.700008 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:34:45.699987 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:35:48.921702 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:48.921666 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d"] Sep 17 07:35:48.922152 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:48.921986 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="registry" containerID="cri-o://8525f3ea8a7ae5d0bdf200e925928e7e691b7a19e312d4d7f07c17bd7dfe022c" gracePeriod=30 Sep 17 07:35:48.922152 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:48.922014 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="offline" containerID="cri-o://b948c0efcf7420d104bd9b370269bf0c687a9e6d2c5bc6801f5e04a24a404aa8" gracePeriod=30 Sep 17 07:35:48.922152 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:48.922014 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="online" containerID="cri-o://94280fe506c05d1926f832e3f99924c51e37f1731efffaa8b1bec56deb5ff95c" gracePeriod=30 Sep 17 07:35:48.954919 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:48.954888 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-w9btr"] Sep 17 07:35:48.955095 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:48.955061 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-w9btr" podUID="7411ed1c-7274-4d6f-988f-8b22a4fc5065" containerName="postgres" containerID="cri-o://515884a95ff6b62aa0d5f05e25532866dd173398dac69a2706ae9badadf96802" gracePeriod=30 Sep 17 07:35:48.969722 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:48.969699 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-f6g8p"] Sep 17 07:35:48.969869 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:48.969851 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/redis-69bbd798b6-f6g8p" podUID="c19cfb66-3f5a-4f63-bc92-bb1133e511c1" containerName="redis" containerID="cri-o://8c3475b86edce062a822ee5d8b35aa54ccdd70a73880665b125be8b417e1bb0a" gracePeriod=30 Sep 17 07:35:49.231029 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.231008 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-w9btr" Sep 17 07:35:49.233946 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.233927 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-f6g8p" Sep 17 07:35:49.267893 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.267866 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/7411ed1c-7274-4d6f-988f-8b22a4fc5065-postgresdata\") pod \"7411ed1c-7274-4d6f-988f-8b22a4fc5065\" (UID: \"7411ed1c-7274-4d6f-988f-8b22a4fc5065\") " Sep 17 07:35:49.268147 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.267924 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9sfs\" (UniqueName: \"kubernetes.io/projected/7411ed1c-7274-4d6f-988f-8b22a4fc5065-kube-api-access-b9sfs\") pod \"7411ed1c-7274-4d6f-988f-8b22a4fc5065\" (UID: \"7411ed1c-7274-4d6f-988f-8b22a4fc5065\") " Sep 17 07:35:49.268147 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.268002 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mh745\" (UniqueName: \"kubernetes.io/projected/c19cfb66-3f5a-4f63-bc92-bb1133e511c1-kube-api-access-mh745\") pod \"c19cfb66-3f5a-4f63-bc92-bb1133e511c1\" (UID: \"c19cfb66-3f5a-4f63-bc92-bb1133e511c1\") " Sep 17 07:35:49.270783 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.270752 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7411ed1c-7274-4d6f-988f-8b22a4fc5065-kube-api-access-b9sfs" (OuterVolumeSpecName: "kube-api-access-b9sfs") pod "7411ed1c-7274-4d6f-988f-8b22a4fc5065" (UID: "7411ed1c-7274-4d6f-988f-8b22a4fc5065"). InnerVolumeSpecName "kube-api-access-b9sfs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:35:49.270892 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.270797 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c19cfb66-3f5a-4f63-bc92-bb1133e511c1-kube-api-access-mh745" (OuterVolumeSpecName: "kube-api-access-mh745") pod "c19cfb66-3f5a-4f63-bc92-bb1133e511c1" (UID: "c19cfb66-3f5a-4f63-bc92-bb1133e511c1"). InnerVolumeSpecName "kube-api-access-mh745". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:35:49.300676 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.300630 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7411ed1c-7274-4d6f-988f-8b22a4fc5065-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "7411ed1c-7274-4d6f-988f-8b22a4fc5065" (UID: "7411ed1c-7274-4d6f-988f-8b22a4fc5065"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:35:49.368785 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.368758 2563 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/7411ed1c-7274-4d6f-988f-8b22a4fc5065-postgresdata\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:35:49.368785 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.368785 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9sfs\" (UniqueName: \"kubernetes.io/projected/7411ed1c-7274-4d6f-988f-8b22a4fc5065-kube-api-access-b9sfs\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:35:49.368949 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.368795 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mh745\" (UniqueName: \"kubernetes.io/projected/c19cfb66-3f5a-4f63-bc92-bb1133e511c1-kube-api-access-mh745\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:35:49.505925 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.505833 2563 generic.go:356] "Generic (PLEG): container finished" podID="7411ed1c-7274-4d6f-988f-8b22a4fc5065" containerID="515884a95ff6b62aa0d5f05e25532866dd173398dac69a2706ae9badadf96802" exitCode=0 Sep 17 07:35:49.505925 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.505885 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-w9btr" Sep 17 07:35:49.506132 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.505920 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-w9btr" event={"ID":"7411ed1c-7274-4d6f-988f-8b22a4fc5065","Type":"ContainerDied","Data":"515884a95ff6b62aa0d5f05e25532866dd173398dac69a2706ae9badadf96802"} Sep 17 07:35:49.506132 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.505967 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-w9btr" event={"ID":"7411ed1c-7274-4d6f-988f-8b22a4fc5065","Type":"ContainerDied","Data":"b692c39894e5cf7b710926528663917bc01c60b3197432ca30fc7cbbcf8bc79d"} Sep 17 07:35:49.506132 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.505992 2563 scope.go:117] "RemoveContainer" containerID="515884a95ff6b62aa0d5f05e25532866dd173398dac69a2706ae9badadf96802" Sep 17 07:35:49.507731 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.507704 2563 generic.go:356] "Generic (PLEG): container finished" podID="c19cfb66-3f5a-4f63-bc92-bb1133e511c1" containerID="8c3475b86edce062a822ee5d8b35aa54ccdd70a73880665b125be8b417e1bb0a" exitCode=0 Sep 17 07:35:49.507953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.507749 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-f6g8p" event={"ID":"c19cfb66-3f5a-4f63-bc92-bb1133e511c1","Type":"ContainerDied","Data":"8c3475b86edce062a822ee5d8b35aa54ccdd70a73880665b125be8b417e1bb0a"} Sep 17 07:35:49.507953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.507762 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-f6g8p" Sep 17 07:35:49.508065 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.507776 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-f6g8p" event={"ID":"c19cfb66-3f5a-4f63-bc92-bb1133e511c1","Type":"ContainerDied","Data":"3577773fd5e09eb41d88c263deaea3eed95ce9af290f4dd3dadf4911bcc9ef59"} Sep 17 07:35:49.515754 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.515731 2563 scope.go:117] "RemoveContainer" containerID="515884a95ff6b62aa0d5f05e25532866dd173398dac69a2706ae9badadf96802" Sep 17 07:35:49.516034 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:35:49.516015 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"515884a95ff6b62aa0d5f05e25532866dd173398dac69a2706ae9badadf96802\": container with ID starting with 515884a95ff6b62aa0d5f05e25532866dd173398dac69a2706ae9badadf96802 not found: ID does not exist" containerID="515884a95ff6b62aa0d5f05e25532866dd173398dac69a2706ae9badadf96802" Sep 17 07:35:49.516099 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.516041 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"515884a95ff6b62aa0d5f05e25532866dd173398dac69a2706ae9badadf96802"} err="failed to get container status \"515884a95ff6b62aa0d5f05e25532866dd173398dac69a2706ae9badadf96802\": rpc error: code = NotFound desc = could not find container \"515884a95ff6b62aa0d5f05e25532866dd173398dac69a2706ae9badadf96802\": container with ID starting with 515884a95ff6b62aa0d5f05e25532866dd173398dac69a2706ae9badadf96802 not found: ID does not exist" Sep 17 07:35:49.516099 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.516056 2563 scope.go:117] "RemoveContainer" containerID="8c3475b86edce062a822ee5d8b35aa54ccdd70a73880665b125be8b417e1bb0a" Sep 17 07:35:49.524218 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.524205 2563 scope.go:117] "RemoveContainer" containerID="8c3475b86edce062a822ee5d8b35aa54ccdd70a73880665b125be8b417e1bb0a" Sep 17 07:35:49.524591 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:35:49.524573 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c3475b86edce062a822ee5d8b35aa54ccdd70a73880665b125be8b417e1bb0a\": container with ID starting with 8c3475b86edce062a822ee5d8b35aa54ccdd70a73880665b125be8b417e1bb0a not found: ID does not exist" containerID="8c3475b86edce062a822ee5d8b35aa54ccdd70a73880665b125be8b417e1bb0a" Sep 17 07:35:49.524664 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.524603 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c3475b86edce062a822ee5d8b35aa54ccdd70a73880665b125be8b417e1bb0a"} err="failed to get container status \"8c3475b86edce062a822ee5d8b35aa54ccdd70a73880665b125be8b417e1bb0a\": rpc error: code = NotFound desc = could not find container \"8c3475b86edce062a822ee5d8b35aa54ccdd70a73880665b125be8b417e1bb0a\": container with ID starting with 8c3475b86edce062a822ee5d8b35aa54ccdd70a73880665b125be8b417e1bb0a not found: ID does not exist" Sep 17 07:35:49.525511 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.525493 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-w9btr"] Sep 17 07:35:49.530643 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.530610 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-w9btr"] Sep 17 07:35:49.547126 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.547099 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-f6g8p"] Sep 17 07:35:49.560207 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:49.560178 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-69bbd798b6-f6g8p"] Sep 17 07:35:50.207138 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:50.207101 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused" Sep 17 07:35:50.516594 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:50.516510 2563 generic.go:356] "Generic (PLEG): container finished" podID="d198d556-3626-4394-938d-2ca4266cec0c" containerID="94280fe506c05d1926f832e3f99924c51e37f1731efffaa8b1bec56deb5ff95c" exitCode=0 Sep 17 07:35:50.516739 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:50.516587 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" event={"ID":"d198d556-3626-4394-938d-2ca4266cec0c","Type":"ContainerDied","Data":"94280fe506c05d1926f832e3f99924c51e37f1731efffaa8b1bec56deb5ff95c"} Sep 17 07:35:51.313976 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:51.313945 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7411ed1c-7274-4d6f-988f-8b22a4fc5065" path="/var/lib/kubelet/pods/7411ed1c-7274-4d6f-988f-8b22a4fc5065/volumes" Sep 17 07:35:51.314368 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:35:51.314309 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c19cfb66-3f5a-4f63-bc92-bb1133e511c1" path="/var/lib/kubelet/pods/c19cfb66-3f5a-4f63-bc92-bb1133e511c1/volumes" Sep 17 07:36:00.207705 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:00.207665 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused" Sep 17 07:36:10.207930 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:10.207889 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused" Sep 17 07:36:10.208317 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:10.208006 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:36:19.563029 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.563005 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:36:19.634643 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.634611 2563 generic.go:356] "Generic (PLEG): container finished" podID="d198d556-3626-4394-938d-2ca4266cec0c" containerID="b948c0efcf7420d104bd9b370269bf0c687a9e6d2c5bc6801f5e04a24a404aa8" exitCode=137 Sep 17 07:36:19.634643 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.634641 2563 generic.go:356] "Generic (PLEG): container finished" podID="d198d556-3626-4394-938d-2ca4266cec0c" containerID="8525f3ea8a7ae5d0bdf200e925928e7e691b7a19e312d4d7f07c17bd7dfe022c" exitCode=137 Sep 17 07:36:19.634822 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.634660 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" Sep 17 07:36:19.634822 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.634670 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" event={"ID":"d198d556-3626-4394-938d-2ca4266cec0c","Type":"ContainerDied","Data":"b948c0efcf7420d104bd9b370269bf0c687a9e6d2c5bc6801f5e04a24a404aa8"} Sep 17 07:36:19.634822 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.634729 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" event={"ID":"d198d556-3626-4394-938d-2ca4266cec0c","Type":"ContainerDied","Data":"8525f3ea8a7ae5d0bdf200e925928e7e691b7a19e312d4d7f07c17bd7dfe022c"} Sep 17 07:36:19.634822 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.634758 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d" event={"ID":"d198d556-3626-4394-938d-2ca4266cec0c","Type":"ContainerDied","Data":"ba67f13e6fd2b3d42e49a49ae09c6a49d8350e2e92621ce6b7e8c54990e784e8"} Sep 17 07:36:19.634822 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.634789 2563 scope.go:117] "RemoveContainer" containerID="b948c0efcf7420d104bd9b370269bf0c687a9e6d2c5bc6801f5e04a24a404aa8" Sep 17 07:36:19.644212 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.644197 2563 scope.go:117] "RemoveContainer" containerID="94280fe506c05d1926f832e3f99924c51e37f1731efffaa8b1bec56deb5ff95c" Sep 17 07:36:19.651816 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.651802 2563 scope.go:117] "RemoveContainer" containerID="8525f3ea8a7ae5d0bdf200e925928e7e691b7a19e312d4d7f07c17bd7dfe022c" Sep 17 07:36:19.659091 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.659070 2563 scope.go:117] "RemoveContainer" containerID="072cb8658328198f8cbcffd250607a34d5f68d9460b697214d06fcf06a925974" Sep 17 07:36:19.666076 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.666060 2563 scope.go:117] "RemoveContainer" containerID="d4d349d58ddcf35dc8107b9d3f1a64edd6357b76eea3b380286014155b1f21e4" Sep 17 07:36:19.672965 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.672947 2563 scope.go:117] "RemoveContainer" containerID="b948c0efcf7420d104bd9b370269bf0c687a9e6d2c5bc6801f5e04a24a404aa8" Sep 17 07:36:19.673247 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:36:19.673230 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b948c0efcf7420d104bd9b370269bf0c687a9e6d2c5bc6801f5e04a24a404aa8\": container with ID starting with b948c0efcf7420d104bd9b370269bf0c687a9e6d2c5bc6801f5e04a24a404aa8 not found: ID does not exist" containerID="b948c0efcf7420d104bd9b370269bf0c687a9e6d2c5bc6801f5e04a24a404aa8" Sep 17 07:36:19.673290 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.673255 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b948c0efcf7420d104bd9b370269bf0c687a9e6d2c5bc6801f5e04a24a404aa8"} err="failed to get container status \"b948c0efcf7420d104bd9b370269bf0c687a9e6d2c5bc6801f5e04a24a404aa8\": rpc error: code = NotFound desc = could not find container \"b948c0efcf7420d104bd9b370269bf0c687a9e6d2c5bc6801f5e04a24a404aa8\": container with ID starting with b948c0efcf7420d104bd9b370269bf0c687a9e6d2c5bc6801f5e04a24a404aa8 not found: ID does not exist" Sep 17 07:36:19.673290 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.673272 2563 scope.go:117] "RemoveContainer" containerID="94280fe506c05d1926f832e3f99924c51e37f1731efffaa8b1bec56deb5ff95c" Sep 17 07:36:19.673517 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:36:19.673500 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94280fe506c05d1926f832e3f99924c51e37f1731efffaa8b1bec56deb5ff95c\": container with ID starting with 94280fe506c05d1926f832e3f99924c51e37f1731efffaa8b1bec56deb5ff95c not found: ID does not exist" containerID="94280fe506c05d1926f832e3f99924c51e37f1731efffaa8b1bec56deb5ff95c" Sep 17 07:36:19.673559 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.673522 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94280fe506c05d1926f832e3f99924c51e37f1731efffaa8b1bec56deb5ff95c"} err="failed to get container status \"94280fe506c05d1926f832e3f99924c51e37f1731efffaa8b1bec56deb5ff95c\": rpc error: code = NotFound desc = could not find container \"94280fe506c05d1926f832e3f99924c51e37f1731efffaa8b1bec56deb5ff95c\": container with ID starting with 94280fe506c05d1926f832e3f99924c51e37f1731efffaa8b1bec56deb5ff95c not found: ID does not exist" Sep 17 07:36:19.673559 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.673534 2563 scope.go:117] "RemoveContainer" containerID="8525f3ea8a7ae5d0bdf200e925928e7e691b7a19e312d4d7f07c17bd7dfe022c" Sep 17 07:36:19.673754 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:36:19.673734 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8525f3ea8a7ae5d0bdf200e925928e7e691b7a19e312d4d7f07c17bd7dfe022c\": container with ID starting with 8525f3ea8a7ae5d0bdf200e925928e7e691b7a19e312d4d7f07c17bd7dfe022c not found: ID does not exist" containerID="8525f3ea8a7ae5d0bdf200e925928e7e691b7a19e312d4d7f07c17bd7dfe022c" Sep 17 07:36:19.673793 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.673760 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8525f3ea8a7ae5d0bdf200e925928e7e691b7a19e312d4d7f07c17bd7dfe022c"} err="failed to get container status \"8525f3ea8a7ae5d0bdf200e925928e7e691b7a19e312d4d7f07c17bd7dfe022c\": rpc error: code = NotFound desc = could not find container \"8525f3ea8a7ae5d0bdf200e925928e7e691b7a19e312d4d7f07c17bd7dfe022c\": container with ID starting with 8525f3ea8a7ae5d0bdf200e925928e7e691b7a19e312d4d7f07c17bd7dfe022c not found: ID does not exist" Sep 17 07:36:19.673793 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.673775 2563 scope.go:117] "RemoveContainer" containerID="072cb8658328198f8cbcffd250607a34d5f68d9460b697214d06fcf06a925974" Sep 17 07:36:19.674038 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:36:19.674020 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"072cb8658328198f8cbcffd250607a34d5f68d9460b697214d06fcf06a925974\": container with ID starting with 072cb8658328198f8cbcffd250607a34d5f68d9460b697214d06fcf06a925974 not found: ID does not exist" containerID="072cb8658328198f8cbcffd250607a34d5f68d9460b697214d06fcf06a925974" Sep 17 07:36:19.674089 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.674046 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"072cb8658328198f8cbcffd250607a34d5f68d9460b697214d06fcf06a925974"} err="failed to get container status \"072cb8658328198f8cbcffd250607a34d5f68d9460b697214d06fcf06a925974\": rpc error: code = NotFound desc = could not find container \"072cb8658328198f8cbcffd250607a34d5f68d9460b697214d06fcf06a925974\": container with ID starting with 072cb8658328198f8cbcffd250607a34d5f68d9460b697214d06fcf06a925974 not found: ID does not exist" Sep 17 07:36:19.674089 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.674066 2563 scope.go:117] "RemoveContainer" containerID="d4d349d58ddcf35dc8107b9d3f1a64edd6357b76eea3b380286014155b1f21e4" Sep 17 07:36:19.674303 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:36:19.674282 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4d349d58ddcf35dc8107b9d3f1a64edd6357b76eea3b380286014155b1f21e4\": container with ID starting with d4d349d58ddcf35dc8107b9d3f1a64edd6357b76eea3b380286014155b1f21e4 not found: ID does not exist" containerID="d4d349d58ddcf35dc8107b9d3f1a64edd6357b76eea3b380286014155b1f21e4" Sep 17 07:36:19.674385 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.674306 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4d349d58ddcf35dc8107b9d3f1a64edd6357b76eea3b380286014155b1f21e4"} err="failed to get container status \"d4d349d58ddcf35dc8107b9d3f1a64edd6357b76eea3b380286014155b1f21e4\": rpc error: code = NotFound desc = could not find container \"d4d349d58ddcf35dc8107b9d3f1a64edd6357b76eea3b380286014155b1f21e4\": container with ID starting with d4d349d58ddcf35dc8107b9d3f1a64edd6357b76eea3b380286014155b1f21e4 not found: ID does not exist" Sep 17 07:36:19.674385 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.674341 2563 scope.go:117] "RemoveContainer" containerID="b948c0efcf7420d104bd9b370269bf0c687a9e6d2c5bc6801f5e04a24a404aa8" Sep 17 07:36:19.674620 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.674603 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b948c0efcf7420d104bd9b370269bf0c687a9e6d2c5bc6801f5e04a24a404aa8"} err="failed to get container status \"b948c0efcf7420d104bd9b370269bf0c687a9e6d2c5bc6801f5e04a24a404aa8\": rpc error: code = NotFound desc = could not find container \"b948c0efcf7420d104bd9b370269bf0c687a9e6d2c5bc6801f5e04a24a404aa8\": container with ID starting with b948c0efcf7420d104bd9b370269bf0c687a9e6d2c5bc6801f5e04a24a404aa8 not found: ID does not exist" Sep 17 07:36:19.674667 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.674621 2563 scope.go:117] "RemoveContainer" containerID="94280fe506c05d1926f832e3f99924c51e37f1731efffaa8b1bec56deb5ff95c" Sep 17 07:36:19.674837 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.674818 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94280fe506c05d1926f832e3f99924c51e37f1731efffaa8b1bec56deb5ff95c"} err="failed to get container status \"94280fe506c05d1926f832e3f99924c51e37f1731efffaa8b1bec56deb5ff95c\": rpc error: code = NotFound desc = could not find container \"94280fe506c05d1926f832e3f99924c51e37f1731efffaa8b1bec56deb5ff95c\": container with ID starting with 94280fe506c05d1926f832e3f99924c51e37f1731efffaa8b1bec56deb5ff95c not found: ID does not exist" Sep 17 07:36:19.674904 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.674839 2563 scope.go:117] "RemoveContainer" containerID="8525f3ea8a7ae5d0bdf200e925928e7e691b7a19e312d4d7f07c17bd7dfe022c" Sep 17 07:36:19.675070 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.675053 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8525f3ea8a7ae5d0bdf200e925928e7e691b7a19e312d4d7f07c17bd7dfe022c"} err="failed to get container status \"8525f3ea8a7ae5d0bdf200e925928e7e691b7a19e312d4d7f07c17bd7dfe022c\": rpc error: code = NotFound desc = could not find container \"8525f3ea8a7ae5d0bdf200e925928e7e691b7a19e312d4d7f07c17bd7dfe022c\": container with ID starting with 8525f3ea8a7ae5d0bdf200e925928e7e691b7a19e312d4d7f07c17bd7dfe022c not found: ID does not exist" Sep 17 07:36:19.675070 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.675069 2563 scope.go:117] "RemoveContainer" containerID="072cb8658328198f8cbcffd250607a34d5f68d9460b697214d06fcf06a925974" Sep 17 07:36:19.675281 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.675266 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"072cb8658328198f8cbcffd250607a34d5f68d9460b697214d06fcf06a925974"} err="failed to get container status \"072cb8658328198f8cbcffd250607a34d5f68d9460b697214d06fcf06a925974\": rpc error: code = NotFound desc = could not find container \"072cb8658328198f8cbcffd250607a34d5f68d9460b697214d06fcf06a925974\": container with ID starting with 072cb8658328198f8cbcffd250607a34d5f68d9460b697214d06fcf06a925974 not found: ID does not exist" Sep 17 07:36:19.675349 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.675282 2563 scope.go:117] "RemoveContainer" containerID="d4d349d58ddcf35dc8107b9d3f1a64edd6357b76eea3b380286014155b1f21e4" Sep 17 07:36:19.675496 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.675480 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4d349d58ddcf35dc8107b9d3f1a64edd6357b76eea3b380286014155b1f21e4"} err="failed to get container status \"d4d349d58ddcf35dc8107b9d3f1a64edd6357b76eea3b380286014155b1f21e4\": rpc error: code = NotFound desc = could not find container \"d4d349d58ddcf35dc8107b9d3f1a64edd6357b76eea3b380286014155b1f21e4\": container with ID starting with d4d349d58ddcf35dc8107b9d3f1a64edd6357b76eea3b380286014155b1f21e4 not found: ID does not exist" Sep 17 07:36:19.703809 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.703788 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhxk2\" (UniqueName: \"kubernetes.io/projected/d198d556-3626-4394-938d-2ca4266cec0c-kube-api-access-bhxk2\") pod \"d198d556-3626-4394-938d-2ca4266cec0c\" (UID: \"d198d556-3626-4394-938d-2ca4266cec0c\") " Sep 17 07:36:19.703905 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.703827 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d198d556-3626-4394-938d-2ca4266cec0c-feast-data\") pod \"d198d556-3626-4394-938d-2ca4266cec0c\" (UID: \"d198d556-3626-4394-938d-2ca4266cec0c\") " Sep 17 07:36:19.703905 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.703850 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d198d556-3626-4394-938d-2ca4266cec0c-offline-tls\") pod \"d198d556-3626-4394-938d-2ca4266cec0c\" (UID: \"d198d556-3626-4394-938d-2ca4266cec0c\") " Sep 17 07:36:19.703905 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.703871 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d198d556-3626-4394-938d-2ca4266cec0c-registry-tls\") pod \"d198d556-3626-4394-938d-2ca4266cec0c\" (UID: \"d198d556-3626-4394-938d-2ca4266cec0c\") " Sep 17 07:36:19.704059 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.703916 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d198d556-3626-4394-938d-2ca4266cec0c-online-tls\") pod \"d198d556-3626-4394-938d-2ca4266cec0c\" (UID: \"d198d556-3626-4394-938d-2ca4266cec0c\") " Sep 17 07:36:19.706019 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.705994 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d198d556-3626-4394-938d-2ca4266cec0c-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "d198d556-3626-4394-938d-2ca4266cec0c" (UID: "d198d556-3626-4394-938d-2ca4266cec0c"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:36:19.706107 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.706035 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d198d556-3626-4394-938d-2ca4266cec0c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d198d556-3626-4394-938d-2ca4266cec0c" (UID: "d198d556-3626-4394-938d-2ca4266cec0c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:36:19.706534 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.706505 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d198d556-3626-4394-938d-2ca4266cec0c-online-tls" (OuterVolumeSpecName: "online-tls") pod "d198d556-3626-4394-938d-2ca4266cec0c" (UID: "d198d556-3626-4394-938d-2ca4266cec0c"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:36:19.706534 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.706516 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d198d556-3626-4394-938d-2ca4266cec0c-kube-api-access-bhxk2" (OuterVolumeSpecName: "kube-api-access-bhxk2") pod "d198d556-3626-4394-938d-2ca4266cec0c" (UID: "d198d556-3626-4394-938d-2ca4266cec0c"). InnerVolumeSpecName "kube-api-access-bhxk2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:36:19.711998 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.711979 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d198d556-3626-4394-938d-2ca4266cec0c-feast-data" (OuterVolumeSpecName: "feast-data") pod "d198d556-3626-4394-938d-2ca4266cec0c" (UID: "d198d556-3626-4394-938d-2ca4266cec0c"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:36:19.804596 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.804571 2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d198d556-3626-4394-938d-2ca4266cec0c-feast-data\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:36:19.804596 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.804593 2563 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d198d556-3626-4394-938d-2ca4266cec0c-offline-tls\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:36:19.804703 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.804601 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d198d556-3626-4394-938d-2ca4266cec0c-registry-tls\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:36:19.804703 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.804609 2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d198d556-3626-4394-938d-2ca4266cec0c-online-tls\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:36:19.804703 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.804617 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhxk2\" (UniqueName: \"kubernetes.io/projected/d198d556-3626-4394-938d-2ca4266cec0c-kube-api-access-bhxk2\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:36:19.967771 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.967714 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d"] Sep 17 07:36:19.974424 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:19.974398 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-64475b79d6-xwk7d"] Sep 17 07:36:21.319932 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:36:21.319897 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d198d556-3626-4394-938d-2ca4266cec0c" path="/var/lib/kubelet/pods/d198d556-3626-4394-938d-2ca4266cec0c/volumes" Sep 17 07:38:39.010067 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.010036 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-7p2w2"] Sep 17 07:38:39.010557 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.010396 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="feast-init" Sep 17 07:38:39.010557 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.010409 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="feast-init" Sep 17 07:38:39.010557 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.010417 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="feast-apply" Sep 17 07:38:39.010557 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.010422 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="feast-apply" Sep 17 07:38:39.010557 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.010431 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="offline" Sep 17 07:38:39.010557 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.010437 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="offline" Sep 17 07:38:39.010557 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.010445 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c19cfb66-3f5a-4f63-bc92-bb1133e511c1" containerName="redis" Sep 17 07:38:39.010557 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.010450 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c19cfb66-3f5a-4f63-bc92-bb1133e511c1" containerName="redis" Sep 17 07:38:39.010557 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.010461 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="registry" Sep 17 07:38:39.010557 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.010465 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="registry" Sep 17 07:38:39.010557 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.010474 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="online" Sep 17 07:38:39.010557 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.010479 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="online" Sep 17 07:38:39.010557 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.010487 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7411ed1c-7274-4d6f-988f-8b22a4fc5065" containerName="postgres" Sep 17 07:38:39.010557 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.010492 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7411ed1c-7274-4d6f-988f-8b22a4fc5065" containerName="postgres" Sep 17 07:38:39.010557 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.010538 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c19cfb66-3f5a-4f63-bc92-bb1133e511c1" containerName="redis" Sep 17 07:38:39.010557 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.010547 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="online" Sep 17 07:38:39.010557 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.010552 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7411ed1c-7274-4d6f-988f-8b22a4fc5065" containerName="postgres" Sep 17 07:38:39.010557 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.010557 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="offline" Sep 17 07:38:39.011275 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.010574 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d198d556-3626-4394-938d-2ca4266cec0c" containerName="registry" Sep 17 07:38:39.013448 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.013431 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-7p2w2" Sep 17 07:38:39.020398 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.020377 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Sep 17 07:38:39.026099 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.021113 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Sep 17 07:38:39.026099 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.021134 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-gmtbd\"" Sep 17 07:38:39.063251 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.063216 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-7p2w2"] Sep 17 07:38:39.076538 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.076506 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkxsx\" (UniqueName: \"kubernetes.io/projected/1120c67b-9f79-4b27-b66c-2ad88d220ff6-kube-api-access-qkxsx\") pod \"redis-69bbd798b6-7p2w2\" (UID: \"1120c67b-9f79-4b27-b66c-2ad88d220ff6\") " pod="test-ns-feast-rest/redis-69bbd798b6-7p2w2" Sep 17 07:38:39.177642 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.177595 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkxsx\" (UniqueName: \"kubernetes.io/projected/1120c67b-9f79-4b27-b66c-2ad88d220ff6-kube-api-access-qkxsx\") pod \"redis-69bbd798b6-7p2w2\" (UID: \"1120c67b-9f79-4b27-b66c-2ad88d220ff6\") " pod="test-ns-feast-rest/redis-69bbd798b6-7p2w2" Sep 17 07:38:39.190792 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.190755 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkxsx\" (UniqueName: \"kubernetes.io/projected/1120c67b-9f79-4b27-b66c-2ad88d220ff6-kube-api-access-qkxsx\") pod \"redis-69bbd798b6-7p2w2\" (UID: \"1120c67b-9f79-4b27-b66c-2ad88d220ff6\") " pod="test-ns-feast-rest/redis-69bbd798b6-7p2w2" Sep 17 07:38:39.329395 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.329275 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-7p2w2" Sep 17 07:38:39.621636 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:39.621546 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-7p2w2"] Sep 17 07:38:39.625127 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:38:39.625099 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1120c67b_9f79_4b27_b66c_2ad88d220ff6.slice/crio-d00dce3bbb2d34c895210b6a886f6668d79d3741e907c66a7a16ed7826de6a67 WatchSource:0}: Error finding container d00dce3bbb2d34c895210b6a886f6668d79d3741e907c66a7a16ed7826de6a67: Status 404 returned error can't find the container with id d00dce3bbb2d34c895210b6a886f6668d79d3741e907c66a7a16ed7826de6a67 Sep 17 07:38:40.195248 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:40.195212 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-7p2w2" event={"ID":"1120c67b-9f79-4b27-b66c-2ad88d220ff6","Type":"ContainerStarted","Data":"b81ffc04b3d05ae435f6e62a10b4fce548b2fa3a6ffa365ff04db786389cf27d"} Sep 17 07:38:40.195248 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:40.195249 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-7p2w2" event={"ID":"1120c67b-9f79-4b27-b66c-2ad88d220ff6","Type":"ContainerStarted","Data":"d00dce3bbb2d34c895210b6a886f6668d79d3741e907c66a7a16ed7826de6a67"} Sep 17 07:38:44.467493 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:44.467439 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/redis-69bbd798b6-7p2w2" podStartSLOduration=6.467420883 podStartE2EDuration="6.467420883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:38:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:38:40.292462448 +0000 UTC m=+1127.478195419" watchObservedRunningTime="2026-09-17 07:38:44.467420883 +0000 UTC m=+1131.653153855" Sep 17 07:38:44.467967 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:44.467680 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-2hvkt"] Sep 17 07:38:44.470910 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:44.470890 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-2hvkt" Sep 17 07:38:44.475029 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:44.475006 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Sep 17 07:38:44.485171 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:44.485143 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-2hvkt"] Sep 17 07:38:44.524860 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:44.524823 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/99441f68-8328-4396-b011-b1a3eb720a3c-postgresdata\") pod \"postgres-864846d4f5-2hvkt\" (UID: \"99441f68-8328-4396-b011-b1a3eb720a3c\") " pod="test-ns-feast-rest/postgres-864846d4f5-2hvkt" Sep 17 07:38:44.524860 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:44.524864 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqzw6\" (UniqueName: \"kubernetes.io/projected/99441f68-8328-4396-b011-b1a3eb720a3c-kube-api-access-kqzw6\") pod \"postgres-864846d4f5-2hvkt\" (UID: \"99441f68-8328-4396-b011-b1a3eb720a3c\") " pod="test-ns-feast-rest/postgres-864846d4f5-2hvkt" Sep 17 07:38:44.625878 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:44.625841 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqzw6\" (UniqueName: \"kubernetes.io/projected/99441f68-8328-4396-b011-b1a3eb720a3c-kube-api-access-kqzw6\") pod \"postgres-864846d4f5-2hvkt\" (UID: \"99441f68-8328-4396-b011-b1a3eb720a3c\") " pod="test-ns-feast-rest/postgres-864846d4f5-2hvkt" Sep 17 07:38:44.626045 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:44.625994 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/99441f68-8328-4396-b011-b1a3eb720a3c-postgresdata\") pod \"postgres-864846d4f5-2hvkt\" (UID: \"99441f68-8328-4396-b011-b1a3eb720a3c\") " pod="test-ns-feast-rest/postgres-864846d4f5-2hvkt" Sep 17 07:38:44.626371 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:44.626351 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/99441f68-8328-4396-b011-b1a3eb720a3c-postgresdata\") pod \"postgres-864846d4f5-2hvkt\" (UID: \"99441f68-8328-4396-b011-b1a3eb720a3c\") " pod="test-ns-feast-rest/postgres-864846d4f5-2hvkt" Sep 17 07:38:44.636524 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:44.636497 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqzw6\" (UniqueName: \"kubernetes.io/projected/99441f68-8328-4396-b011-b1a3eb720a3c-kube-api-access-kqzw6\") pod \"postgres-864846d4f5-2hvkt\" (UID: \"99441f68-8328-4396-b011-b1a3eb720a3c\") " pod="test-ns-feast-rest/postgres-864846d4f5-2hvkt" Sep 17 07:38:44.779856 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:44.779760 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-2hvkt" Sep 17 07:38:45.066609 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:45.066522 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-2hvkt"] Sep 17 07:38:45.069272 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:38:45.069247 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99441f68_8328_4396_b011_b1a3eb720a3c.slice/crio-9e41da0e21bcab4e117a690aec73ff57d0896ee91557b8996134267d842d4e25 WatchSource:0}: Error finding container 9e41da0e21bcab4e117a690aec73ff57d0896ee91557b8996134267d842d4e25: Status 404 returned error can't find the container with id 9e41da0e21bcab4e117a690aec73ff57d0896ee91557b8996134267d842d4e25 Sep 17 07:38:45.216038 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:45.215992 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-2hvkt" event={"ID":"99441f68-8328-4396-b011-b1a3eb720a3c","Type":"ContainerStarted","Data":"bf7595bba72712345286ead24d51bbb227be046649b06bb6ada77f98a5d8107a"} Sep 17 07:38:45.216038 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:45.216045 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-2hvkt" event={"ID":"99441f68-8328-4396-b011-b1a3eb720a3c","Type":"ContainerStarted","Data":"9e41da0e21bcab4e117a690aec73ff57d0896ee91557b8996134267d842d4e25"} Sep 17 07:38:50.351936 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:50.351874 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-2hvkt" podStartSLOduration=6.351854961 podStartE2EDuration="6.351854961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:38:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:38:45.243501161 +0000 UTC m=+1132.429234134" watchObservedRunningTime="2026-09-17 07:38:50.351854961 +0000 UTC m=+1137.537587931" Sep 17 07:38:50.352892 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:50.352873 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj"] Sep 17 07:38:50.360841 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:50.360811 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:38:50.367988 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:50.367963 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\"" Sep 17 07:38:50.368281 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:50.368256 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Sep 17 07:38:50.368366 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:50.368344 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-fvr8m\"" Sep 17 07:38:50.368433 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:50.368344 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\"" Sep 17 07:38:50.463158 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:50.463129 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj"] Sep 17 07:38:50.480309 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:50.480275 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-feast-data\") pod \"feast-credit-scoring-bdbc68465-8vnvj\" (UID: \"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d\") " pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:38:50.480309 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:50.480308 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-registry-tls\") pod \"feast-credit-scoring-bdbc68465-8vnvj\" (UID: \"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d\") " pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:38:50.480515 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:50.480357 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhcp2\" (UniqueName: \"kubernetes.io/projected/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-kube-api-access-dhcp2\") pod \"feast-credit-scoring-bdbc68465-8vnvj\" (UID: \"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d\") " pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:38:50.480515 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:50.480399 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-online-tls\") pod \"feast-credit-scoring-bdbc68465-8vnvj\" (UID: \"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d\") " pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:38:50.581622 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:50.581579 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-feast-data\") pod \"feast-credit-scoring-bdbc68465-8vnvj\" (UID: \"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d\") " pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:38:50.581622 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:50.581624 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-registry-tls\") pod \"feast-credit-scoring-bdbc68465-8vnvj\" (UID: \"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d\") " pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:38:50.581891 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:50.581650 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhcp2\" (UniqueName: \"kubernetes.io/projected/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-kube-api-access-dhcp2\") pod \"feast-credit-scoring-bdbc68465-8vnvj\" (UID: \"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d\") " pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:38:50.581891 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:50.581689 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-online-tls\") pod \"feast-credit-scoring-bdbc68465-8vnvj\" (UID: \"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d\") " pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:38:50.581891 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:38:50.581851 2563 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-credit-scoring-online-tls: secret "feast-credit-scoring-online-tls" not found Sep 17 07:38:50.582053 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:38:50.581915 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-online-tls podName:a6ab7cda-50f7-45b0-8d7b-33d6ab22809d nodeName:}" failed. No retries permitted until 2026-09-17 07:38:51.081895472 +0000 UTC m=+1138.267628437 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-online-tls") pod "feast-credit-scoring-bdbc68465-8vnvj" (UID: "a6ab7cda-50f7-45b0-8d7b-33d6ab22809d") : secret "feast-credit-scoring-online-tls" not found Sep 17 07:38:50.582111 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:50.582056 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-feast-data\") pod \"feast-credit-scoring-bdbc68465-8vnvj\" (UID: \"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d\") " pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:38:50.584203 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:50.584184 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-registry-tls\") pod \"feast-credit-scoring-bdbc68465-8vnvj\" (UID: \"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d\") " pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:38:50.596989 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:50.596963 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhcp2\" (UniqueName: \"kubernetes.io/projected/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-kube-api-access-dhcp2\") pod \"feast-credit-scoring-bdbc68465-8vnvj\" (UID: \"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d\") " pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:38:51.087182 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:51.087148 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-online-tls\") pod \"feast-credit-scoring-bdbc68465-8vnvj\" (UID: \"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d\") " pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:38:51.089814 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:51.089789 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-online-tls\") pod \"feast-credit-scoring-bdbc68465-8vnvj\" (UID: \"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d\") " pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:38:51.270674 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:51.270634 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:38:51.561740 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:51.561698 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj"] Sep 17 07:38:51.563893 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:38:51.563865 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6ab7cda_50f7_45b0_8d7b_33d6ab22809d.slice/crio-0caa88c0c2532dd586bd83789176b3be61a1a937f33b01f1cc79a80d8de23a98 WatchSource:0}: Error finding container 0caa88c0c2532dd586bd83789176b3be61a1a937f33b01f1cc79a80d8de23a98: Status 404 returned error can't find the container with id 0caa88c0c2532dd586bd83789176b3be61a1a937f33b01f1cc79a80d8de23a98 Sep 17 07:38:52.245916 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:52.245879 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" event={"ID":"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d","Type":"ContainerStarted","Data":"e601397e76894f0a6c18ecaabda996d09303fc1842fb3a3aafea73f0d920416f"} Sep 17 07:38:52.246092 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:52.245935 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" event={"ID":"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d","Type":"ContainerStarted","Data":"0caa88c0c2532dd586bd83789176b3be61a1a937f33b01f1cc79a80d8de23a98"} Sep 17 07:38:53.251105 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:53.251071 2563 generic.go:356] "Generic (PLEG): container finished" podID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" containerID="e601397e76894f0a6c18ecaabda996d09303fc1842fb3a3aafea73f0d920416f" exitCode=0 Sep 17 07:38:53.251520 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:53.251161 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" event={"ID":"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d","Type":"ContainerDied","Data":"e601397e76894f0a6c18ecaabda996d09303fc1842fb3a3aafea73f0d920416f"} Sep 17 07:38:54.257401 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:38:54.257368 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" event={"ID":"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d","Type":"ContainerStarted","Data":"db2843e1d730267a8b07694f34469453a9b8986135e5723f57800595aec3371b"} Sep 17 07:39:00.288955 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:00.288922 2563 generic.go:356] "Generic (PLEG): container finished" podID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" containerID="db2843e1d730267a8b07694f34469453a9b8986135e5723f57800595aec3371b" exitCode=0 Sep 17 07:39:00.289294 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:00.289000 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" event={"ID":"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d","Type":"ContainerDied","Data":"db2843e1d730267a8b07694f34469453a9b8986135e5723f57800595aec3371b"} Sep 17 07:39:00.294390 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:00.294372 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Sep 17 07:39:01.299196 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:01.299155 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" event={"ID":"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d","Type":"ContainerStarted","Data":"9227e3b55b9ad4036dbf7a80f229c2135f26367c466f6ba75fbc34099608005d"} Sep 17 07:39:01.299595 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:01.299243 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" event={"ID":"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d","Type":"ContainerStarted","Data":"454e3e24a8b0d80f72451a0c6727f58aac74a3f99561e9fdf3e468d025f265b4"} Sep 17 07:39:01.340182 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:01.340123 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" podStartSLOduration=11.340102693 podStartE2EDuration="11.340102693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:38:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:39:01.338385548 +0000 UTC m=+1148.524118523" watchObservedRunningTime="2026-09-17 07:39:01.340102693 +0000 UTC m=+1148.525835666" Sep 17 07:39:03.271714 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:03.271670 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:39:03.272091 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:03.271745 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:39:03.273219 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:03.273184 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" podUID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.36:6571: connect: connection refused" Sep 17 07:39:03.273359 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:03.273224 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" podUID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.36:6567/health\": dial tcp 10.133.0.36:6567: connect: connection refused" Sep 17 07:39:06.271790 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:06.271741 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" podUID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.36:6571: connect: connection refused" Sep 17 07:39:06.271790 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:06.271772 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" podUID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.36:6567/health\": dial tcp 10.133.0.36:6567: connect: connection refused" Sep 17 07:39:09.271752 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:09.271694 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" podUID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.36:6571: connect: connection refused" Sep 17 07:39:09.277658 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:09.277631 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:39:09.278034 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:09.278013 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:39:09.280704 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:09.280684 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:39:11.271078 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:11.271046 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:39:12.272213 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:12.272185 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:39:12.348380 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:12.348339 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:39:16.091570 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:16.091486 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j"] Sep 17 07:39:16.096146 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:16.096121 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:39:16.100222 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:16.100202 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\"" Sep 17 07:39:16.100463 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:16.100446 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\"" Sep 17 07:39:16.100579 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:16.100560 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-q4mlt\"" Sep 17 07:39:16.101006 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:16.100988 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Sep 17 07:39:16.118606 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:16.118575 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j"] Sep 17 07:39:16.205817 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:16.205786 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0415409e-10f3-4cf1-8bfb-a40a0e093106-registry-tls\") pod \"feast-driver-ranking-597785cc78-wx96j\" (UID: \"0415409e-10f3-4cf1-8bfb-a40a0e093106\") " pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:39:16.205953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:16.205822 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mth96\" (UniqueName: \"kubernetes.io/projected/0415409e-10f3-4cf1-8bfb-a40a0e093106-kube-api-access-mth96\") pod \"feast-driver-ranking-597785cc78-wx96j\" (UID: \"0415409e-10f3-4cf1-8bfb-a40a0e093106\") " pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:39:16.205953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:16.205863 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0415409e-10f3-4cf1-8bfb-a40a0e093106-online-tls\") pod \"feast-driver-ranking-597785cc78-wx96j\" (UID: \"0415409e-10f3-4cf1-8bfb-a40a0e093106\") " pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:39:16.205953 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:16.205886 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0415409e-10f3-4cf1-8bfb-a40a0e093106-feast-data\") pod \"feast-driver-ranking-597785cc78-wx96j\" (UID: \"0415409e-10f3-4cf1-8bfb-a40a0e093106\") " pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:39:16.306663 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:16.306638 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0415409e-10f3-4cf1-8bfb-a40a0e093106-registry-tls\") pod \"feast-driver-ranking-597785cc78-wx96j\" (UID: \"0415409e-10f3-4cf1-8bfb-a40a0e093106\") " pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:39:16.306809 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:16.306669 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mth96\" (UniqueName: \"kubernetes.io/projected/0415409e-10f3-4cf1-8bfb-a40a0e093106-kube-api-access-mth96\") pod \"feast-driver-ranking-597785cc78-wx96j\" (UID: \"0415409e-10f3-4cf1-8bfb-a40a0e093106\") " pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:39:16.306809 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:16.306695 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0415409e-10f3-4cf1-8bfb-a40a0e093106-online-tls\") pod \"feast-driver-ranking-597785cc78-wx96j\" (UID: \"0415409e-10f3-4cf1-8bfb-a40a0e093106\") " pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:39:16.306809 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:16.306718 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0415409e-10f3-4cf1-8bfb-a40a0e093106-feast-data\") pod \"feast-driver-ranking-597785cc78-wx96j\" (UID: \"0415409e-10f3-4cf1-8bfb-a40a0e093106\") " pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:39:16.307142 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:16.307124 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0415409e-10f3-4cf1-8bfb-a40a0e093106-feast-data\") pod \"feast-driver-ranking-597785cc78-wx96j\" (UID: \"0415409e-10f3-4cf1-8bfb-a40a0e093106\") " pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:39:16.309245 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:16.309224 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0415409e-10f3-4cf1-8bfb-a40a0e093106-online-tls\") pod \"feast-driver-ranking-597785cc78-wx96j\" (UID: \"0415409e-10f3-4cf1-8bfb-a40a0e093106\") " pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:39:16.309353 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:16.309263 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0415409e-10f3-4cf1-8bfb-a40a0e093106-registry-tls\") pod \"feast-driver-ranking-597785cc78-wx96j\" (UID: \"0415409e-10f3-4cf1-8bfb-a40a0e093106\") " pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:39:16.317172 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:16.317150 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mth96\" (UniqueName: \"kubernetes.io/projected/0415409e-10f3-4cf1-8bfb-a40a0e093106-kube-api-access-mth96\") pod \"feast-driver-ranking-597785cc78-wx96j\" (UID: \"0415409e-10f3-4cf1-8bfb-a40a0e093106\") " pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:39:16.408000 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:16.407936 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:39:16.690945 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:16.690916 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j"] Sep 17 07:39:16.693391 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:39:16.693373 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0415409e_10f3_4cf1_8bfb_a40a0e093106.slice/crio-65cac1f9a23955e0d535ceb906e1c6678d845997df855db6936f94f5d15eddfe WatchSource:0}: Error finding container 65cac1f9a23955e0d535ceb906e1c6678d845997df855db6936f94f5d15eddfe: Status 404 returned error can't find the container with id 65cac1f9a23955e0d535ceb906e1c6678d845997df855db6936f94f5d15eddfe Sep 17 07:39:17.372524 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:17.372490 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" event={"ID":"0415409e-10f3-4cf1-8bfb-a40a0e093106","Type":"ContainerStarted","Data":"ba9fe431695de5ffb88807464efa47b659d9300217aad7ef05dbab29d3d22389"} Sep 17 07:39:17.372920 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:17.372544 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" event={"ID":"0415409e-10f3-4cf1-8bfb-a40a0e093106","Type":"ContainerStarted","Data":"65cac1f9a23955e0d535ceb906e1c6678d845997df855db6936f94f5d15eddfe"} Sep 17 07:39:22.393434 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:22.393402 2563 generic.go:356] "Generic (PLEG): container finished" podID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerID="ba9fe431695de5ffb88807464efa47b659d9300217aad7ef05dbab29d3d22389" exitCode=0 Sep 17 07:39:22.393923 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:22.393484 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" event={"ID":"0415409e-10f3-4cf1-8bfb-a40a0e093106","Type":"ContainerDied","Data":"ba9fe431695de5ffb88807464efa47b659d9300217aad7ef05dbab29d3d22389"} Sep 17 07:39:23.400873 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:23.400837 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" event={"ID":"0415409e-10f3-4cf1-8bfb-a40a0e093106","Type":"ContainerStarted","Data":"a6e9e76052a365cb346ce3e78e64e894f1ca2081f0ecd7e1a5dc07eaf4689eb7"} Sep 17 07:39:29.427691 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:29.427628 2563 generic.go:356] "Generic (PLEG): container finished" podID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerID="a6e9e76052a365cb346ce3e78e64e894f1ca2081f0ecd7e1a5dc07eaf4689eb7" exitCode=0 Sep 17 07:39:29.428009 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:29.427702 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" event={"ID":"0415409e-10f3-4cf1-8bfb-a40a0e093106","Type":"ContainerDied","Data":"a6e9e76052a365cb346ce3e78e64e894f1ca2081f0ecd7e1a5dc07eaf4689eb7"} Sep 17 07:39:30.438873 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:30.438831 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" event={"ID":"0415409e-10f3-4cf1-8bfb-a40a0e093106","Type":"ContainerStarted","Data":"b5144b3a85ef09f74e9aad9b65b8ad7b57f73877df2e63f40c5100b2365d8a5d"} Sep 17 07:39:30.438873 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:30.438882 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" event={"ID":"0415409e-10f3-4cf1-8bfb-a40a0e093106","Type":"ContainerStarted","Data":"1a3ba1af967809bb5cc9772feae99ad95692a7ad34d0fbfb1677158c79781531"} Sep 17 07:39:30.474010 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:30.473951 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" podStartSLOduration=14.473931144 podStartE2EDuration="14.473931144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:39:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:39:30.473054626 +0000 UTC m=+1177.658787600" watchObservedRunningTime="2026-09-17 07:39:30.473931144 +0000 UTC m=+1177.659664117" Sep 17 07:39:31.409149 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:31.409119 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:39:31.409317 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:31.409175 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:39:31.410674 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:31.410644 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" podUID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.37:6571: connect: connection refused" Sep 17 07:39:31.410813 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:31.410693 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" podUID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerName="online" probeResult="failure" output="Get \"https://10.133.0.37:6567/health\": dial tcp 10.133.0.37:6567: connect: connection refused" Sep 17 07:39:34.409157 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:34.409101 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" podUID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerName="online" probeResult="failure" output="Get \"https://10.133.0.37:6567/health\": dial tcp 10.133.0.37:6567: connect: connection refused" Sep 17 07:39:34.409691 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:34.409106 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" podUID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.37:6571: connect: connection refused" Sep 17 07:39:36.408799 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:36.408763 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:39:36.408799 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:36.408823 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:39:37.408704 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:37.408663 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" podUID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.37:6571: connect: connection refused" Sep 17 07:39:37.417237 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:37.417210 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:39:37.421615 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:37.421590 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:39:40.409215 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:40.409159 2563 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" podUID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.37:6571: connect: connection refused" Sep 17 07:39:43.408645 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:43.408614 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:39:43.409375 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:39:43.409356 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:41:04.231644 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:04.231612 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-8699f9b4c7-dpjp8"] Sep 17 07:41:04.232062 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:04.231859 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-8699f9b4c7-dpjp8" podUID="96e0c094-1439-4724-bfa0-f0c3c79f0a17" containerName="manager" containerID="cri-o://3231cae819af892886f0f65f247092ad361229da6fb8d7a95099bb85a75d7257" gracePeriod=10 Sep 17 07:41:04.325660 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:04.325629 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="feast-operator-system/feast-operator-controller-manager-8699f9b4c7-dpjp8" podUID="96e0c094-1439-4724-bfa0-f0c3c79f0a17" containerName="manager" probeResult="failure" output="Get \"http://10.133.0.27:8081/readyz\": dial tcp 10.133.0.27:8081: connect: connection refused" Sep 17 07:41:04.470757 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:04.470735 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-8699f9b4c7-dpjp8" Sep 17 07:41:04.582575 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:04.582498 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqm7f\" (UniqueName: \"kubernetes.io/projected/96e0c094-1439-4724-bfa0-f0c3c79f0a17-kube-api-access-cqm7f\") pod \"96e0c094-1439-4724-bfa0-f0c3c79f0a17\" (UID: \"96e0c094-1439-4724-bfa0-f0c3c79f0a17\") " Sep 17 07:41:04.584849 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:04.584827 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96e0c094-1439-4724-bfa0-f0c3c79f0a17-kube-api-access-cqm7f" (OuterVolumeSpecName: "kube-api-access-cqm7f") pod "96e0c094-1439-4724-bfa0-f0c3c79f0a17" (UID: "96e0c094-1439-4724-bfa0-f0c3c79f0a17"). InnerVolumeSpecName "kube-api-access-cqm7f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:41:04.684150 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:04.684118 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqm7f\" (UniqueName: \"kubernetes.io/projected/96e0c094-1439-4724-bfa0-f0c3c79f0a17-kube-api-access-cqm7f\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:41:04.788175 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:04.788134 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j"] Sep 17 07:41:04.788699 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:04.788668 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" podUID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerName="registry" containerID="cri-o://1a3ba1af967809bb5cc9772feae99ad95692a7ad34d0fbfb1677158c79781531" gracePeriod=30 Sep 17 07:41:04.788916 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:04.788728 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" podUID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerName="online" containerID="cri-o://b5144b3a85ef09f74e9aad9b65b8ad7b57f73877df2e63f40c5100b2365d8a5d" gracePeriod=30 Sep 17 07:41:04.790162 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:04.790114 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj"] Sep 17 07:41:04.790363 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:04.790307 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" podUID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" containerName="registry" containerID="cri-o://454e3e24a8b0d80f72451a0c6727f58aac74a3f99561e9fdf3e468d025f265b4" gracePeriod=30 Sep 17 07:41:04.790467 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:04.790426 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" podUID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" containerName="online" containerID="cri-o://9227e3b55b9ad4036dbf7a80f229c2135f26367c466f6ba75fbc34099608005d" gracePeriod=30 Sep 17 07:41:04.853933 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:04.853858 2563 generic.go:356] "Generic (PLEG): container finished" podID="96e0c094-1439-4724-bfa0-f0c3c79f0a17" containerID="3231cae819af892886f0f65f247092ad361229da6fb8d7a95099bb85a75d7257" exitCode=0 Sep 17 07:41:04.853933 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:04.853908 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-8699f9b4c7-dpjp8" event={"ID":"96e0c094-1439-4724-bfa0-f0c3c79f0a17","Type":"ContainerDied","Data":"3231cae819af892886f0f65f247092ad361229da6fb8d7a95099bb85a75d7257"} Sep 17 07:41:04.853933 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:04.853912 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-8699f9b4c7-dpjp8" Sep 17 07:41:04.854153 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:04.853942 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-8699f9b4c7-dpjp8" event={"ID":"96e0c094-1439-4724-bfa0-f0c3c79f0a17","Type":"ContainerDied","Data":"5f74e8dd62e36d4d811e230a3605d7d55a8ec797f6e130b489bd033d181d110a"} Sep 17 07:41:04.854153 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:04.853964 2563 scope.go:117] "RemoveContainer" containerID="3231cae819af892886f0f65f247092ad361229da6fb8d7a95099bb85a75d7257" Sep 17 07:41:04.864786 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:04.864766 2563 scope.go:117] "RemoveContainer" containerID="3231cae819af892886f0f65f247092ad361229da6fb8d7a95099bb85a75d7257" Sep 17 07:41:04.865173 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:41:04.865154 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3231cae819af892886f0f65f247092ad361229da6fb8d7a95099bb85a75d7257\": container with ID starting with 3231cae819af892886f0f65f247092ad361229da6fb8d7a95099bb85a75d7257 not found: ID does not exist" containerID="3231cae819af892886f0f65f247092ad361229da6fb8d7a95099bb85a75d7257" Sep 17 07:41:04.865236 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:04.865188 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3231cae819af892886f0f65f247092ad361229da6fb8d7a95099bb85a75d7257"} err="failed to get container status \"3231cae819af892886f0f65f247092ad361229da6fb8d7a95099bb85a75d7257\": rpc error: code = NotFound desc = could not find container \"3231cae819af892886f0f65f247092ad361229da6fb8d7a95099bb85a75d7257\": container with ID starting with 3231cae819af892886f0f65f247092ad361229da6fb8d7a95099bb85a75d7257 not found: ID does not exist" Sep 17 07:41:04.881069 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:04.881042 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-8699f9b4c7-dpjp8"] Sep 17 07:41:04.885348 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:04.885314 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-8699f9b4c7-dpjp8"] Sep 17 07:41:05.316887 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:05.316855 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96e0c094-1439-4724-bfa0-f0c3c79f0a17" path="/var/lib/kubelet/pods/96e0c094-1439-4724-bfa0-f0c3c79f0a17/volumes" Sep 17 07:41:05.860507 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:05.860436 2563 generic.go:356] "Generic (PLEG): container finished" podID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" containerID="9227e3b55b9ad4036dbf7a80f229c2135f26367c466f6ba75fbc34099608005d" exitCode=0 Sep 17 07:41:05.860628 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:05.860508 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" event={"ID":"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d","Type":"ContainerDied","Data":"9227e3b55b9ad4036dbf7a80f229c2135f26367c466f6ba75fbc34099608005d"} Sep 17 07:41:05.863781 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:05.863760 2563 generic.go:356] "Generic (PLEG): container finished" podID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerID="b5144b3a85ef09f74e9aad9b65b8ad7b57f73877df2e63f40c5100b2365d8a5d" exitCode=0 Sep 17 07:41:05.863884 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:05.863808 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" event={"ID":"0415409e-10f3-4cf1-8bfb-a40a0e093106","Type":"ContainerDied","Data":"b5144b3a85ef09f74e9aad9b65b8ad7b57f73877df2e63f40c5100b2365d8a5d"} Sep 17 07:41:06.731228 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:06.731189 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-2hvkt"] Sep 17 07:41:06.731694 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:06.731426 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-2hvkt" podUID="99441f68-8328-4396-b011-b1a3eb720a3c" containerName="postgres" containerID="cri-o://bf7595bba72712345286ead24d51bbb227be046649b06bb6ada77f98a5d8107a" gracePeriod=30 Sep 17 07:41:06.738349 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:06.738309 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-7p2w2"] Sep 17 07:41:06.738685 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:06.738659 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/redis-69bbd798b6-7p2w2" podUID="1120c67b-9f79-4b27-b66c-2ad88d220ff6" containerName="redis" containerID="cri-o://b81ffc04b3d05ae435f6e62a10b4fce548b2fa3a6ffa365ff04db786389cf27d" gracePeriod=30 Sep 17 07:41:06.869849 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:06.869821 2563 generic.go:356] "Generic (PLEG): container finished" podID="1120c67b-9f79-4b27-b66c-2ad88d220ff6" containerID="b81ffc04b3d05ae435f6e62a10b4fce548b2fa3a6ffa365ff04db786389cf27d" exitCode=0 Sep 17 07:41:06.869987 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:06.869893 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-7p2w2" event={"ID":"1120c67b-9f79-4b27-b66c-2ad88d220ff6","Type":"ContainerDied","Data":"b81ffc04b3d05ae435f6e62a10b4fce548b2fa3a6ffa365ff04db786389cf27d"} Sep 17 07:41:06.871578 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:06.871557 2563 generic.go:356] "Generic (PLEG): container finished" podID="99441f68-8328-4396-b011-b1a3eb720a3c" containerID="bf7595bba72712345286ead24d51bbb227be046649b06bb6ada77f98a5d8107a" exitCode=0 Sep 17 07:41:06.871688 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:06.871633 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-2hvkt" event={"ID":"99441f68-8328-4396-b011-b1a3eb720a3c","Type":"ContainerDied","Data":"bf7595bba72712345286ead24d51bbb227be046649b06bb6ada77f98a5d8107a"} Sep 17 07:41:06.989864 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:06.989841 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-2hvkt" Sep 17 07:41:06.992637 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:06.992621 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-7p2w2" Sep 17 07:41:07.103920 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:07.103886 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqzw6\" (UniqueName: \"kubernetes.io/projected/99441f68-8328-4396-b011-b1a3eb720a3c-kube-api-access-kqzw6\") pod \"99441f68-8328-4396-b011-b1a3eb720a3c\" (UID: \"99441f68-8328-4396-b011-b1a3eb720a3c\") " Sep 17 07:41:07.104079 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:07.104004 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/99441f68-8328-4396-b011-b1a3eb720a3c-postgresdata\") pod \"99441f68-8328-4396-b011-b1a3eb720a3c\" (UID: \"99441f68-8328-4396-b011-b1a3eb720a3c\") " Sep 17 07:41:07.104079 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:07.104040 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkxsx\" (UniqueName: \"kubernetes.io/projected/1120c67b-9f79-4b27-b66c-2ad88d220ff6-kube-api-access-qkxsx\") pod \"1120c67b-9f79-4b27-b66c-2ad88d220ff6\" (UID: \"1120c67b-9f79-4b27-b66c-2ad88d220ff6\") " Sep 17 07:41:07.106170 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:07.106142 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99441f68-8328-4396-b011-b1a3eb720a3c-kube-api-access-kqzw6" (OuterVolumeSpecName: "kube-api-access-kqzw6") pod "99441f68-8328-4396-b011-b1a3eb720a3c" (UID: "99441f68-8328-4396-b011-b1a3eb720a3c"). InnerVolumeSpecName "kube-api-access-kqzw6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:41:07.106294 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:07.106180 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1120c67b-9f79-4b27-b66c-2ad88d220ff6-kube-api-access-qkxsx" (OuterVolumeSpecName: "kube-api-access-qkxsx") pod "1120c67b-9f79-4b27-b66c-2ad88d220ff6" (UID: "1120c67b-9f79-4b27-b66c-2ad88d220ff6"). InnerVolumeSpecName "kube-api-access-qkxsx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:41:07.133301 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:07.133270 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99441f68-8328-4396-b011-b1a3eb720a3c-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "99441f68-8328-4396-b011-b1a3eb720a3c" (UID: "99441f68-8328-4396-b011-b1a3eb720a3c"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:41:07.204961 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:07.204938 2563 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/99441f68-8328-4396-b011-b1a3eb720a3c-postgresdata\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:41:07.204961 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:07.204962 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkxsx\" (UniqueName: \"kubernetes.io/projected/1120c67b-9f79-4b27-b66c-2ad88d220ff6-kube-api-access-qkxsx\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:41:07.205110 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:07.204973 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqzw6\" (UniqueName: \"kubernetes.io/projected/99441f68-8328-4396-b011-b1a3eb720a3c-kube-api-access-kqzw6\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:41:07.418131 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:07.418095 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" podUID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerName="online" probeResult="failure" output="Get \"https://10.133.0.37:6567/health\": dial tcp 10.133.0.37:6567: connect: connection refused" Sep 17 07:41:07.877169 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:07.877138 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-7p2w2" Sep 17 07:41:07.877539 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:07.877141 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-7p2w2" event={"ID":"1120c67b-9f79-4b27-b66c-2ad88d220ff6","Type":"ContainerDied","Data":"d00dce3bbb2d34c895210b6a886f6668d79d3741e907c66a7a16ed7826de6a67"} Sep 17 07:41:07.877539 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:07.877260 2563 scope.go:117] "RemoveContainer" containerID="b81ffc04b3d05ae435f6e62a10b4fce548b2fa3a6ffa365ff04db786389cf27d" Sep 17 07:41:07.878667 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:07.878644 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-2hvkt" event={"ID":"99441f68-8328-4396-b011-b1a3eb720a3c","Type":"ContainerDied","Data":"9e41da0e21bcab4e117a690aec73ff57d0896ee91557b8996134267d842d4e25"} Sep 17 07:41:07.878667 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:07.878660 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-2hvkt" Sep 17 07:41:07.885616 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:07.885601 2563 scope.go:117] "RemoveContainer" containerID="bf7595bba72712345286ead24d51bbb227be046649b06bb6ada77f98a5d8107a" Sep 17 07:41:07.925213 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:07.925188 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-7p2w2"] Sep 17 07:41:07.935019 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:07.935001 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-7p2w2"] Sep 17 07:41:07.955268 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:07.955247 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-2hvkt"] Sep 17 07:41:07.980838 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:07.980816 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-2hvkt"] Sep 17 07:41:09.278791 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:09.278745 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" podUID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.36:6567/health\": dial tcp 10.133.0.36:6567: connect: connection refused" Sep 17 07:41:09.313146 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:09.313112 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1120c67b-9f79-4b27-b66c-2ad88d220ff6" path="/var/lib/kubelet/pods/1120c67b-9f79-4b27-b66c-2ad88d220ff6/volumes" Sep 17 07:41:09.313536 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:09.313517 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99441f68-8328-4396-b011-b1a3eb720a3c" path="/var/lib/kubelet/pods/99441f68-8328-4396-b011-b1a3eb720a3c/volumes" Sep 17 07:41:17.418611 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:17.418565 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" podUID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerName="online" probeResult="failure" output="Get \"https://10.133.0.37:6567/health\": dial tcp 10.133.0.37:6567: connect: connection refused" Sep 17 07:41:19.278797 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:19.278759 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" podUID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.36:6567/health\": dial tcp 10.133.0.36:6567: connect: connection refused" Sep 17 07:41:27.418549 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:27.418511 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" podUID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerName="online" probeResult="failure" output="Get \"https://10.133.0.37:6567/health\": dial tcp 10.133.0.37:6567: connect: connection refused" Sep 17 07:41:27.418982 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:27.418637 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:41:29.278206 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:29.278159 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" podUID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" containerName="online" probeResult="failure" output="Get \"https://10.133.0.36:6567/health\": dial tcp 10.133.0.36:6567: connect: connection refused" Sep 17 07:41:29.278603 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:29.278294 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:41:35.001692 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.001661 2563 generic.go:356] "Generic (PLEG): container finished" podID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerID="1a3ba1af967809bb5cc9772feae99ad95692a7ad34d0fbfb1677158c79781531" exitCode=137 Sep 17 07:41:35.002074 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.001730 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" event={"ID":"0415409e-10f3-4cf1-8bfb-a40a0e093106","Type":"ContainerDied","Data":"1a3ba1af967809bb5cc9772feae99ad95692a7ad34d0fbfb1677158c79781531"} Sep 17 07:41:35.004258 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.004236 2563 generic.go:356] "Generic (PLEG): container finished" podID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" containerID="454e3e24a8b0d80f72451a0c6727f58aac74a3f99561e9fdf3e468d025f265b4" exitCode=137 Sep 17 07:41:35.004381 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.004304 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" event={"ID":"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d","Type":"ContainerDied","Data":"454e3e24a8b0d80f72451a0c6727f58aac74a3f99561e9fdf3e468d025f265b4"} Sep 17 07:41:35.465039 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.465018 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:41:35.467767 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.467751 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:41:35.517136 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.517112 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-registry-tls\") pod \"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d\" (UID: \"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d\") " Sep 17 07:41:35.517268 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.517147 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0415409e-10f3-4cf1-8bfb-a40a0e093106-online-tls\") pod \"0415409e-10f3-4cf1-8bfb-a40a0e093106\" (UID: \"0415409e-10f3-4cf1-8bfb-a40a0e093106\") " Sep 17 07:41:35.517268 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.517163 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0415409e-10f3-4cf1-8bfb-a40a0e093106-feast-data\") pod \"0415409e-10f3-4cf1-8bfb-a40a0e093106\" (UID: \"0415409e-10f3-4cf1-8bfb-a40a0e093106\") " Sep 17 07:41:35.517268 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.517221 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0415409e-10f3-4cf1-8bfb-a40a0e093106-registry-tls\") pod \"0415409e-10f3-4cf1-8bfb-a40a0e093106\" (UID: \"0415409e-10f3-4cf1-8bfb-a40a0e093106\") " Sep 17 07:41:35.517268 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.517247 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mth96\" (UniqueName: \"kubernetes.io/projected/0415409e-10f3-4cf1-8bfb-a40a0e093106-kube-api-access-mth96\") pod \"0415409e-10f3-4cf1-8bfb-a40a0e093106\" (UID: \"0415409e-10f3-4cf1-8bfb-a40a0e093106\") " Sep 17 07:41:35.517512 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.517292 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhcp2\" (UniqueName: \"kubernetes.io/projected/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-kube-api-access-dhcp2\") pod \"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d\" (UID: \"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d\") " Sep 17 07:41:35.517512 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.517310 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-feast-data\") pod \"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d\" (UID: \"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d\") " Sep 17 07:41:35.517512 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.517361 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-online-tls\") pod \"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d\" (UID: \"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d\") " Sep 17 07:41:35.518143 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.518044 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0415409e-10f3-4cf1-8bfb-a40a0e093106-feast-data" (OuterVolumeSpecName: "feast-data") pod "0415409e-10f3-4cf1-8bfb-a40a0e093106" (UID: "0415409e-10f3-4cf1-8bfb-a40a0e093106"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:41:35.520088 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.520057 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0415409e-10f3-4cf1-8bfb-a40a0e093106-online-tls" (OuterVolumeSpecName: "online-tls") pod "0415409e-10f3-4cf1-8bfb-a40a0e093106" (UID: "0415409e-10f3-4cf1-8bfb-a40a0e093106"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:41:35.520181 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.520125 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-online-tls" (OuterVolumeSpecName: "online-tls") pod "a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" (UID: "a6ab7cda-50f7-45b0-8d7b-33d6ab22809d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:41:35.520359 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.520337 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0415409e-10f3-4cf1-8bfb-a40a0e093106-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0415409e-10f3-4cf1-8bfb-a40a0e093106" (UID: "0415409e-10f3-4cf1-8bfb-a40a0e093106"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:41:35.520447 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.520374 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-kube-api-access-dhcp2" (OuterVolumeSpecName: "kube-api-access-dhcp2") pod "a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" (UID: "a6ab7cda-50f7-45b0-8d7b-33d6ab22809d"). InnerVolumeSpecName "kube-api-access-dhcp2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:41:35.520573 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.520549 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0415409e-10f3-4cf1-8bfb-a40a0e093106-kube-api-access-mth96" (OuterVolumeSpecName: "kube-api-access-mth96") pod "0415409e-10f3-4cf1-8bfb-a40a0e093106" (UID: "0415409e-10f3-4cf1-8bfb-a40a0e093106"). InnerVolumeSpecName "kube-api-access-mth96". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:41:35.520844 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.520829 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" (UID: "a6ab7cda-50f7-45b0-8d7b-33d6ab22809d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 07:41:35.530598 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.530573 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-feast-data" (OuterVolumeSpecName: "feast-data") pod "a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" (UID: "a6ab7cda-50f7-45b0-8d7b-33d6ab22809d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:41:35.618346 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.618253 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-registry-tls\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:41:35.618346 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.618280 2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0415409e-10f3-4cf1-8bfb-a40a0e093106-online-tls\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:41:35.618346 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.618289 2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0415409e-10f3-4cf1-8bfb-a40a0e093106-feast-data\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:41:35.618346 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.618297 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0415409e-10f3-4cf1-8bfb-a40a0e093106-registry-tls\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:41:35.618346 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.618305 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mth96\" (UniqueName: \"kubernetes.io/projected/0415409e-10f3-4cf1-8bfb-a40a0e093106-kube-api-access-mth96\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:41:35.618346 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.618315 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhcp2\" (UniqueName: \"kubernetes.io/projected/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-kube-api-access-dhcp2\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:41:35.618346 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.618343 2563 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-feast-data\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:41:35.618346 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:35.618352 2563 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d-online-tls\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:41:36.011782 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:36.011756 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" Sep 17 07:41:36.012214 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:36.011775 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j" event={"ID":"0415409e-10f3-4cf1-8bfb-a40a0e093106","Type":"ContainerDied","Data":"65cac1f9a23955e0d535ceb906e1c6678d845997df855db6936f94f5d15eddfe"} Sep 17 07:41:36.012214 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:36.011856 2563 scope.go:117] "RemoveContainer" containerID="b5144b3a85ef09f74e9aad9b65b8ad7b57f73877df2e63f40c5100b2365d8a5d" Sep 17 07:41:36.014484 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:36.014453 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" event={"ID":"a6ab7cda-50f7-45b0-8d7b-33d6ab22809d","Type":"ContainerDied","Data":"0caa88c0c2532dd586bd83789176b3be61a1a937f33b01f1cc79a80d8de23a98"} Sep 17 07:41:36.014601 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:36.014507 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj" Sep 17 07:41:36.022580 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:36.022570 2563 scope.go:117] "RemoveContainer" containerID="1a3ba1af967809bb5cc9772feae99ad95692a7ad34d0fbfb1677158c79781531" Sep 17 07:41:36.030654 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:36.030638 2563 scope.go:117] "RemoveContainer" containerID="a6e9e76052a365cb346ce3e78e64e894f1ca2081f0ecd7e1a5dc07eaf4689eb7" Sep 17 07:41:36.037050 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:36.037024 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j"] Sep 17 07:41:36.038896 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:36.038878 2563 scope.go:117] "RemoveContainer" containerID="ba9fe431695de5ffb88807464efa47b659d9300217aad7ef05dbab29d3d22389" Sep 17 07:41:36.041678 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:36.041653 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-597785cc78-wx96j"] Sep 17 07:41:36.046855 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:36.046840 2563 scope.go:117] "RemoveContainer" containerID="9227e3b55b9ad4036dbf7a80f229c2135f26367c466f6ba75fbc34099608005d" Sep 17 07:41:36.054419 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:36.054398 2563 scope.go:117] "RemoveContainer" containerID="454e3e24a8b0d80f72451a0c6727f58aac74a3f99561e9fdf3e468d025f265b4" Sep 17 07:41:36.054535 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:36.054511 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj"] Sep 17 07:41:36.059731 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:36.059711 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-bdbc68465-8vnvj"] Sep 17 07:41:36.062496 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:36.062481 2563 scope.go:117] "RemoveContainer" containerID="db2843e1d730267a8b07694f34469453a9b8986135e5723f57800595aec3371b" Sep 17 07:41:36.069414 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:36.069398 2563 scope.go:117] "RemoveContainer" containerID="e601397e76894f0a6c18ecaabda996d09303fc1842fb3a3aafea73f0d920416f" Sep 17 07:41:37.313403 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:37.313373 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0415409e-10f3-4cf1-8bfb-a40a0e093106" path="/var/lib/kubelet/pods/0415409e-10f3-4cf1-8bfb-a40a0e093106/volumes" Sep 17 07:41:37.313882 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:41:37.313867 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" path="/var/lib/kubelet/pods/a6ab7cda-50f7-45b0-8d7b-33d6ab22809d/volumes" Sep 17 07:53:56.941635 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.941599 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-f8l8p/must-gather-bk2lg"] Sep 17 07:53:56.942038 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.941922 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerName="registry" Sep 17 07:53:56.942038 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.941934 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerName="registry" Sep 17 07:53:56.942038 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.941945 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" containerName="registry" Sep 17 07:53:56.942038 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.941950 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" containerName="registry" Sep 17 07:53:56.942038 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.941958 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerName="online" Sep 17 07:53:56.942038 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.941964 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerName="online" Sep 17 07:53:56.942038 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.941974 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99441f68-8328-4396-b011-b1a3eb720a3c" containerName="postgres" Sep 17 07:53:56.942038 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.941979 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="99441f68-8328-4396-b011-b1a3eb720a3c" containerName="postgres" Sep 17 07:53:56.942038 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.941985 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" containerName="feast-init" Sep 17 07:53:56.942038 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.941989 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" containerName="feast-init" Sep 17 07:53:56.942038 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.941997 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1120c67b-9f79-4b27-b66c-2ad88d220ff6" containerName="redis" Sep 17 07:53:56.942038 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.942003 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1120c67b-9f79-4b27-b66c-2ad88d220ff6" containerName="redis" Sep 17 07:53:56.942038 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.942010 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerName="feast-init" Sep 17 07:53:56.942038 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.942015 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerName="feast-init" Sep 17 07:53:56.942038 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.942020 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerName="feast-apply" Sep 17 07:53:56.942038 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.942025 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerName="feast-apply" Sep 17 07:53:56.942038 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.942033 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" containerName="online" Sep 17 07:53:56.942038 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.942038 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" containerName="online" Sep 17 07:53:56.942038 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.942043 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" containerName="feast-apply" Sep 17 07:53:56.942038 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.942049 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" containerName="feast-apply" Sep 17 07:53:56.942676 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.942057 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96e0c094-1439-4724-bfa0-f0c3c79f0a17" containerName="manager" Sep 17 07:53:56.942676 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.942063 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="96e0c094-1439-4724-bfa0-f0c3c79f0a17" containerName="manager" Sep 17 07:53:56.942676 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.942118 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="99441f68-8328-4396-b011-b1a3eb720a3c" containerName="postgres" Sep 17 07:53:56.942676 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.942126 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" containerName="online" Sep 17 07:53:56.942676 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.942132 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1120c67b-9f79-4b27-b66c-2ad88d220ff6" containerName="redis" Sep 17 07:53:56.942676 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.942137 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerName="registry" Sep 17 07:53:56.942676 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.942142 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0415409e-10f3-4cf1-8bfb-a40a0e093106" containerName="online" Sep 17 07:53:56.942676 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.942148 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="96e0c094-1439-4724-bfa0-f0c3c79f0a17" containerName="manager" Sep 17 07:53:56.942676 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.942153 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6ab7cda-50f7-45b0-8d7b-33d6ab22809d" containerName="registry" Sep 17 07:53:56.944984 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.944970 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f8l8p/must-gather-bk2lg" Sep 17 07:53:56.951499 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.951475 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f8l8p\"/\"openshift-service-ca.crt\"" Sep 17 07:53:56.951631 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.951537 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-f8l8p\"/\"default-dockercfg-27xwg\"" Sep 17 07:53:56.952628 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.952608 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f8l8p\"/\"kube-root-ca.crt\"" Sep 17 07:53:56.963689 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:56.963669 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f8l8p/must-gather-bk2lg"] Sep 17 07:53:57.032486 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:57.032462 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8321522e-1753-4e9b-97f1-3e3d5ffabc38-must-gather-output\") pod \"must-gather-bk2lg\" (UID: \"8321522e-1753-4e9b-97f1-3e3d5ffabc38\") " pod="openshift-must-gather-f8l8p/must-gather-bk2lg" Sep 17 07:53:57.032594 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:57.032531 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5m7l\" (UniqueName: \"kubernetes.io/projected/8321522e-1753-4e9b-97f1-3e3d5ffabc38-kube-api-access-r5m7l\") pod \"must-gather-bk2lg\" (UID: \"8321522e-1753-4e9b-97f1-3e3d5ffabc38\") " pod="openshift-must-gather-f8l8p/must-gather-bk2lg" Sep 17 07:53:57.133731 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:57.133693 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8321522e-1753-4e9b-97f1-3e3d5ffabc38-must-gather-output\") pod \"must-gather-bk2lg\" (UID: \"8321522e-1753-4e9b-97f1-3e3d5ffabc38\") " pod="openshift-must-gather-f8l8p/must-gather-bk2lg" Sep 17 07:53:57.133846 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:57.133756 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5m7l\" (UniqueName: \"kubernetes.io/projected/8321522e-1753-4e9b-97f1-3e3d5ffabc38-kube-api-access-r5m7l\") pod \"must-gather-bk2lg\" (UID: \"8321522e-1753-4e9b-97f1-3e3d5ffabc38\") " pod="openshift-must-gather-f8l8p/must-gather-bk2lg" Sep 17 07:53:57.134049 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:57.134031 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8321522e-1753-4e9b-97f1-3e3d5ffabc38-must-gather-output\") pod \"must-gather-bk2lg\" (UID: \"8321522e-1753-4e9b-97f1-3e3d5ffabc38\") " pod="openshift-must-gather-f8l8p/must-gather-bk2lg" Sep 17 07:53:57.153123 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:57.153102 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5m7l\" (UniqueName: \"kubernetes.io/projected/8321522e-1753-4e9b-97f1-3e3d5ffabc38-kube-api-access-r5m7l\") pod \"must-gather-bk2lg\" (UID: \"8321522e-1753-4e9b-97f1-3e3d5ffabc38\") " pod="openshift-must-gather-f8l8p/must-gather-bk2lg" Sep 17 07:53:57.265608 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:57.265583 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f8l8p/must-gather-bk2lg" Sep 17 07:53:57.538650 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:57.538576 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f8l8p/must-gather-bk2lg"] Sep 17 07:53:57.541622 ip-10-0-141-215 kubenswrapper[2563]: W0917 07:53:57.541593 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8321522e_1753_4e9b_97f1_3e3d5ffabc38.slice/crio-55f02f539ac850e066cf683a30a3c9728716b44edec02829f3dfc90e247f422a WatchSource:0}: Error finding container 55f02f539ac850e066cf683a30a3c9728716b44edec02829f3dfc90e247f422a: Status 404 returned error can't find the container with id 55f02f539ac850e066cf683a30a3c9728716b44edec02829f3dfc90e247f422a Sep 17 07:53:57.543597 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:57.543577 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 07:53:57.956403 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:53:57.956369 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f8l8p/must-gather-bk2lg" event={"ID":"8321522e-1753-4e9b-97f1-3e3d5ffabc38","Type":"ContainerStarted","Data":"55f02f539ac850e066cf683a30a3c9728716b44edec02829f3dfc90e247f422a"} Sep 17 07:54:03.986502 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:03.986466 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f8l8p/must-gather-bk2lg" event={"ID":"8321522e-1753-4e9b-97f1-3e3d5ffabc38","Type":"ContainerStarted","Data":"f54cce0dfed23998fcbece89bcad13a26e4e65d431b4c34c312903f9f46fd55c"} Sep 17 07:54:04.992631 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:04.992598 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f8l8p/must-gather-bk2lg" event={"ID":"8321522e-1753-4e9b-97f1-3e3d5ffabc38","Type":"ContainerStarted","Data":"b9ecaacc5c096dc0cb86c1d16717383b12ac9eec084e62c8adfe42dce1166bc5"} Sep 17 07:54:05.014571 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:05.014520 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-f8l8p/must-gather-bk2lg" podStartSLOduration=2.74035153 podStartE2EDuration="9.01450494s" totalImagesPullingTime="6.27415341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 07:53:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:54:05.013656145 +0000 UTC m=+2052.199389118" watchObservedRunningTime="2026-09-17 07:54:05.01450494 +0000 UTC m=+2052.200237912" Sep 17 07:54:14.033257 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:14.033218 2563 generic.go:356] "Generic (PLEG): container finished" podID="8321522e-1753-4e9b-97f1-3e3d5ffabc38" containerID="f54cce0dfed23998fcbece89bcad13a26e4e65d431b4c34c312903f9f46fd55c" exitCode=0 Sep 17 07:54:14.033812 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:14.033265 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f8l8p/must-gather-bk2lg" event={"ID":"8321522e-1753-4e9b-97f1-3e3d5ffabc38","Type":"ContainerDied","Data":"f54cce0dfed23998fcbece89bcad13a26e4e65d431b4c34c312903f9f46fd55c"} Sep 17 07:54:14.033812 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:14.033639 2563 scope.go:117] "RemoveContainer" containerID="f54cce0dfed23998fcbece89bcad13a26e4e65d431b4c34c312903f9f46fd55c" Sep 17 07:54:14.323062 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:14.322964 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-f8l8p_must-gather-bk2lg_8321522e-1753-4e9b-97f1-3e3d5ffabc38/gather/0.log" Sep 17 07:54:18.116394 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:18.116364 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7f254_c285d69f-53b9-4e2a-a0d0-c72e4d592566/global-pull-secret-syncer/0.log" Sep 17 07:54:18.279794 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:18.279762 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9zrxw_47326f02-5916-4fbe-a826-71c298a22c4b/konnectivity-agent/0.log" Sep 17 07:54:18.386545 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:18.386477 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-215.ec2.internal_910049e72d6e50062f9693db2cfac120/haproxy/0.log" Sep 17 07:54:19.686301 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:19.686258 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-f8l8p/must-gather-bk2lg"] Sep 17 07:54:19.686812 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:19.686475 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-f8l8p/must-gather-bk2lg" podUID="8321522e-1753-4e9b-97f1-3e3d5ffabc38" containerName="copy" containerID="cri-o://b9ecaacc5c096dc0cb86c1d16717383b12ac9eec084e62c8adfe42dce1166bc5" gracePeriod=2 Sep 17 07:54:19.694687 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:19.694653 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-f8l8p/must-gather-bk2lg"] Sep 17 07:54:19.914420 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:19.914403 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-f8l8p_must-gather-bk2lg_8321522e-1753-4e9b-97f1-3e3d5ffabc38/copy/0.log" Sep 17 07:54:19.916082 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:19.916064 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f8l8p/must-gather-bk2lg" Sep 17 07:54:19.921923 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:54:19.921901 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-bk2lg\" is forbidden: User \"system:node:ip-10-0-141-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-f8l8p\": no relationship found between node 'ip-10-0-141-215.ec2.internal' and this object" podUID="8321522e-1753-4e9b-97f1-3e3d5ffabc38" pod="openshift-must-gather-f8l8p/must-gather-bk2lg" Sep 17 07:54:20.019475 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:20.019454 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5m7l\" (UniqueName: \"kubernetes.io/projected/8321522e-1753-4e9b-97f1-3e3d5ffabc38-kube-api-access-r5m7l\") pod \"8321522e-1753-4e9b-97f1-3e3d5ffabc38\" (UID: \"8321522e-1753-4e9b-97f1-3e3d5ffabc38\") " Sep 17 07:54:20.019572 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:20.019516 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8321522e-1753-4e9b-97f1-3e3d5ffabc38-must-gather-output\") pod \"8321522e-1753-4e9b-97f1-3e3d5ffabc38\" (UID: \"8321522e-1753-4e9b-97f1-3e3d5ffabc38\") " Sep 17 07:54:20.019866 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:20.019840 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8321522e-1753-4e9b-97f1-3e3d5ffabc38-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "8321522e-1753-4e9b-97f1-3e3d5ffabc38" (UID: "8321522e-1753-4e9b-97f1-3e3d5ffabc38"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 07:54:20.021800 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:20.021779 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8321522e-1753-4e9b-97f1-3e3d5ffabc38-kube-api-access-r5m7l" (OuterVolumeSpecName: "kube-api-access-r5m7l") pod "8321522e-1753-4e9b-97f1-3e3d5ffabc38" (UID: "8321522e-1753-4e9b-97f1-3e3d5ffabc38"). InnerVolumeSpecName "kube-api-access-r5m7l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 07:54:20.057905 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:20.057886 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-f8l8p_must-gather-bk2lg_8321522e-1753-4e9b-97f1-3e3d5ffabc38/copy/0.log" Sep 17 07:54:20.058232 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:20.058214 2563 generic.go:356] "Generic (PLEG): container finished" podID="8321522e-1753-4e9b-97f1-3e3d5ffabc38" containerID="b9ecaacc5c096dc0cb86c1d16717383b12ac9eec084e62c8adfe42dce1166bc5" exitCode=143 Sep 17 07:54:20.058316 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:20.058265 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f8l8p/must-gather-bk2lg" Sep 17 07:54:20.058316 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:20.058293 2563 scope.go:117] "RemoveContainer" containerID="b9ecaacc5c096dc0cb86c1d16717383b12ac9eec084e62c8adfe42dce1166bc5" Sep 17 07:54:20.066204 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:20.066192 2563 scope.go:117] "RemoveContainer" containerID="f54cce0dfed23998fcbece89bcad13a26e4e65d431b4c34c312903f9f46fd55c" Sep 17 07:54:20.070646 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:54:20.070623 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-bk2lg\" is forbidden: User \"system:node:ip-10-0-141-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-f8l8p\": no relationship found between node 'ip-10-0-141-215.ec2.internal' and this object" podUID="8321522e-1753-4e9b-97f1-3e3d5ffabc38" pod="openshift-must-gather-f8l8p/must-gather-bk2lg" Sep 17 07:54:20.073607 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:54:20.073583 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-bk2lg\" is forbidden: User \"system:node:ip-10-0-141-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-f8l8p\": no relationship found between node 'ip-10-0-141-215.ec2.internal' and this object" podUID="8321522e-1753-4e9b-97f1-3e3d5ffabc38" pod="openshift-must-gather-f8l8p/must-gather-bk2lg" Sep 17 07:54:20.076979 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:20.076960 2563 scope.go:117] "RemoveContainer" containerID="b9ecaacc5c096dc0cb86c1d16717383b12ac9eec084e62c8adfe42dce1166bc5" Sep 17 07:54:20.077231 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:54:20.077215 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9ecaacc5c096dc0cb86c1d16717383b12ac9eec084e62c8adfe42dce1166bc5\": container with ID starting with b9ecaacc5c096dc0cb86c1d16717383b12ac9eec084e62c8adfe42dce1166bc5 not found: ID does not exist" containerID="b9ecaacc5c096dc0cb86c1d16717383b12ac9eec084e62c8adfe42dce1166bc5" Sep 17 07:54:20.077292 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:20.077236 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9ecaacc5c096dc0cb86c1d16717383b12ac9eec084e62c8adfe42dce1166bc5"} err="failed to get container status \"b9ecaacc5c096dc0cb86c1d16717383b12ac9eec084e62c8adfe42dce1166bc5\": rpc error: code = NotFound desc = could not find container \"b9ecaacc5c096dc0cb86c1d16717383b12ac9eec084e62c8adfe42dce1166bc5\": container with ID starting with b9ecaacc5c096dc0cb86c1d16717383b12ac9eec084e62c8adfe42dce1166bc5 not found: ID does not exist" Sep 17 07:54:20.077292 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:20.077250 2563 scope.go:117] "RemoveContainer" containerID="f54cce0dfed23998fcbece89bcad13a26e4e65d431b4c34c312903f9f46fd55c" Sep 17 07:54:20.077496 ip-10-0-141-215 kubenswrapper[2563]: E0917 07:54:20.077478 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f54cce0dfed23998fcbece89bcad13a26e4e65d431b4c34c312903f9f46fd55c\": container with ID starting with f54cce0dfed23998fcbece89bcad13a26e4e65d431b4c34c312903f9f46fd55c not found: ID does not exist" containerID="f54cce0dfed23998fcbece89bcad13a26e4e65d431b4c34c312903f9f46fd55c" Sep 17 07:54:20.077554 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:20.077506 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f54cce0dfed23998fcbece89bcad13a26e4e65d431b4c34c312903f9f46fd55c"} err="failed to get container status \"f54cce0dfed23998fcbece89bcad13a26e4e65d431b4c34c312903f9f46fd55c\": rpc error: code = NotFound desc = could not find container \"f54cce0dfed23998fcbece89bcad13a26e4e65d431b4c34c312903f9f46fd55c\": container with ID starting with f54cce0dfed23998fcbece89bcad13a26e4e65d431b4c34c312903f9f46fd55c not found: ID does not exist" Sep 17 07:54:20.120086 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:20.120062 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5m7l\" (UniqueName: \"kubernetes.io/projected/8321522e-1753-4e9b-97f1-3e3d5ffabc38-kube-api-access-r5m7l\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:54:20.120086 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:20.120085 2563 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8321522e-1753-4e9b-97f1-3e3d5ffabc38-must-gather-output\") on node \"ip-10-0-141-215.ec2.internal\" DevicePath \"\"" Sep 17 07:54:21.314037 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:21.314001 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8321522e-1753-4e9b-97f1-3e3d5ffabc38" path="/var/lib/kubelet/pods/8321522e-1753-4e9b-97f1-3e3d5ffabc38/volumes" Sep 17 07:54:22.216742 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:22.216707 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-b4rsw_d07236c4-ae38-4ffc-b1ca-288072181c8d/cluster-monitoring-operator/0.log" Sep 17 07:54:22.274260 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:22.274228 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-w2d89_52adbf5f-ede1-4f89-a64e-3afe3078f954/kube-state-metrics/0.log" Sep 17 07:54:22.315192 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:22.315162 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-w2d89_52adbf5f-ede1-4f89-a64e-3afe3078f954/kube-rbac-proxy-main/0.log" Sep 17 07:54:22.379250 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:22.379217 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-w2d89_52adbf5f-ede1-4f89-a64e-3afe3078f954/kube-rbac-proxy-self/0.log" Sep 17 07:54:22.440053 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:22.440037 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-558649d76f-rfrbj_a25ca0a8-968b-4ef8-bba5-2a975e3c96cd/metrics-server/0.log" Sep 17 07:54:22.654752 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:22.654688 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hcrn9_8465f6f8-337f-44a6-9277-bb24929f2158/node-exporter/0.log" Sep 17 07:54:22.695063 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:22.695047 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hcrn9_8465f6f8-337f-44a6-9277-bb24929f2158/kube-rbac-proxy/0.log" Sep 17 07:54:22.735774 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:22.735756 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hcrn9_8465f6f8-337f-44a6-9277-bb24929f2158/init-textfile/0.log" Sep 17 07:54:23.504713 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:23.504681 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-whl5j_37cc8e33-52ac-48a9-b2df-58b931c58dc2/prometheus-operator/0.log" Sep 17 07:54:23.542651 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:23.542629 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-whl5j_37cc8e33-52ac-48a9-b2df-58b931c58dc2/kube-rbac-proxy/0.log" Sep 17 07:54:23.637831 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:23.637805 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-766fc79c5-qqd9l_08d633b1-b5be-41fc-a04d-ccaaa3e8fad6/telemeter-client/0.log" Sep 17 07:54:23.670988 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:23.670967 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-766fc79c5-qqd9l_08d633b1-b5be-41fc-a04d-ccaaa3e8fad6/reload/0.log" Sep 17 07:54:23.699363 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:23.699339 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-766fc79c5-qqd9l_08d633b1-b5be-41fc-a04d-ccaaa3e8fad6/kube-rbac-proxy/0.log" Sep 17 07:54:23.744200 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:23.744179 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d97b757c9-zmgln_879a10c5-3d92-4906-a0cf-84f104ce2a25/thanos-query/0.log" Sep 17 07:54:23.783075 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:23.783012 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d97b757c9-zmgln_879a10c5-3d92-4906-a0cf-84f104ce2a25/kube-rbac-proxy-web/0.log" Sep 17 07:54:23.834509 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:23.834488 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d97b757c9-zmgln_879a10c5-3d92-4906-a0cf-84f104ce2a25/kube-rbac-proxy/0.log" Sep 17 07:54:23.911171 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:23.911153 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d97b757c9-zmgln_879a10c5-3d92-4906-a0cf-84f104ce2a25/prom-label-proxy/0.log" Sep 17 07:54:23.954863 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:23.954847 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d97b757c9-zmgln_879a10c5-3d92-4906-a0cf-84f104ce2a25/kube-rbac-proxy-rules/0.log" Sep 17 07:54:23.984510 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:23.984492 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d97b757c9-zmgln_879a10c5-3d92-4906-a0cf-84f104ce2a25/kube-rbac-proxy-metrics/0.log" Sep 17 07:54:24.858157 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:24.858124 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt"] Sep 17 07:54:24.858542 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:24.858507 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8321522e-1753-4e9b-97f1-3e3d5ffabc38" containerName="gather" Sep 17 07:54:24.858542 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:24.858521 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8321522e-1753-4e9b-97f1-3e3d5ffabc38" containerName="gather" Sep 17 07:54:24.858542 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:24.858540 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8321522e-1753-4e9b-97f1-3e3d5ffabc38" containerName="copy" Sep 17 07:54:24.858642 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:24.858546 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8321522e-1753-4e9b-97f1-3e3d5ffabc38" containerName="copy" Sep 17 07:54:24.858642 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:24.858603 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8321522e-1753-4e9b-97f1-3e3d5ffabc38" containerName="copy" Sep 17 07:54:24.858642 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:24.858614 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8321522e-1753-4e9b-97f1-3e3d5ffabc38" containerName="gather" Sep 17 07:54:24.862193 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:24.862173 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt" Sep 17 07:54:24.866527 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:24.866507 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xsbln\"/\"default-dockercfg-fpw4b\"" Sep 17 07:54:24.867670 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:24.867647 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xsbln\"/\"kube-root-ca.crt\"" Sep 17 07:54:24.867776 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:24.867652 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xsbln\"/\"openshift-service-ca.crt\"" Sep 17 07:54:24.872706 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:24.872684 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt"] Sep 17 07:54:24.958842 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:24.958810 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0ece2126-2407-4016-9432-b757bf96122d-proc\") pod \"perf-node-gather-daemonset-lmmrt\" (UID: \"0ece2126-2407-4016-9432-b757bf96122d\") " pod="openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt" Sep 17 07:54:24.958985 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:24.958850 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0ece2126-2407-4016-9432-b757bf96122d-podres\") pod \"perf-node-gather-daemonset-lmmrt\" (UID: \"0ece2126-2407-4016-9432-b757bf96122d\") " pod="openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt" Sep 17 07:54:24.958985 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:24.958940 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsjln\" (UniqueName: \"kubernetes.io/projected/0ece2126-2407-4016-9432-b757bf96122d-kube-api-access-jsjln\") pod \"perf-node-gather-daemonset-lmmrt\" (UID: \"0ece2126-2407-4016-9432-b757bf96122d\") " pod="openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt" Sep 17 07:54:24.958985 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:24.958978 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ece2126-2407-4016-9432-b757bf96122d-sys\") pod \"perf-node-gather-daemonset-lmmrt\" (UID: \"0ece2126-2407-4016-9432-b757bf96122d\") " pod="openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt" Sep 17 07:54:24.959108 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:24.959004 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0ece2126-2407-4016-9432-b757bf96122d-lib-modules\") pod \"perf-node-gather-daemonset-lmmrt\" (UID: \"0ece2126-2407-4016-9432-b757bf96122d\") " pod="openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt" Sep 17 07:54:25.049461 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:25.049441 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-cqgql_8de810bd-a229-40bc-b8e5-9d02eeb9a501/networking-console-plugin/0.log" Sep 17 07:54:25.059465 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:25.059438 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0ece2126-2407-4016-9432-b757bf96122d-podres\") pod \"perf-node-gather-daemonset-lmmrt\" (UID: \"0ece2126-2407-4016-9432-b757bf96122d\") " pod="openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt" Sep 17 07:54:25.059561 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:25.059510 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsjln\" (UniqueName: \"kubernetes.io/projected/0ece2126-2407-4016-9432-b757bf96122d-kube-api-access-jsjln\") pod \"perf-node-gather-daemonset-lmmrt\" (UID: \"0ece2126-2407-4016-9432-b757bf96122d\") " pod="openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt" Sep 17 07:54:25.059561 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:25.059537 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ece2126-2407-4016-9432-b757bf96122d-sys\") pod \"perf-node-gather-daemonset-lmmrt\" (UID: \"0ece2126-2407-4016-9432-b757bf96122d\") " pod="openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt" Sep 17 07:54:25.059561 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:25.059557 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0ece2126-2407-4016-9432-b757bf96122d-lib-modules\") pod \"perf-node-gather-daemonset-lmmrt\" (UID: \"0ece2126-2407-4016-9432-b757bf96122d\") " pod="openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt" Sep 17 07:54:25.059701 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:25.059580 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0ece2126-2407-4016-9432-b757bf96122d-proc\") pod \"perf-node-gather-daemonset-lmmrt\" (UID: \"0ece2126-2407-4016-9432-b757bf96122d\") " pod="openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt" Sep 17 07:54:25.059701 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:25.059594 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0ece2126-2407-4016-9432-b757bf96122d-podres\") pod \"perf-node-gather-daemonset-lmmrt\" (UID: \"0ece2126-2407-4016-9432-b757bf96122d\") " pod="openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt" Sep 17 07:54:25.059701 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:25.059637 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0ece2126-2407-4016-9432-b757bf96122d-sys\") pod \"perf-node-gather-daemonset-lmmrt\" (UID: \"0ece2126-2407-4016-9432-b757bf96122d\") " pod="openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt" Sep 17 07:54:25.059819 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:25.059700 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0ece2126-2407-4016-9432-b757bf96122d-proc\") pod \"perf-node-gather-daemonset-lmmrt\" (UID: \"0ece2126-2407-4016-9432-b757bf96122d\") " pod="openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt" Sep 17 07:54:25.059819 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:25.059723 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0ece2126-2407-4016-9432-b757bf96122d-lib-modules\") pod \"perf-node-gather-daemonset-lmmrt\" (UID: \"0ece2126-2407-4016-9432-b757bf96122d\") " pod="openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt" Sep 17 07:54:25.073169 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:25.073147 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsjln\" (UniqueName: \"kubernetes.io/projected/0ece2126-2407-4016-9432-b757bf96122d-kube-api-access-jsjln\") pod \"perf-node-gather-daemonset-lmmrt\" (UID: \"0ece2126-2407-4016-9432-b757bf96122d\") " pod="openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt" Sep 17 07:54:25.172466 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:25.172413 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt" Sep 17 07:54:25.445867 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:25.445844 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt"] Sep 17 07:54:25.894705 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:25.894671 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-57c46cc9c7-dfmx4_a7791fe6-9290-4cb6-b8ce-6280bfd843d6/console/0.log" Sep 17 07:54:26.084243 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:26.084208 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt" event={"ID":"0ece2126-2407-4016-9432-b757bf96122d","Type":"ContainerStarted","Data":"73259d6e1c589cb5b8c088b2af471e36af4e8c4db8d5bcfbb816cd854fc13964"} Sep 17 07:54:26.084423 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:26.084291 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt" event={"ID":"0ece2126-2407-4016-9432-b757bf96122d","Type":"ContainerStarted","Data":"7205b9b52362a84b4191026b5a1454b1b0f4c676ebaf480b419db1e50fac857f"} Sep 17 07:54:26.084423 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:26.084364 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt" Sep 17 07:54:26.112941 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:26.112899 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt" podStartSLOduration=2.112877755 podStartE2EDuration="2.112877755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 07:54:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 07:54:26.111706564 +0000 UTC m=+2073.297439535" watchObservedRunningTime="2026-09-17 07:54:26.112877755 +0000 UTC m=+2073.298610770" Sep 17 07:54:26.429252 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:26.429222 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-sjghs_cd69a0d7-2a67-4f2e-9331-10d0874d1063/volume-data-source-validator/0.log" Sep 17 07:54:27.424106 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:27.424076 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tr629_f3305365-b6fc-4a79-97c8-63209d1d71ea/dns/0.log" Sep 17 07:54:27.470938 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:27.470914 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tr629_f3305365-b6fc-4a79-97c8-63209d1d71ea/kube-rbac-proxy/0.log" Sep 17 07:54:27.665822 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:27.665787 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tnmcf_9c0e9d0a-00c1-479b-a811-68a9ca231e9f/dns-node-resolver/0.log" Sep 17 07:54:28.188550 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:28.188521 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7685554dd6-cwqw2_8d8627ac-e7f3-41f9-bbc4-1a09bd285ddc/registry/0.log" Sep 17 07:54:28.216725 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:28.216703 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-k5dtg_ddbb72ee-6e56-4549-ac75-f8857a377394/node-ca/0.log" Sep 17 07:54:29.444317 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:29.444285 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hll2c_79621566-efae-43dd-9386-30a4a2aed04b/serve-healthcheck-canary/0.log" Sep 17 07:54:29.811681 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:29.811651 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-fwkfn_59c7192a-8c88-4409-ba0a-811cccbc2112/insights-operator/0.log" Sep 17 07:54:29.811979 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:29.811965 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-fwkfn_59c7192a-8c88-4409-ba0a-811cccbc2112/insights-operator/1.log" Sep 17 07:54:29.833418 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:29.833399 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lzhlj_552cd700-3fb9-4411-9eda-3374a359f156/kube-rbac-proxy/0.log" Sep 17 07:54:29.856234 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:29.856219 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lzhlj_552cd700-3fb9-4411-9eda-3374a359f156/exporter/0.log" Sep 17 07:54:29.879887 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:29.879867 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lzhlj_552cd700-3fb9-4411-9eda-3374a359f156/extractor/0.log" Sep 17 07:54:32.098712 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:32.098688 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xsbln/perf-node-gather-daemonset-lmmrt" Sep 17 07:54:35.063985 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:35.063948 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-4gxzj_b4e5f925-07d2-41f5-b94b-f275630921ed/kube-storage-version-migrator-operator/1.log" Sep 17 07:54:35.064801 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:35.064780 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-4gxzj_b4e5f925-07d2-41f5-b94b-f275630921ed/kube-storage-version-migrator-operator/0.log" Sep 17 07:54:36.020501 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:36.020473 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2krnr_fd894e06-a948-4955-a07a-dec7e32ca8b0/kube-multus-additional-cni-plugins/0.log" Sep 17 07:54:36.051890 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:36.051869 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2krnr_fd894e06-a948-4955-a07a-dec7e32ca8b0/egress-router-binary-copy/0.log" Sep 17 07:54:36.077423 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:36.077405 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2krnr_fd894e06-a948-4955-a07a-dec7e32ca8b0/cni-plugins/0.log" Sep 17 07:54:36.106788 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:36.106741 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2krnr_fd894e06-a948-4955-a07a-dec7e32ca8b0/bond-cni-plugin/0.log" Sep 17 07:54:36.133751 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:36.133730 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2krnr_fd894e06-a948-4955-a07a-dec7e32ca8b0/routeoverride-cni/0.log" Sep 17 07:54:36.160892 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:36.160871 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2krnr_fd894e06-a948-4955-a07a-dec7e32ca8b0/whereabouts-cni-bincopy/0.log" Sep 17 07:54:36.184678 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:36.184656 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2krnr_fd894e06-a948-4955-a07a-dec7e32ca8b0/whereabouts-cni/0.log" Sep 17 07:54:36.605671 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:36.605644 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-rwlkw_0f66f8d4-3fa1-461b-8925-f68c138aafd2/kube-multus/0.log" Sep 17 07:54:36.729556 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:36.729531 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-x7484_69c30c82-b237-4fc3-a96c-61840e73b7a7/network-metrics-daemon/0.log" Sep 17 07:54:36.756023 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:36.755999 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-x7484_69c30c82-b237-4fc3-a96c-61840e73b7a7/kube-rbac-proxy/0.log" Sep 17 07:54:37.931963 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:37.931932 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-68n7z_5861ec16-ae09-4421-a393-fe23e69fa0cd/ovn-controller/0.log" Sep 17 07:54:37.971367 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:37.971312 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-68n7z_5861ec16-ae09-4421-a393-fe23e69fa0cd/ovn-acl-logging/0.log" Sep 17 07:54:38.001729 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:38.001705 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-68n7z_5861ec16-ae09-4421-a393-fe23e69fa0cd/kube-rbac-proxy-node/0.log" Sep 17 07:54:38.024463 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:38.024439 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-68n7z_5861ec16-ae09-4421-a393-fe23e69fa0cd/kube-rbac-proxy-ovn-metrics/0.log" Sep 17 07:54:38.045306 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:38.045283 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-68n7z_5861ec16-ae09-4421-a393-fe23e69fa0cd/northd/0.log" Sep 17 07:54:38.083187 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:38.083174 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-68n7z_5861ec16-ae09-4421-a393-fe23e69fa0cd/nbdb/0.log" Sep 17 07:54:38.108317 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:38.108299 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-68n7z_5861ec16-ae09-4421-a393-fe23e69fa0cd/sbdb/0.log" Sep 17 07:54:38.155943 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:38.155921 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-68n7z_5861ec16-ae09-4421-a393-fe23e69fa0cd/ovnkube-controller/0.log" Sep 17 07:54:39.373512 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:39.373480 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-m6v2q_f5733389-a2ac-47dd-9fb4-8ba44621adb9/network-check-target-container/0.log" Sep 17 07:54:40.352467 ip-10-0-141-215 kubenswrapper[2563]: I0917 07:54:40.352442 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-b42pw_197c1697-1d3f-414c-bc8d-5919174461cd/iptables-alerter/0.log"